2024-05-10T10:17:48.101 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T10:17:48.102 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} --archive /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904 --owner scheduled_dparmar@teuthology --name dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi -- /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/orig.config.yaml 2024-05-10T10:17:48.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T10:17:48.255 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904 branch: wip-63896-from-wip-rishabh-testing-20240501.193033 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7700904' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-63896-from-wip-rishabh-testing-20240501.193033 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d39fa08867836ac070c47cb66c782fa559fd480b subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: d39fa08867836ac070c47cb66c782fa559fd480b kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-63896-from-wip-rishabh-testing-20240501.193033 sha1: d39fa08867836ac070c47cb66c782fa559fd480b owner: scheduled_dparmar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1698 sha1: d39fa08867836ac070c47cb66c782fa559fd480b sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-63896-from-wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d39fa08867836ac070c47cb66c782fa559fd480b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCcNlzza/Wth97WXmsA16V6N0RanwyKC1rwpVsJpKJlklIvO5iXt6PS9+aaMjV2oJt7RwoHC38k+XGAbNE+0rc= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAKMvqwK2l+mQbHzOhMHA4sdBAWjsxoMVsjPP/cSzda6NBCOlbJnNsSuMj4o5dAuvQV7UhS94n80ApSnOQMit2I= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIvswHVDPfo+n1QAdxqgSrGktq16rQECYi/memjoqFAv2uM7Nj0kV01C9uJgq2GBwJymI7g2SA4qtqNOWRdDApc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_07:35:43 tube: smithi user: dparmar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T10:17:48.255 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa; will attempt to use it 2024-05-10T10:17:48.256 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks 2024-05-10T10:17:48.256 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T10:17:48.258 INFO:teuthology.task.internal:Checking packages... 2024-05-10T10:17:48.283 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd39fa08867836ac070c47cb66c782fa559fd480b' 2024-05-10T10:17:48.284 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T10:17:48.284 INFO:teuthology.packaging:ref: None 2024-05-10T10:17:48.284 INFO:teuthology.packaging:tag: None 2024-05-10T10:17:48.284 INFO:teuthology.packaging:branch: wip-63896-from-wip-rishabh-testing-20240501.193033 2024-05-10T10:17:48.284 INFO:teuthology.packaging:sha1: d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:17:48.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-63896-from-wip-rishabh-testing-20240501.193033 2024-05-10T10:17:48.431 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3449.gd39fa088 2024-05-10T10:17:48.432 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T10:17:48.438 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T10:17:48.438 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T10:17:48.526 INFO:teuthology.task.internal:Saving configuration 2024-05-10T10:17:48.546 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T10:17:48.551 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:17:48.577 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:12:39.721000', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCcNlzza/Wth97WXmsA16V6N0RanwyKC1rwpVsJpKJlklIvO5iXt6PS9+aaMjV2oJt7RwoHC38k+XGAbNE+0rc='} 2024-05-10T10:17:48.602 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:12:39.722898', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAKMvqwK2l+mQbHzOhMHA4sdBAWjsxoMVsjPP/cSzda6NBCOlbJnNsSuMj4o5dAuvQV7UhS94n80ApSnOQMit2I='} 2024-05-10T10:17:48.622 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:12:39.724592', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIvswHVDPfo+n1QAdxqgSrGktq16rQECYi/memjoqFAv2uM7Nj0kV01C9uJgq2GBwJymI7g2SA4qtqNOWRdDApc='} 2024-05-10T10:17:48.622 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T10:17:48.627 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-10T10:17:48.627 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-10T10:17:48.628 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-10T10:17:48.628 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T10:17:48.778 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0f5344c0d0>, signals=[15]) 2024-05-10T10:17:48.779 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T10:17:48.785 INFO:teuthology.task.internal:Opening connections... 2024-05-10T10:17:48.785 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:17:48.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:17:48.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-05-10T10:17:48.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:17:48.950 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-05-10T10:17:48.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:17:49.030 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T10:17:49.037 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-05-10T10:17:49.055 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-05-10T10:17:49.056 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-05-10T10:17:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:17:49.113 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:17:49.113 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:17:49.113 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:17:49.113 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:17:49.113 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:17:49.113 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-05-10T10:17:49.137 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-05-10T10:17:49.156 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-05-10T10:17:49.156 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-05-10T10:17:49.214 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-05-10T10:17:49.214 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:17:49.215 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:17:49.216 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-05-10T10:17:49.237 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-05-10T10:17:49.255 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-05-10T10:17:49.255 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-05-10T10:17:49.312 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-05-10T10:17:49.312 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-05-10T10:17:49.312 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-05-10T10:17:49.312 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:17:49.312 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-05-10T10:17:49.312 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:17:49.312 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:17:49.313 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-05-10T10:17:49.313 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:17:49.313 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:17:49.313 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:17:49.313 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:17:49.313 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:17:49.313 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:17:49.313 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-05-10T10:17:49.337 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T10:17:49.347 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T10:17:49.352 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T10:17:49.353 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:17:49.355 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:17:49.358 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:17:49.375 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T10:17:49.381 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T10:17:49.381 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:17:49.413 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:17:49.415 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:17:49.536 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T10:17:49.608 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T10:17:49.623 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-05-10T10:17:49.623 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-05-10T10:17:49.623 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:17:49.624 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:17:49.624 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:17:49.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:17:49.643 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-05-10T10:17:49.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:17:49.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:17:49.644 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-05-10T10:17:49.645 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-05-10T10:17:49.699 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:17:49.699 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.14.0-437.el9.x86_64 2024-05-10T10:17:49.700 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum install -y kernel 2024-05-10T10:17:49.701 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:17:49.701 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-437.el9.x86_64 2024-05-10T10:17:49.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-05-10T10:17:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:17:49.703 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-437.el9.x86_64 2024-05-10T10:17:49.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-05-10T10:17:52.004 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-10T10:17:52.238 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-10T10:17:52.275 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.1 MB 00:00 2024-05-10T10:17:54.000 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-10T10:17:54.838 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-10T10:17:57.355 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 19 MB 00:04 2024-05-10T10:17:58.651 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-05-10T10:17:59.756 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-05-10T10:18:00.452 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-10T10:18:01.357 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-10T10:18:01.597 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.2 MB 00:00 2024-05-10T10:18:01.964 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-10T10:18:02.666 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-10T10:18:03.365 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-10T10:18:04.408 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-10T10:18:07.128 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-10T10:18:07.808 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-05-10T10:18:09.143 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:18:09.143 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:18:09.232 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-05-10T10:18:09.240 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-05-10T10:18:09.240 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-05-10T10:18:09.240 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-05-10T10:18:09.240 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-05-10T10:18:09.240 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-05-10T10:18:09.241 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:18:09.242 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 78 M 2024-05-10T10:18:09.242 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 123 M 2024-05-10T10:18:09.242 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-05-10T10:18:09.600 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-10T10:18:09.813 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.6 MB/s | 484 kB 00:00 2024-05-10T10:18:09.814 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:18:09.814 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:18:09.912 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-05-10T10:18:09.917 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-05-10T10:18:09.917 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-05-10T10:18:09.917 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-05-10T10:18:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:18:09.919 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 78 M 2024-05-10T10:18:09.919 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 123 M 2024-05-10T10:18:09.919 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-05-10T10:18:11.146 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 9.5 MB/s | 15 MB 00:01 2024-05-10T10:18:11.260 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 410 kB/s | 484 kB 00:01 2024-05-10T10:18:11.622 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:18:11.623 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:18:11.727 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-05-10T10:18:11.743 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-05-10T10:18:11.744 INFO:teuthology.orchestra.run.smithi195.stdout:Install 4 Packages 2024-05-10T10:18:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:18:11.744 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 78 M 2024-05-10T10:18:11.744 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 123 M 2024-05-10T10:18:11.744 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-05-10T10:18:11.901 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 8.5 MB/s | 15 MB 00:01 2024-05-10T10:18:12.330 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-10T10:18:12.693 INFO:teuthology.orchestra.run.smithi195.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 533 kB/s | 484 kB 00:00 2024-05-10T10:18:12.710 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 13 MB/s | 34 MB 00:02 2024-05-10T10:18:13.584 INFO:teuthology.orchestra.run.smithi195.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 8.6 MB/s | 15 MB 00:01 2024-05-10T10:18:14.044 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 10 MB/s | 28 MB 00:02 2024-05-10T10:18:14.044 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:18:14.044 INFO:teuthology.orchestra.run.smithi112.stdout:Total 19 MB/s | 78 MB 00:04 2024-05-10T10:18:14.343 INFO:teuthology.orchestra.run.smithi195.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 13 MB/s | 34 MB 00:02 2024-05-10T10:18:14.728 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-05-10T10:18:15.070 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-05-10T10:18:15.071 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-05-10T10:18:15.143 INFO:teuthology.orchestra.run.smithi195.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 11 MB/s | 28 MB 00:02 2024-05-10T10:18:15.143 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:18:15.143 INFO:teuthology.orchestra.run.smithi195.stdout:Total 23 MB/s | 78 MB 00:03 2024-05-10T10:18:15.822 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-05-10T10:18:16.028 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-05-10T10:18:16.028 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-05-10T10:18:16.167 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-05-10T10:18:16.167 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-05-10T10:18:16.321 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 4.3 MB/s | 28 MB 00:06 2024-05-10T10:18:16.321 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:18:16.322 INFO:teuthology.orchestra.run.smithi148.stdout:Total 11 MB/s | 78 MB 00:07 2024-05-10T10:18:17.006 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-05-10T10:18:17.095 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-05-10T10:18:17.095 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-05-10T10:18:17.346 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-05-10T10:18:17.346 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-05-10T10:18:18.305 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-05-10T10:18:18.305 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-05-10T10:18:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-05-10T10:18:19.159 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:18:19.244 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:18:19.502 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-05-10T10:18:20.102 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:18:20.185 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:18:20.756 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-05-10T10:18:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:18:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:18:21.905 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:18:22.040 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:18:22.855 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:18:23.005 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:18:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:18:24.314 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:18:26.753 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:18:26.793 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:18:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:18:27.741 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:18:29.058 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:18:29.183 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:18:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:18:32.467 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:18:33.934 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:01.163 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:01.201 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:01.519 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:01.519 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:19:01.519 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:19:01.519 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:19:01.931 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:01.970 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:02.279 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:02.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:19:02.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:19:02.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:19:02.299 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:02.299 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:19:02.299 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-05-10T10:19:02.300 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:02.300 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:19:02.300 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:19:02.300 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:19:02.300 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:19:02.300 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-05-10T10:19:02.662 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:19:02.741 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:02.742 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:19:02.742 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:19:02.742 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:19:02.742 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:19:02.742 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-10T10:19:02.742 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:19:02.742 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:19:02.792 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:02.792 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:19:02.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-05-10T10:19:02.833 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 68.el9 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8264254 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:19:02.834 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:19:02.835 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:19:02.836 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:19:02.836 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:19:02.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:19:03.081 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-05-10T10:19:03.214 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:03.252 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:03.422 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:19:03.493 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:03.494 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:19:03.494 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:19:03.494 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:19:03.494 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:19:03.494 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-10T10:19:03.494 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:19:03.494 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:19:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-05-10T10:19:03.544 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:03.544 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:19:03.544 DEBUG:teuthology.orchestra.run.smithi195:> sudo rpm -qi grub2-tools 2024-05-10T10:19:03.574 INFO:teuthology.orchestra.run.smithi195.stdout:Name : grub2-tools 2024-05-10T10:19:03.574 INFO:teuthology.orchestra.run.smithi195.stdout:Epoch : 1 2024-05-10T10:19:03.574 INFO:teuthology.orchestra.run.smithi195.stdout:Version : 2.06 2024-05-10T10:19:03.574 INFO:teuthology.orchestra.run.smithi195.stdout:Release : 68.el9 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: x86_64 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Group : Unspecified 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Size : 8264254 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:License : GPLv3+ 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Packager : builder@centos.org 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Vendor : CentOS 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Summary : Support tools for GRUB. 2024-05-10T10:19:03.575 INFO:teuthology.orchestra.run.smithi195.stdout:Description : 2024-05-10T10:19:03.576 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:19:03.576 INFO:teuthology.orchestra.run.smithi195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:19:03.576 INFO:teuthology.orchestra.run.smithi195.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:19:03.576 INFO:teuthology.orchestra.run.smithi195.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:19:03.576 INFO:teuthology.orchestra.run.smithi195.stdout:hardware devices. 2024-05-10T10:19:03.576 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:19:03.576 INFO:teuthology.orchestra.run.smithi195.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:19:03.577 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:19:03.577 INFO:teuthology.task.kernel:Updating grub on smithi195 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:19:03.577 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:19:03.609 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:03.609 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:19:03.609 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:19:03.609 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:19:04.189 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-05-10T10:19:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:19:04.413 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-05-10T10:19:04.817 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:19:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:19:04.910 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:04.911 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:19:04.911 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:19:04.911 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:19:04.911 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:19:04.911 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-10T10:19:04.911 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:19:04.911 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:19:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-05-10T10:19:04.925 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-05-10T10:19:04.941 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.MC2xsqnu58 2024-05-10T10:19:04.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MC2xsqnu58 2024-05-10T10:19:04.963 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:19:04.963 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:19:04.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 68.el9 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8264254 2024-05-10T10:19:05.001 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:19:05.002 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-05-10T10:19:05.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:19:05.003 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:19:05.003 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:19:05.004 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:19:05.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:19:05.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.MC2xsqnu58 2024-05-10T10:19:05.155 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.MC2xsqnu58 is 6KB 2024-05-10T10:19:05.205 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.MC2xsqnu58 2024-05-10T10:19:05.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:19:05.281 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:19:05.281 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:19:05.281 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:19:05.282 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:19:05.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:19:05.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-05-10T10:19:05.635 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-05-10T10:19:05.635 INFO:teuthology.orchestra.run.smithi195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:19:05.674 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-05-10T10:19:05.675 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-05-10T10:19:05.691 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.t91uv8pbPL 2024-05-10T10:19:05.691 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.t91uv8pbPL 2024-05-10T10:19:05.757 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.t91uv8pbPL 2024-05-10T10:19:05.893 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.t91uv8pbPL is 6KB 2024-05-10T10:19:05.942 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.t91uv8pbPL 2024-05-10T10:19:05.956 DEBUG:teuthology.orchestra.run.smithi195:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:19:06.017 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:19:06.018 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:19:06.018 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:19:06.018 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:19:06.019 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:19:06.161 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-05-10T10:19:07.017 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:19:07.043 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-05-10T10:19:07.045 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-05-10T10:19:07.075 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.Zwrw85P6gj 2024-05-10T10:19:07.075 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Zwrw85P6gj 2024-05-10T10:19:07.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.Zwrw85P6gj 2024-05-10T10:19:07.271 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.Zwrw85P6gj is 6KB 2024-05-10T10:19:07.321 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.Zwrw85P6gj 2024-05-10T10:19:07.335 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:19:07.397 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:19:07.397 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:19:07.397 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:19:07.397 INFO:teuthology.orchestra.run.smithi148.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:19:07.398 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:19:07.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-05-10T10:19:35.456 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-10T10:19:35.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-10T10:19:35.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:19:36.165 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-10T10:19:36.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-10T10:19:36.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:19:37.541 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-10T10:19:37.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-05-10T10:19:37.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:20:12.191 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-05-10T10:20:21.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-05-10T10:20:21.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:20:35.474 DEBUG:teuthology.orchestra.remote:timed out 2024-05-10T10:20:36.169 DEBUG:teuthology.orchestra.remote:timed out 2024-05-10T10:20:39.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-05-10T10:20:44.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-10T10:20:44.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:20:44.952 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-05-10T10:20:45.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-10T10:20:45.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:20:45.365 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-10T10:20:45.365 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:20:45.365 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-05-10T10:20:45.382 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:20:45.382 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:20:45.382 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:20:45.382 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:20:46.383 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-10T10:20:46.383 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-10T10:20:46.454 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2024-05-10T10:20:46.502 DEBUG:teuthology.parallel:result is None 2024-05-10T10:20:48.867 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-05-10T10:20:49.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-10T10:20:49.265 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:20:49.266 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-05-10T10:20:49.280 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:20:49.280 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:20:49.280 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:20:49.280 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:20:50.281 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-10T10:20:50.282 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-10T10:20:50.362 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-05-10T10:20:50.401 DEBUG:teuthology.parallel:result is None 2024-05-10T10:20:51.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-05-10T10:20:51.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:20:52.182 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-05-10T10:20:52.684 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-05-10T10:20:52.684 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:20:52.685 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-05-10T10:20:52.700 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:20:52.701 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:20:52.701 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:20:52.701 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:20:53.702 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-10T10:20:53.702 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-10T10:20:53.767 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-05-10T10:20:53.807 DEBUG:teuthology.parallel:result is None 2024-05-10T10:20:53.808 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T10:20:53.815 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T10:20:53.816 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:20:53.819 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:20:53.821 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:20:53.837 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T10:20:53.843 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T10:20:53.851 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T10:20:53.851 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:20:53.878 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:20:53.880 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:20:54.215 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T10:20:54.221 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T10:20:54.222 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:20:54.224 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:20:54.227 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:20:54.253 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:20:54.256 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:20:54.265 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:20:54.267 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:20:54.267 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:20:54.278 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:20:54.279 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T10:20:54.286 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T10:20:54.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:20:54.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:20:54.322 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:20:54.346 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T10:20:54.355 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T10:20:54.355 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:20:54.379 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:20:54.387 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:20:54.401 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:20:54.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:20:54.531 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:20:54.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:20:54.658 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:20:54.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:20:54.724 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:20:54.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:20:54.816 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:20:54.938 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:20:54.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:20:55.004 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:20:55.055 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:20:55.115 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:20:55.208 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:20:55.264 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:20:55.264 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:20:55.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-05-10T10:20:55.350 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-05-10T10:20:55.352 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-05-10T10:20:55.386 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:20:55.411 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:20:55.458 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:20:55.880 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T10:20:55.888 INFO:teuthology.task.internal:Starting timer... 2024-05-10T10:20:55.888 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T10:20:55.909 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T10:20:55.918 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-10T10:20:55.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-05-10T10:20:56.014 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-05-10T10:20:56.015 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-05-10T10:20:56.067 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-05-10T10:20:56.069 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-05-10T10:20:56.190 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-05-10T10:20:56.192 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T10:20:56.192 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-05-10T10:20:56.212 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-05-10T10:20:56.213 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-05-10T10:20:56.229 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-05-10T10:20:56.230 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-05-10T10:20:56.300 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-05-10T10:20:56.300 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-05-10T10:20:56.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:20:56.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:20:56.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:20:56.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:20:56.356 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:20:56.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:20:56.381 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T10:20:56.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:20:56.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:20:56.482 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:20:56.518 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T10:20:56.528 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-10T10:20:56.536 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-10T10:20:56.668 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T10:20:56.684 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T10:20:56.686 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi148.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T10:24:55.115 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-05-10T10:24:55.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-10T10:24:55.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:24:55.195 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-05-10T10:24:55.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-10T10:24:55.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-05-10T10:24:55.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:24:55.362 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-05-10T10:24:55.442 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-05-10T10:24:55.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-10T10:24:55.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:24:55.524 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-05-10T10:24:55.614 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-10T10:24:55.615 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T10:24:55.625 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T10:24:55.625 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T10:24:55.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:24:55.628 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T10:24:55.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:24:55.630 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T10:24:55.630 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:24:55.656 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T10:24:55.659 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T10:24:55.670 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T10:24:55.674 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T10:24:55.684 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T10:24:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-05-10T10:24:55.703 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-05-10T10:24:55.703 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T10:24:55.707 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-05-10T10:24:55.711 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-05-10T10:24:55.720 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T10:24:55.722 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-05-10T10:24:55.725 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T10:24:55.733 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-05-10T10:24:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T10:24:55.740 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T10:24:55.747 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T10:24:55.760 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T10:24:55.789 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:24:55.792 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:24:55.792 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-05-10T10:24:55.792 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.792 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.792 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.792 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.794 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:24:55.797 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:24:55.797 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-05-10T10:24:55.797 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.797 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.798 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.798 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.814 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:24:55.817 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:24:55.817 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-05-10T10:24:55.818 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.818 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.818 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.818 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:24:55.819 INFO:teuthology.run_tasks:Running task install... 2024-05-10T10:24:55.827 DEBUG:teuthology.task.install:project ceph 2024-05-10T10:24:55.827 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'}} 2024-05-10T10:24:55.827 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'} 2024-05-10T10:24:55.828 INFO:teuthology.task.install:Using flavor: default 2024-05-10T10:24:55.833 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T10:24:55.834 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T10:24:55.834 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T10:24:55.834 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:24:55.836 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T10:24:55.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:24:55.837 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T10:24:55.838 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:24:55.986 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T10:24:55.986 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T10:24:55.997 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T10:24:55.997 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T10:24:56.070 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T10:24:56.070 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T10:24:56.105 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T10:24:56.105 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:24:56.105 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T10:24:56.131 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T10:24:56.131 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T10:24:56.177 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T10:24:56.178 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:24:56.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T10:24:56.198 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T10:24:56.203 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T10:24:56.203 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T10:24:56.239 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T10:24:56.239 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:24:56.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T10:24:56.266 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T10:24:56.266 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T10:24:56.269 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T10:24:56.274 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T10:24:56.333 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T10:24:56.338 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-05-10T10:24:56.339 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-05-10T10:24:56.344 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T10:24:56.409 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T10:24:56.409 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-05-10T10:24:56.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-05-10T10:24:56.473 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-05-10T10:24:56.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-05-10T10:24:56.579 INFO:teuthology.orchestra.run.smithi195.stdout:54 files removed 2024-05-10T10:24:56.607 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T10:24:56.653 INFO:teuthology.orchestra.run.smithi148.stdout:54 files removed 2024-05-10T10:24:56.683 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T10:24:56.704 INFO:teuthology.orchestra.run.smithi112.stdout:54 files removed 2024-05-10T10:24:56.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T10:24:57.373 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 171 kB/s | 84 kB 00:00 2024-05-10T10:24:57.477 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 178 kB/s | 84 kB 00:00 2024-05-10T10:24:57.584 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 134 kB/s | 84 kB 00:00 2024-05-10T10:24:58.141 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-05-10T10:24:58.270 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-05-10T10:24:58.276 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-05-10T10:24:58.674 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-10T10:24:58.853 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-05-10T10:24:58.869 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-05-10T10:24:59.320 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.1 MB 00:00 2024-05-10T10:24:59.499 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-10T10:25:00.561 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.1 MB 00:01 2024-05-10T10:25:01.338 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-10T10:25:02.245 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 32 MB/s | 19 MB 00:00 2024-05-10T10:25:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 19 MB 00:03 2024-05-10T10:25:05.423 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-10T10:25:06.816 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 16 kB 00:00 2024-05-10T10:25:07.986 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-10T10:25:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-10T10:25:09.639 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 16 kB 00:00 2024-05-10T10:25:09.701 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.2 MB 00:01 2024-05-10T10:25:10.865 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-10T10:25:11.169 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-05-10T10:25:12.212 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-10T10:25:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-10T10:25:15.519 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:25:15.519 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:25:15.530 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T10:25:15.530 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T10:25:15.531 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T10:25:15.532 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T10:25:15.533 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T10:25:15.536 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T10:25:15.536 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T10:25:15.596 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-05-10T10:25:15.614 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-05-10T10:25:15.614 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-05-10T10:25:15.614 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-05-10T10:25:15.614 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-05-10T10:25:15.614 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T10:25:15.614 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T10:25:15.614 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T10:25:15.615 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T10:25:15.616 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T10:25:15.617 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T10:25:15.618 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T10:25:15.619 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T10:25:15.620 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T10:25:15.621 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T10:25:15.622 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T10:25:15.623 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T10:25:15.624 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T10:25:15.625 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 8 Packages 2024-05-10T10:25:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:25:15.627 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 226 M 2024-05-10T10:25:15.627 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-05-10T10:25:16.016 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-10T10:25:16.336 INFO:teuthology.orchestra.run.smithi112.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-10T10:25:16.728 INFO:teuthology.orchestra.run.smithi112.stdout:(2/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 2.2 MB/s | 890 kB 00:00 2024-05-10T10:25:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:(3/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 7.7 MB/s | 5.4 MB 00:00 2024-05-10T10:25:16.820 INFO:teuthology.orchestra.run.smithi112.stdout:(4/156): ceph-immutable-object-cache-19.0.0-344 1.5 MB/s | 145 kB 00:00 2024-05-10T10:25:16.998 INFO:teuthology.orchestra.run.smithi112.stdout:(5/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 12 MB/s | 2.4 MB 00:00 2024-05-10T10:25:17.046 INFO:teuthology.orchestra.run.smithi112.stdout:(6/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-05-10T10:25:17.354 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-10T10:25:17.371 INFO:teuthology.orchestra.run.smithi112.stdout:(7/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 13 MB/s | 4.7 MB 00:00 2024-05-10T10:25:18.280 INFO:teuthology.orchestra.run.smithi112.stdout:(8/156): ceph-radosgw-19.0.0-3449.gd39fa088.el9 12 MB/s | 11 MB 00:00 2024-05-10T10:25:18.342 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:25:18.342 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:25:18.346 INFO:teuthology.orchestra.run.smithi112.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 379 kB/s | 25 kB 00:00 2024-05-10T10:25:18.353 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T10:25:18.353 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T10:25:18.354 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T10:25:18.355 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T10:25:18.356 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T10:25:18.359 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T10:25:18.359 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T10:25:18.420 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T10:25:18.436 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T10:25:18.437 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:25:18.438 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T10:25:18.439 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T10:25:18.440 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T10:25:18.441 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T10:25:18.442 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T10:25:18.443 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T10:25:18.444 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T10:25:18.445 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T10:25:18.446 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T10:25:18.447 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:25:18.448 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-05-10T10:25:18.448 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-05-10T10:25:18.448 INFO:teuthology.orchestra.run.smithi148.stdout:Install 148 Packages 2024-05-10T10:25:18.448 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 8 Packages 2024-05-10T10:25:18.448 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:25:18.448 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 226 M 2024-05-10T10:25:18.448 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-05-10T10:25:18.714 INFO:teuthology.orchestra.run.smithi112.stdout:(10/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x8 10 MB/s | 17 MB 00:01 2024-05-10T10:25:18.780 INFO:teuthology.orchestra.run.smithi112.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 3.3 MB/s | 225 kB 00:00 2024-05-10T10:25:18.872 INFO:teuthology.orchestra.run.smithi112.stdout:(12/156): cephfs-shell-19.0.0-3449.gd39fa088.el 295 kB/s | 27 kB 00:00 2024-05-10T10:25:18.964 INFO:teuthology.orchestra.run.smithi112.stdout:(13/156): libcephfs-devel-19.0.0-3449.gd39fa088 346 kB/s | 32 kB 00:00 2024-05-10T10:25:19.192 INFO:teuthology.orchestra.run.smithi148.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-10T10:25:19.589 INFO:teuthology.orchestra.run.smithi112.stdout:(14/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 1.2 MB/s | 746 kB 00:00 2024-05-10T10:25:19.689 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:25:19.689 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:25:19.700 INFO:teuthology.orchestra.run.smithi195.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T10:25:19.701 INFO:teuthology.orchestra.run.smithi195.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T10:25:19.702 INFO:teuthology.orchestra.run.smithi195.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T10:25:19.703 INFO:teuthology.orchestra.run.smithi195.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T10:25:19.705 INFO:teuthology.orchestra.run.smithi195.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T10:25:19.706 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T10:25:19.707 INFO:teuthology.orchestra.run.smithi112.stdout:(15/156): libcephsqlite-19.0.0-3449.gd39fa088.e 1.4 MB/s | 163 kB 00:00 2024-05-10T10:25:19.707 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T10:25:19.769 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-05-10T10:25:19.795 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-05-10T10:25:19.795 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-05-10T10:25:19.795 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-05-10T10:25:19.795 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-05-10T10:25:19.795 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T10:25:19.795 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T10:25:19.796 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T10:25:19.797 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T10:25:19.798 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T10:25:19.798 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T10:25:19.798 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T10:25:19.798 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T10:25:19.798 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T10:25:19.798 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T10:25:19.798 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T10:25:19.799 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T10:25:19.800 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T10:25:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T10:25:19.802 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T10:25:19.803 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T10:25:19.803 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T10:25:19.803 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T10:25:19.803 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T10:25:19.803 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T10:25:19.803 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T10:25:19.803 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T10:25:19.804 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T10:25:19.805 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:25:19.805 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T10:25:19.805 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T10:25:19.805 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T10:25:19.805 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T10:25:19.806 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T10:25:19.806 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T10:25:19.806 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T10:25:19.806 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T10:25:19.806 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T10:25:19.806 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T10:25:19.806 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T10:25:19.807 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T10:25:19.808 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T10:25:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T10:25:19.810 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T10:25:19.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T10:25:19.812 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout:Install 148 Packages 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 8 Packages 2024-05-10T10:25:19.813 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:25:19.814 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 226 M 2024-05-10T10:25:19.814 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-05-10T10:25:19.823 INFO:teuthology.orchestra.run.smithi112.stdout:(16/156): librados-devel-19.0.0-3449.gd39fa088. 1.1 MB/s | 127 kB 00:00 2024-05-10T10:25:20.009 INFO:teuthology.orchestra.run.smithi148.stdout:(2/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 4.6 MB/s | 5.4 MB 00:01 2024-05-10T10:25:20.060 INFO:teuthology.orchestra.run.smithi148.stdout:(3/156): ceph-immutable-object-cache-19.0.0-344 2.8 MB/s | 145 kB 00:00 2024-05-10T10:25:20.165 INFO:teuthology.orchestra.run.smithi112.stdout:(17/156): ceph-common-19.0.0-3449.gd39fa088.el9 5.2 MB/s | 21 MB 00:04 2024-05-10T10:25:20.193 INFO:teuthology.orchestra.run.smithi148.stdout:(4/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 889 kB/s | 890 kB 00:01 2024-05-10T10:25:20.335 INFO:teuthology.orchestra.run.smithi148.stdout:(5/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-05-10T10:25:20.357 INFO:teuthology.orchestra.run.smithi112.stdout:(18/156): libradosstriper1-19.0.0-3449.gd39fa08 938 kB/s | 500 kB 00:00 2024-05-10T10:25:20.458 INFO:teuthology.orchestra.run.smithi112.stdout:(19/156): python3-ceph-argparse-19.0.0-3449.gd3 451 kB/s | 45 kB 00:00 2024-05-10T10:25:20.566 INFO:teuthology.orchestra.run.smithi112.stdout:(20/156): python3-ceph-common-19.0.0-3449.gd39f 1.2 MB/s | 135 kB 00:00 2024-05-10T10:25:20.664 INFO:teuthology.orchestra.run.smithi195.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-10T10:25:20.675 INFO:teuthology.orchestra.run.smithi112.stdout:(21/156): python3-cephfs-19.0.0-3449.gd39fa088. 1.5 MB/s | 161 kB 00:00 2024-05-10T10:25:20.792 INFO:teuthology.orchestra.run.smithi112.stdout:(22/156): python3-rados-19.0.0-3449.gd39fa088.e 2.7 MB/s | 321 kB 00:00 2024-05-10T10:25:20.892 INFO:teuthology.orchestra.run.smithi112.stdout:(23/156): python3-rbd-19.0.0-3449.gd39fa088.el9 2.9 MB/s | 298 kB 00:00 2024-05-10T10:25:20.968 INFO:teuthology.orchestra.run.smithi112.stdout:(24/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 6.7 MB/s | 5.4 MB 00:00 2024-05-10T10:25:20.985 INFO:teuthology.orchestra.run.smithi148.stdout:(6/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 7.2 MB/s | 4.7 MB 00:00 2024-05-10T10:25:20.993 INFO:teuthology.orchestra.run.smithi112.stdout:(25/156): python3-rgw-19.0.0-3449.gd39fa088.el9 990 kB/s | 99 kB 00:00 2024-05-10T10:25:21.034 INFO:teuthology.orchestra.run.smithi112.stdout:(26/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 1.3 MB/s | 85 kB 00:00 2024-05-10T10:25:21.060 INFO:teuthology.orchestra.run.smithi148.stdout:(7/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-05-10T10:25:21.135 INFO:teuthology.orchestra.run.smithi112.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-10T10:25:21.235 INFO:teuthology.orchestra.run.smithi112.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 342 kB/s | 34 kB 00:00 2024-05-10T10:25:21.344 INFO:teuthology.orchestra.run.smithi112.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 1.3 MB/s | 142 kB 00:00 2024-05-10T10:25:21.703 INFO:teuthology.orchestra.run.smithi112.stdout:(30/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 4.4 MB/s | 3.1 MB 00:00 2024-05-10T10:25:22.031 INFO:teuthology.orchestra.run.smithi195.stdout:(2/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-05-10T10:25:22.078 INFO:teuthology.orchestra.run.smithi112.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 4.8 MB/s | 3.5 MB 00:00 2024-05-10T10:25:22.097 INFO:teuthology.orchestra.run.smithi195.stdout:(3/156): ceph-immutable-object-cache-19.0.0-344 2.1 MB/s | 145 kB 00:00 2024-05-10T10:25:22.178 INFO:teuthology.orchestra.run.smithi112.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 2.7 MB/s | 274 kB 00:00 2024-05-10T10:25:22.279 INFO:teuthology.orchestra.run.smithi112.stdout:(33/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 493 kB/s | 49 kB 00:00 2024-05-10T10:25:22.388 INFO:teuthology.orchestra.run.smithi112.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3449.gd 152 kB/s | 16 kB 00:00 2024-05-10T10:25:22.548 INFO:teuthology.orchestra.run.smithi195.stdout:(4/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-05-10T10:25:22.806 INFO:teuthology.orchestra.run.smithi195.stdout:(5/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 5.9 MB/s | 1.5 MB 00:00 2024-05-10T10:25:22.946 INFO:teuthology.orchestra.run.smithi112.stdout:(35/156): ceph-volume-19.0.0-3449.gd39fa088.el9 489 kB/s | 273 kB 00:00 2024-05-10T10:25:22.998 INFO:teuthology.orchestra.run.smithi195.stdout:(6/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 381 kB/s | 890 kB 00:02 2024-05-10T10:25:23.396 INFO:teuthology.orchestra.run.smithi112.stdout:(36/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 1.7 MB/s | 763 kB 00:00 2024-05-10T10:25:23.497 INFO:teuthology.orchestra.run.smithi112.stdout:(37/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 182 kB/s | 18 kB 00:00 2024-05-10T10:25:23.574 INFO:teuthology.orchestra.run.smithi195.stdout:(7/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 6.1 MB/s | 4.7 MB 00:00 2024-05-10T10:25:23.689 INFO:teuthology.orchestra.run.smithi112.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-05-10T10:25:23.864 INFO:teuthology.orchestra.run.smithi112.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-05-10T10:25:23.897 INFO:teuthology.orchestra.run.smithi112.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-05-10T10:25:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-10T10:25:23.994 INFO:teuthology.orchestra.run.smithi148.stdout:(8/156): ceph-radosgw-19.0.0-3449.gd39fa088.el9 3.6 MB/s | 11 MB 00:02 2024-05-10T10:25:24.073 INFO:teuthology.orchestra.run.smithi112.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-10T10:25:24.098 INFO:teuthology.orchestra.run.smithi112.stdout:(43/156): libquadmath-11.4.1-3.el9.x86_64.rpm 835 kB/s | 195 kB 00:00 2024-05-10T10:25:24.102 INFO:teuthology.orchestra.run.smithi148.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 233 kB/s | 25 kB 00:00 2024-05-10T10:25:24.132 INFO:teuthology.orchestra.run.smithi112.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-10T10:25:24.165 INFO:teuthology.orchestra.run.smithi112.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-10T10:25:24.191 INFO:teuthology.orchestra.run.smithi112.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-10T10:25:24.233 INFO:teuthology.orchestra.run.smithi112.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-10T10:25:24.383 INFO:teuthology.orchestra.run.smithi112.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-10T10:25:24.425 INFO:teuthology.orchestra.run.smithi112.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-05-10T10:25:24.450 INFO:teuthology.orchestra.run.smithi112.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-10T10:25:24.476 INFO:teuthology.orchestra.run.smithi112.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-05-10T10:25:24.501 INFO:teuthology.orchestra.run.smithi112.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-10T10:25:24.526 INFO:teuthology.orchestra.run.smithi112.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-05-10T10:25:24.568 INFO:teuthology.orchestra.run.smithi112.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-05-10T10:25:24.627 INFO:teuthology.orchestra.run.smithi112.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-05-10T10:25:24.652 INFO:teuthology.orchestra.run.smithi112.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-10T10:25:24.694 INFO:teuthology.orchestra.run.smithi112.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-05-10T10:25:24.720 INFO:teuthology.orchestra.run.smithi112.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-05-10T10:25:24.736 INFO:teuthology.orchestra.run.smithi148.stdout:(10/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x8 4.5 MB/s | 17 MB 00:03 2024-05-10T10:25:24.740 INFO:teuthology.orchestra.run.smithi195.stdout:(8/156): ceph-radosgw-19.0.0-3449.gd39fa088.el9 9.1 MB/s | 11 MB 00:01 2024-05-10T10:25:24.753 INFO:teuthology.orchestra.run.smithi112.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-10T10:25:24.795 INFO:teuthology.orchestra.run.smithi112.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-10T10:25:24.807 INFO:teuthology.orchestra.run.smithi195.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 379 kB/s | 25 kB 00:00 2024-05-10T10:25:24.812 INFO:teuthology.orchestra.run.smithi112.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 432 kB/s | 25 kB 00:00 2024-05-10T10:25:24.819 INFO:teuthology.orchestra.run.smithi148.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 2.6 MB/s | 225 kB 00:00 2024-05-10T10:25:24.829 INFO:teuthology.orchestra.run.smithi112.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-10T10:25:24.855 INFO:teuthology.orchestra.run.smithi112.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-05-10T10:25:24.880 INFO:teuthology.orchestra.run.smithi112.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-10T10:25:24.886 INFO:teuthology.orchestra.run.smithi148.stdout:(12/156): cephfs-shell-19.0.0-3449.gd39fa088.el 406 kB/s | 27 kB 00:00 2024-05-10T10:25:24.897 INFO:teuthology.orchestra.run.smithi112.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 35 kB 00:00 2024-05-10T10:25:24.962 INFO:teuthology.orchestra.run.smithi148.stdout:(13/156): libcephfs-devel-19.0.0-3449.gd39fa088 424 kB/s | 32 kB 00:00 2024-05-10T10:25:24.997 INFO:teuthology.orchestra.run.smithi112.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-05-10T10:25:25.056 INFO:teuthology.orchestra.run.smithi112.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-05-10T10:25:25.098 INFO:teuthology.orchestra.run.smithi112.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-10T10:25:25.131 INFO:teuthology.orchestra.run.smithi112.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-10T10:25:25.157 INFO:teuthology.orchestra.run.smithi112.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-10T10:25:25.187 INFO:teuthology.orchestra.run.smithi148.stdout:(14/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 3.2 MB/s | 746 kB 00:00 2024-05-10T10:25:25.224 INFO:teuthology.orchestra.run.smithi112.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-10T10:25:25.271 INFO:teuthology.orchestra.run.smithi148.stdout:(15/156): libcephsqlite-19.0.0-3449.gd39fa088.e 1.9 MB/s | 163 kB 00:00 2024-05-10T10:25:25.363 INFO:teuthology.orchestra.run.smithi148.stdout:(16/156): librados-devel-19.0.0-3449.gd39fa088. 1.3 MB/s | 127 kB 00:00 2024-05-10T10:25:25.513 INFO:teuthology.orchestra.run.smithi148.stdout:(17/156): libradosstriper1-19.0.0-3449.gd39fa08 3.3 MB/s | 500 kB 00:00 2024-05-10T10:25:25.640 INFO:teuthology.orchestra.run.smithi112.stdout:(72/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 6.7 MB/s | 49 MB 00:07 2024-05-10T10:25:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 198 kB/s | 87 kB 00:00 2024-05-10T10:25:25.741 INFO:teuthology.orchestra.run.smithi112.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 9.5 MB/s | 5.8 MB 00:00 2024-05-10T10:25:25.766 INFO:teuthology.orchestra.run.smithi112.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-10T10:25:25.792 INFO:teuthology.orchestra.run.smithi112.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-10T10:25:25.817 INFO:teuthology.orchestra.run.smithi112.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-10T10:25:25.842 INFO:teuthology.orchestra.run.smithi112.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-05-10T10:25:25.867 INFO:teuthology.orchestra.run.smithi112.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-10T10:25:25.893 INFO:teuthology.orchestra.run.smithi112.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-05-10T10:25:25.935 INFO:teuthology.orchestra.run.smithi112.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-10T10:25:26.010 INFO:teuthology.orchestra.run.smithi112.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-05-10T10:25:26.035 INFO:teuthology.orchestra.run.smithi112.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-10T10:25:26.061 INFO:teuthology.orchestra.run.smithi112.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-10T10:25:26.086 INFO:teuthology.orchestra.run.smithi112.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-05-10T10:25:26.111 INFO:teuthology.orchestra.run.smithi112.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-10T10:25:26.137 INFO:teuthology.orchestra.run.smithi112.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-10T10:25:26.162 INFO:teuthology.orchestra.run.smithi112.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-10T10:25:26.187 INFO:teuthology.orchestra.run.smithi112.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-10T10:25:26.213 INFO:teuthology.orchestra.run.smithi112.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-10T10:25:26.288 INFO:teuthology.orchestra.run.smithi112.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-05-10T10:25:26.313 INFO:teuthology.orchestra.run.smithi112.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-10T10:25:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-05-10T10:25:26.364 INFO:teuthology.orchestra.run.smithi112.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-10T10:25:26.389 INFO:teuthology.orchestra.run.smithi112.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-10T10:25:26.414 INFO:teuthology.orchestra.run.smithi112.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-10T10:25:26.440 INFO:teuthology.orchestra.run.smithi112.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-10T10:25:26.456 INFO:teuthology.orchestra.run.smithi112.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-05-10T10:25:26.482 INFO:teuthology.orchestra.run.smithi112.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-10T10:25:26.499 INFO:teuthology.orchestra.run.smithi112.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-10T10:25:26.516 INFO:teuthology.orchestra.run.smithi112.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-05-10T10:25:26.558 INFO:teuthology.orchestra.run.smithi195.stdout:(10/156): ceph-common-19.0.0-3449.gd39fa088.el9 3.4 MB/s | 21 MB 00:06 2024-05-10T10:25:26.558 INFO:teuthology.orchestra.run.smithi112.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-05-10T10:25:26.621 INFO:teuthology.orchestra.run.smithi148.stdout:(18/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 4.8 MB/s | 5.4 MB 00:01 2024-05-10T10:25:26.680 INFO:teuthology.orchestra.run.smithi148.stdout:(19/156): python3-ceph-argparse-19.0.0-3449.gd3 778 kB/s | 45 kB 00:00 2024-05-10T10:25:26.691 INFO:teuthology.orchestra.run.smithi195.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 1.6 MB/s | 225 kB 00:00 2024-05-10T10:25:26.739 INFO:teuthology.orchestra.run.smithi148.stdout:(20/156): python3-ceph-common-19.0.0-3449.gd39f 2.3 MB/s | 135 kB 00:00 2024-05-10T10:25:26.758 INFO:teuthology.orchestra.run.smithi112.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-10T10:25:26.774 INFO:teuthology.orchestra.run.smithi195.stdout:(12/156): cephfs-shell-19.0.0-3449.gd39fa088.el 325 kB/s | 27 kB 00:00 2024-05-10T10:25:26.814 INFO:teuthology.orchestra.run.smithi148.stdout:(21/156): python3-cephfs-19.0.0-3449.gd39fa088. 2.1 MB/s | 161 kB 00:00 2024-05-10T10:25:26.817 INFO:teuthology.orchestra.run.smithi112.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-05-10T10:25:26.867 INFO:teuthology.orchestra.run.smithi195.stdout:(13/156): libcephfs-devel-19.0.0-3449.gd39fa088 346 kB/s | 32 kB 00:00 2024-05-10T10:25:26.923 INFO:teuthology.orchestra.run.smithi148.stdout:(22/156): python3-rados-19.0.0-3449.gd39fa088.e 2.9 MB/s | 321 kB 00:00 2024-05-10T10:25:27.056 INFO:teuthology.orchestra.run.smithi148.stdout:(23/156): python3-rbd-19.0.0-3449.gd39fa088.el9 2.2 MB/s | 298 kB 00:00 2024-05-10T10:25:27.109 INFO:teuthology.orchestra.run.smithi195.stdout:(14/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 3.0 MB/s | 746 kB 00:00 2024-05-10T10:25:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:(24/156): python3-rgw-19.0.0-3449.gd39fa088.el9 1.3 MB/s | 99 kB 00:00 2024-05-10T10:25:27.200 INFO:teuthology.orchestra.run.smithi195.stdout:(15/156): libcephsqlite-19.0.0-3449.gd39fa088.e 1.7 MB/s | 163 kB 00:00 2024-05-10T10:25:27.215 INFO:teuthology.orchestra.run.smithi148.stdout:(25/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-10T10:25:27.217 INFO:teuthology.orchestra.run.smithi112.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 49 kB/s | 19 kB 00:00 2024-05-10T10:25:27.267 INFO:teuthology.orchestra.run.smithi195.stdout:(16/156): librados-devel-19.0.0-3449.gd39fa088. 1.9 MB/s | 127 kB 00:00 2024-05-10T10:25:27.434 INFO:teuthology.orchestra.run.smithi195.stdout:(17/156): libradosstriper1-19.0.0-3449.gd39fa08 2.9 MB/s | 500 kB 00:00 2024-05-10T10:25:27.442 INFO:teuthology.orchestra.run.smithi112.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 620 kB/s | 548 kB 00:00 2024-05-10T10:25:27.468 INFO:teuthology.orchestra.run.smithi112.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 453 kB/s | 309 kB 00:00 2024-05-10T10:25:27.543 INFO:teuthology.orchestra.run.smithi112.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-10T10:25:27.585 INFO:teuthology.orchestra.run.smithi112.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-05-10T10:25:27.652 INFO:teuthology.orchestra.run.smithi112.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-05-10T10:25:27.719 INFO:teuthology.orchestra.run.smithi112.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-10T10:25:27.887 INFO:teuthology.orchestra.run.smithi112.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-05-10T10:25:27.928 INFO:teuthology.orchestra.run.smithi112.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-05-10T10:25:27.966 INFO:teuthology.orchestra.run.smithi148.stdout:(26/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 4.1 MB/s | 3.1 MB 00:00 2024-05-10T10:25:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-05-10T10:25:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 349 kB/s | 43 kB 00:00 2024-05-10T10:25:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-05-10T10:25:28.058 INFO:teuthology.orchestra.run.smithi148.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-10T10:25:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-10T10:25:28.121 INFO:teuthology.orchestra.run.smithi112.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-10T10:25:28.141 INFO:teuthology.orchestra.run.smithi148.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 411 kB/s | 34 kB 00:00 2024-05-10T10:25:28.171 INFO:teuthology.orchestra.run.smithi112.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-10T10:25:28.196 INFO:teuthology.orchestra.run.smithi112.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-10T10:25:28.225 INFO:teuthology.orchestra.run.smithi148.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 1.7 MB/s | 142 kB 00:00 2024-05-10T10:25:28.230 INFO:teuthology.orchestra.run.smithi112.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-05-10T10:25:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-10T10:25:28.339 INFO:teuthology.orchestra.run.smithi112.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-10T10:25:28.364 INFO:teuthology.orchestra.run.smithi112.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-10T10:25:28.398 INFO:teuthology.orchestra.run.smithi112.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-05-10T10:25:28.440 INFO:teuthology.orchestra.run.smithi112.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-10T10:25:28.466 INFO:teuthology.orchestra.run.smithi112.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-10T10:25:28.507 INFO:teuthology.orchestra.run.smithi112.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-05-10T10:25:28.549 INFO:teuthology.orchestra.run.smithi112.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-10T10:25:28.616 INFO:teuthology.orchestra.run.smithi112.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-10T10:25:28.641 INFO:teuthology.orchestra.run.smithi112.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-05-10T10:25:28.643 INFO:teuthology.orchestra.run.smithi195.stdout:(18/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 4.4 MB/s | 5.4 MB 00:01 2024-05-10T10:25:28.666 INFO:teuthology.orchestra.run.smithi112.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-05-10T10:25:28.725 INFO:teuthology.orchestra.run.smithi112.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-05-10T10:25:28.735 INFO:teuthology.orchestra.run.smithi195.stdout:(19/156): python3-ceph-argparse-19.0.0-3449.gd3 495 kB/s | 45 kB 00:00 2024-05-10T10:25:28.775 INFO:teuthology.orchestra.run.smithi112.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-10T10:25:28.801 INFO:teuthology.orchestra.run.smithi112.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-05-10T10:25:28.834 INFO:teuthology.orchestra.run.smithi112.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-05-10T10:25:28.835 INFO:teuthology.orchestra.run.smithi195.stdout:(20/156): python3-ceph-common-19.0.0-3449.gd39f 1.3 MB/s | 135 kB 00:00 2024-05-10T10:25:28.885 INFO:teuthology.orchestra.run.smithi112.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-10T10:25:28.910 INFO:teuthology.orchestra.run.smithi112.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-05-10T10:25:28.935 INFO:teuthology.orchestra.run.smithi195.stdout:(21/156): python3-cephfs-19.0.0-3449.gd39fa088. 1.6 MB/s | 161 kB 00:00 2024-05-10T10:25:28.943 INFO:teuthology.orchestra.run.smithi112.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-10T10:25:28.994 INFO:teuthology.orchestra.run.smithi112.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-05-10T10:25:29.017 INFO:teuthology.orchestra.run.smithi148.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 4.5 MB/s | 3.5 MB 00:00 2024-05-10T10:25:29.019 INFO:teuthology.orchestra.run.smithi112.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-05-10T10:25:29.036 INFO:teuthology.orchestra.run.smithi195.stdout:(22/156): python3-rados-19.0.0-3449.gd39fa088.e 3.1 MB/s | 321 kB 00:00 2024-05-10T10:25:29.053 INFO:teuthology.orchestra.run.smithi112.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-05-10T10:25:29.103 INFO:teuthology.orchestra.run.smithi112.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-10T10:25:29.128 INFO:teuthology.orchestra.run.smithi112.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-05-10T10:25:29.136 INFO:teuthology.orchestra.run.smithi195.stdout:(23/156): python3-rbd-19.0.0-3449.gd39fa088.el9 2.9 MB/s | 298 kB 00:00 2024-05-10T10:25:29.187 INFO:teuthology.orchestra.run.smithi112.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-10T10:25:29.212 INFO:teuthology.orchestra.run.smithi112.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-05-10T10:25:29.236 INFO:teuthology.orchestra.run.smithi195.stdout:(24/156): python3-rgw-19.0.0-3449.gd39fa088.el9 994 kB/s | 99 kB 00:00 2024-05-10T10:25:29.246 INFO:teuthology.orchestra.run.smithi112.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-10T10:25:29.337 INFO:teuthology.orchestra.run.smithi195.stdout:(25/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 857 kB/s | 85 kB 00:00 2024-05-10T10:25:29.388 INFO:teuthology.orchestra.run.smithi112.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-05-10T10:25:29.701 INFO:teuthology.orchestra.run.smithi148.stdout:(31/156): ceph-common-19.0.0-3449.gd39fa088.el9 1.9 MB/s | 21 MB 00:10 2024-05-10T10:25:29.738 INFO:teuthology.orchestra.run.smithi112.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 6.6 MB/s | 2.3 MB 00:00 2024-05-10T10:25:29.779 INFO:teuthology.orchestra.run.smithi195.stdout:(26/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 9.9 MB/s | 49 MB 00:04 2024-05-10T10:25:29.830 INFO:teuthology.orchestra.run.smithi112.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-05-10T10:25:29.851 INFO:teuthology.orchestra.run.smithi148.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 1.8 MB/s | 274 kB 00:00 2024-05-10T10:25:29.879 INFO:teuthology.orchestra.run.smithi195.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-10T10:25:29.880 INFO:teuthology.orchestra.run.smithi112.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 838 kB/s | 42 kB 00:00 2024-05-10T10:25:29.939 INFO:teuthology.orchestra.run.smithi112.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi148.stdout:(33/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 494 kB/s | 49 kB 00:00 2024-05-10T10:25:29.954 INFO:teuthology.orchestra.run.smithi195.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 458 kB/s | 34 kB 00:00 2024-05-10T10:25:30.038 INFO:teuthology.orchestra.run.smithi195.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 1.7 MB/s | 142 kB 00:00 2024-05-10T10:25:30.043 INFO:teuthology.orchestra.run.smithi148.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3449.gd 179 kB/s | 16 kB 00:00 2024-05-10T10:25:30.047 INFO:teuthology.orchestra.run.smithi112.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-05-10T10:25:30.096 INFO:teuthology.orchestra.run.smithi195.stdout:(30/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 4.1 MB/s | 3.1 MB 00:00 2024-05-10T10:25:30.098 INFO:teuthology.orchestra.run.smithi112.stdout:(154/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-05-10T10:25:30.173 INFO:teuthology.orchestra.run.smithi112.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-05-10T10:25:30.210 INFO:teuthology.orchestra.run.smithi148.stdout:(35/156): ceph-volume-19.0.0-3449.gd39fa088.el9 1.6 MB/s | 273 kB 00:00 2024-05-10T10:25:30.457 INFO:teuthology.orchestra.run.smithi112.stdout:(156/156): librados2-19.0.0-3449.gd39fa088.el9. 2.7 MB/s | 3.4 MB 00:01 2024-05-10T10:25:30.462 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:25:30.463 INFO:teuthology.orchestra.run.smithi112.stdout:Total 15 MB/s | 226 MB 00:14 2024-05-10T10:25:30.547 INFO:teuthology.orchestra.run.smithi195.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 7.0 MB/s | 3.5 MB 00:00 2024-05-10T10:25:30.585 INFO:teuthology.orchestra.run.smithi148.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-05-10T10:25:30.614 INFO:teuthology.orchestra.run.smithi195.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 4.0 MB/s | 274 kB 00:00 2024-05-10T10:25:30.619 INFO:teuthology.orchestra.run.smithi148.stdout:(37/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 1.8 MB/s | 763 kB 00:00 2024-05-10T10:25:30.636 INFO:teuthology.orchestra.run.smithi148.stdout:(38/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 365 kB/s | 18 kB 00:00 2024-05-10T10:25:30.664 INFO:teuthology.orchestra.run.smithi195.stdout:(33/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 991 kB/s | 49 kB 00:00 2024-05-10T10:25:30.714 INFO:teuthology.orchestra.run.smithi195.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3449.gd 329 kB/s | 16 kB 00:00 2024-05-10T10:25:30.753 INFO:teuthology.orchestra.run.smithi148.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 602 kB/s | 80 kB 00:00 2024-05-10T10:25:30.781 INFO:teuthology.orchestra.run.smithi195.stdout:(35/156): ceph-volume-19.0.0-3449.gd39fa088.el9 4.0 MB/s | 273 kB 00:00 2024-05-10T10:25:30.820 INFO:teuthology.orchestra.run.smithi148.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-05-10T10:25:30.845 INFO:teuthology.orchestra.run.smithi148.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-10T10:25:30.878 INFO:teuthology.orchestra.run.smithi148.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-05-10T10:25:30.882 INFO:teuthology.orchestra.run.smithi195.stdout:(36/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 7.5 MB/s | 763 kB 00:00 2024-05-10T10:25:30.921 INFO:teuthology.orchestra.run.smithi148.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-10T10:25:30.940 INFO:teuthology.orchestra.run.smithi195.stdout:(37/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 314 kB/s | 18 kB 00:00 2024-05-10T10:25:30.963 INFO:teuthology.orchestra.run.smithi148.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-05-10T10:25:30.988 INFO:teuthology.orchestra.run.smithi148.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-10T10:25:31.014 INFO:teuthology.orchestra.run.smithi148.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-10T10:25:31.039 INFO:teuthology.orchestra.run.smithi148.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-10T10:25:31.249 INFO:teuthology.orchestra.run.smithi195.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-05-10T10:25:31.464 INFO:teuthology.orchestra.run.smithi148.stdout:(48/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 6.7 MB/s | 49 MB 00:07 2024-05-10T10:25:31.506 INFO:teuthology.orchestra.run.smithi148.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 689 kB 00:00 2024-05-10T10:25:31.548 INFO:teuthology.orchestra.run.smithi148.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-05-10T10:25:31.573 INFO:teuthology.orchestra.run.smithi148.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-05-10T10:25:31.599 INFO:teuthology.orchestra.run.smithi148.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-10T10:25:31.624 INFO:teuthology.orchestra.run.smithi148.stdout:(53/156): boost-program-options-1.75.0-8.el9.x8 672 kB/s | 107 kB 00:00 2024-05-10T10:25:31.649 INFO:teuthology.orchestra.run.smithi148.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-05-10T10:25:31.652 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-05-10T10:25:31.674 INFO:teuthology.orchestra.run.smithi148.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-10T10:25:31.700 INFO:teuthology.orchestra.run.smithi148.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-10T10:25:31.725 INFO:teuthology.orchestra.run.smithi148.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-10T10:25:31.741 INFO:teuthology.orchestra.run.smithi195.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 100 kB/s | 80 kB 00:00 2024-05-10T10:25:31.750 INFO:teuthology.orchestra.run.smithi148.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-10T10:25:31.805 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-05-10T10:25:31.805 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-05-10T10:25:31.809 INFO:teuthology.orchestra.run.smithi148.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-05-10T10:25:31.842 INFO:teuthology.orchestra.run.smithi148.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-10T10:25:31.868 INFO:teuthology.orchestra.run.smithi148.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-10T10:25:31.893 INFO:teuthology.orchestra.run.smithi148.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-10T10:25:31.918 INFO:teuthology.orchestra.run.smithi148.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-10T10:25:31.952 INFO:teuthology.orchestra.run.smithi148.stdout:(64/156): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-05-10T10:25:31.977 INFO:teuthology.orchestra.run.smithi148.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-05-10T10:25:32.003 INFO:teuthology.orchestra.run.smithi148.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-10T10:25:32.111 INFO:teuthology.orchestra.run.smithi148.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-05-10T10:25:32.153 INFO:teuthology.orchestra.run.smithi148.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-10T10:25:32.282 INFO:teuthology.orchestra.run.smithi148.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-05-10T10:25:32.304 INFO:teuthology.orchestra.run.smithi148.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 886 kB/s | 133 kB 00:00 2024-05-10T10:25:32.338 INFO:teuthology.orchestra.run.smithi148.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 3.9 MB/s | 911 kB 00:00 2024-05-10T10:25:32.350 INFO:teuthology.orchestra.run.smithi195.stdout:(40/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x8 1.8 MB/s | 17 MB 00:09 2024-05-10T10:25:32.380 INFO:teuthology.orchestra.run.smithi148.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-10T10:25:32.392 INFO:teuthology.orchestra.run.smithi195.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 720 kB/s | 803 kB 00:01 2024-05-10T10:25:32.430 INFO:teuthology.orchestra.run.smithi148.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.0 MB/s | 1.0 MB 00:00 2024-05-10T10:25:32.522 INFO:teuthology.orchestra.run.smithi148.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-05-10T10:25:32.556 INFO:teuthology.orchestra.run.smithi148.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-10T10:25:32.581 INFO:teuthology.orchestra.run.smithi148.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-10T10:25:32.606 INFO:teuthology.orchestra.run.smithi148.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-10T10:25:32.625 INFO:teuthology.orchestra.run.smithi195.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 221 kB/s | 195 kB 00:00 2024-05-10T10:25:32.631 INFO:teuthology.orchestra.run.smithi148.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-05-10T10:25:32.657 INFO:teuthology.orchestra.run.smithi148.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-10T10:25:32.682 INFO:teuthology.orchestra.run.smithi148.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-05-10T10:25:32.766 INFO:teuthology.orchestra.run.smithi148.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-10T10:25:32.791 INFO:teuthology.orchestra.run.smithi148.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-10T10:25:32.816 INFO:teuthology.orchestra.run.smithi148.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-10T10:25:32.842 INFO:teuthology.orchestra.run.smithi148.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-10T10:25:32.867 INFO:teuthology.orchestra.run.smithi148.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-05-10T10:25:32.892 INFO:teuthology.orchestra.run.smithi148.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-10T10:25:32.918 INFO:teuthology.orchestra.run.smithi148.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-10T10:25:32.951 INFO:teuthology.orchestra.run.smithi148.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-05-10T10:25:32.977 INFO:teuthology.orchestra.run.smithi148.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-05-10T10:25:33.027 INFO:teuthology.orchestra.run.smithi148.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 714 kB/s | 54 kB 00:00 2024-05-10T10:25:33.102 INFO:teuthology.orchestra.run.smithi148.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-05-10T10:25:33.127 INFO:teuthology.orchestra.run.smithi148.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-05-10T10:25:33.153 INFO:teuthology.orchestra.run.smithi148.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-05-10T10:25:33.177 INFO:teuthology.orchestra.run.smithi195.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 307 kB/s | 253 kB 00:00 2024-05-10T10:25:33.178 INFO:teuthology.orchestra.run.smithi148.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-10T10:25:33.203 INFO:teuthology.orchestra.run.smithi148.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-10T10:25:33.228 INFO:teuthology.orchestra.run.smithi148.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-05-10T10:25:33.254 INFO:teuthology.orchestra.run.smithi148.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-10T10:25:33.279 INFO:teuthology.orchestra.run.smithi148.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-10T10:25:33.304 INFO:teuthology.orchestra.run.smithi148.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-10T10:25:33.330 INFO:teuthology.orchestra.run.smithi148.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-10T10:25:33.355 INFO:teuthology.orchestra.run.smithi148.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-05-10T10:25:33.359 INFO:teuthology.orchestra.run.smithi195.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 151 kB/s | 106 kB 00:00 2024-05-10T10:25:33.418 INFO:teuthology.orchestra.run.smithi195.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-10T10:25:33.606 INFO:teuthology.orchestra.run.smithi148.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-10T10:25:33.622 INFO:teuthology.orchestra.run.smithi148.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-05-10T10:25:33.798 INFO:teuthology.orchestra.run.smithi148.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-05-10T10:25:33.968 INFO:teuthology.orchestra.run.smithi195.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 171 kB/s | 135 kB 00:00 2024-05-10T10:25:34.034 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-05-10T10:25:34.035 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-05-10T10:25:34.126 INFO:teuthology.orchestra.run.smithi195.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-05-10T10:25:34.152 INFO:teuthology.orchestra.run.smithi195.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 158 kB/s | 125 kB 00:00 2024-05-10T10:25:34.203 INFO:teuthology.orchestra.run.smithi195.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-05-10T10:25:34.244 INFO:teuthology.orchestra.run.smithi195.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 519 kB/s | 21 kB 00:00 2024-05-10T10:25:34.248 INFO:teuthology.orchestra.run.smithi148.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 43 kB/s | 19 kB 00:00 2024-05-10T10:25:34.277 INFO:teuthology.orchestra.run.smithi195.stdout:(51/156): python3-urllib3-1.26.5-5.el9.noarch.r 258 kB/s | 215 kB 00:00 2024-05-10T10:25:34.303 INFO:teuthology.orchestra.run.smithi195.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-05-10T10:25:34.328 INFO:teuthology.orchestra.run.smithi195.stdout:(53/156): boost-program-options-1.75.0-8.el9.x8 608 kB/s | 107 kB 00:00 2024-05-10T10:25:34.362 INFO:teuthology.orchestra.run.smithi195.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-05-10T10:25:34.387 INFO:teuthology.orchestra.run.smithi195.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-05-10T10:25:34.412 INFO:teuthology.orchestra.run.smithi195.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-10T10:25:34.440 INFO:teuthology.orchestra.run.smithi148.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 378 kB/s | 309 kB 00:00 2024-05-10T10:25:34.471 INFO:teuthology.orchestra.run.smithi195.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-05-10T10:25:34.505 INFO:teuthology.orchestra.run.smithi195.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-05-10T10:25:34.530 INFO:teuthology.orchestra.run.smithi195.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-05-10T10:25:34.548 INFO:teuthology.orchestra.run.smithi148.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-05-10T10:25:34.572 INFO:teuthology.orchestra.run.smithi195.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-05-10T10:25:34.597 INFO:teuthology.orchestra.run.smithi195.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-05-10T10:25:34.623 INFO:teuthology.orchestra.run.smithi195.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-10T10:25:34.648 INFO:teuthology.orchestra.run.smithi195.stdout:(63/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-10T10:25:34.649 INFO:teuthology.orchestra.run.smithi148.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-05-10T10:25:34.673 INFO:teuthology.orchestra.run.smithi195.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-10T10:25:34.690 INFO:teuthology.orchestra.run.smithi195.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 35 kB 00:00 2024-05-10T10:25:34.715 INFO:teuthology.orchestra.run.smithi195.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-10T10:25:34.774 INFO:teuthology.orchestra.run.smithi195.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-05-10T10:25:34.816 INFO:teuthology.orchestra.run.smithi148.stdout:(109/156): abseil-cpp-20211102.0-3.el9.x86_64.r 453 kB/s | 548 kB 00:01 2024-05-10T10:25:34.841 INFO:teuthology.orchestra.run.smithi148.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 351 kB/s | 67 kB 00:00 2024-05-10T10:25:34.849 INFO:teuthology.orchestra.run.smithi195.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-05-10T10:25:34.891 INFO:teuthology.orchestra.run.smithi195.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-10T10:25:34.916 INFO:teuthology.orchestra.run.smithi195.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-10T10:25:34.975 INFO:teuthology.orchestra.run.smithi195.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-10T10:25:35.017 INFO:teuthology.orchestra.run.smithi195.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-10T10:25:35.034 INFO:teuthology.orchestra.run.smithi195.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-10T10:25:35.076 INFO:teuthology.orchestra.run.smithi195.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-10T10:25:35.084 INFO:teuthology.orchestra.run.smithi148.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-05-10T10:25:35.152 INFO:teuthology.orchestra.run.smithi195.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-05-10T10:25:35.176 INFO:teuthology.orchestra.run.smithi195.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-10T10:25:35.202 INFO:teuthology.orchestra.run.smithi195.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-10T10:25:35.227 INFO:teuthology.orchestra.run.smithi195.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-05-10T10:25:35.252 INFO:teuthology.orchestra.run.smithi195.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-10T10:25:35.278 INFO:teuthology.orchestra.run.smithi195.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-10T10:25:35.361 INFO:teuthology.orchestra.run.smithi195.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-10T10:25:35.386 INFO:teuthology.orchestra.run.smithi195.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-10T10:25:35.412 INFO:teuthology.orchestra.run.smithi195.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-10T10:25:35.437 INFO:teuthology.orchestra.run.smithi195.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-10T10:25:35.462 INFO:teuthology.orchestra.run.smithi195.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-05-10T10:25:35.488 INFO:teuthology.orchestra.run.smithi195.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-10T10:25:35.513 INFO:teuthology.orchestra.run.smithi195.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-10T10:25:35.539 INFO:teuthology.orchestra.run.smithi195.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-10T10:25:35.564 INFO:teuthology.orchestra.run.smithi195.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-10T10:25:35.589 INFO:teuthology.orchestra.run.smithi195.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-10T10:25:35.693 INFO:teuthology.orchestra.run.smithi195.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-10T10:25:35.723 INFO:teuthology.orchestra.run.smithi195.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 311 kB/s | 42 kB 00:00 2024-05-10T10:25:35.748 INFO:teuthology.orchestra.run.smithi195.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 812 kB/s | 44 kB 00:00 2024-05-10T10:25:35.774 INFO:teuthology.orchestra.run.smithi195.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-10T10:25:35.799 INFO:teuthology.orchestra.run.smithi195.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-10T10:25:35.824 INFO:teuthology.orchestra.run.smithi195.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-10T10:25:35.833 INFO:teuthology.orchestra.run.smithi148.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 731 kB/s | 548 kB 00:00 2024-05-10T10:25:35.850 INFO:teuthology.orchestra.run.smithi195.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-10T10:25:35.875 INFO:teuthology.orchestra.run.smithi195.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-10T10:25:35.900 INFO:teuthology.orchestra.run.smithi195.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-10T10:25:35.917 INFO:teuthology.orchestra.run.smithi148.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 780 kB/s | 839 kB 00:01 2024-05-10T10:25:35.925 INFO:teuthology.orchestra.run.smithi195.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-10T10:25:35.942 INFO:teuthology.orchestra.run.smithi148.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-05-10T10:25:35.950 INFO:teuthology.orchestra.run.smithi195.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-05-10T10:25:36.017 INFO:teuthology.orchestra.run.smithi148.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-05-10T10:25:36.042 INFO:teuthology.orchestra.run.smithi148.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-05-10T10:25:36.176 INFO:teuthology.orchestra.run.smithi195.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-10T10:25:36.201 INFO:teuthology.orchestra.run.smithi195.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-05-10T10:25:36.285 INFO:teuthology.orchestra.run.smithi148.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 644 kB/s | 172 kB 00:00 2024-05-10T10:25:36.343 INFO:teuthology.orchestra.run.smithi195.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-05-10T10:25:36.460 INFO:teuthology.orchestra.run.smithi195.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-10T10:25:36.485 INFO:teuthology.orchestra.run.smithi195.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-05-10T10:25:36.519 INFO:teuthology.orchestra.run.smithi195.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-10T10:25:36.552 INFO:teuthology.orchestra.run.smithi148.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-05-10T10:25:36.552 INFO:teuthology.orchestra.run.smithi195.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-05-10T10:25:36.578 INFO:teuthology.orchestra.run.smithi195.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-05-10T10:25:36.636 INFO:teuthology.orchestra.run.smithi195.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-10T10:25:36.660 INFO:teuthology.orchestra.run.smithi148.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-05-10T10:25:36.670 INFO:teuthology.orchestra.run.smithi195.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-05-10T10:25:36.719 INFO:teuthology.orchestra.run.smithi148.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 699 kB/s | 303 kB 00:00 2024-05-10T10:25:36.737 INFO:teuthology.orchestra.run.smithi195.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-10T10:25:36.770 INFO:teuthology.orchestra.run.smithi195.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-10T10:25:36.796 INFO:teuthology.orchestra.run.smithi195.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 745 kB/s | 43 kB 00:00 2024-05-10T10:25:36.821 INFO:teuthology.orchestra.run.smithi195.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-05-10T10:25:36.847 INFO:teuthology.orchestra.run.smithi195.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 296 kB/s | 15 kB 00:00 2024-05-10T10:25:36.872 INFO:teuthology.orchestra.run.smithi195.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-05-10T10:25:36.897 INFO:teuthology.orchestra.run.smithi195.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-10T10:25:36.922 INFO:teuthology.orchestra.run.smithi195.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-05-10T10:25:36.948 INFO:teuthology.orchestra.run.smithi195.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-10T10:25:36.973 INFO:teuthology.orchestra.run.smithi195.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 467 kB/s | 35 kB 00:00 2024-05-10T10:25:36.998 INFO:teuthology.orchestra.run.smithi195.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-10T10:25:37.002 INFO:teuthology.orchestra.run.smithi148.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 601 kB/s | 206 kB 00:00 2024-05-10T10:25:37.065 INFO:teuthology.orchestra.run.smithi195.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-10T10:25:37.091 INFO:teuthology.orchestra.run.smithi195.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-10T10:25:37.116 INFO:teuthology.orchestra.run.smithi195.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-10T10:25:37.158 INFO:teuthology.orchestra.run.smithi195.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-05-10T10:25:37.183 INFO:teuthology.orchestra.run.smithi195.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-10T10:25:37.208 INFO:teuthology.orchestra.run.smithi195.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-10T10:25:37.234 INFO:teuthology.orchestra.run.smithi195.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-05-10T10:25:37.261 INFO:teuthology.orchestra.run.smithi148.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 559 kB/s | 144 kB 00:00 2024-05-10T10:25:37.267 INFO:teuthology.orchestra.run.smithi195.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-05-10T10:25:37.293 INFO:teuthology.orchestra.run.smithi195.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-10T10:25:37.326 INFO:teuthology.orchestra.run.smithi195.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-10T10:25:37.352 INFO:teuthology.orchestra.run.smithi195.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-05-10T10:25:37.377 INFO:teuthology.orchestra.run.smithi195.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-10T10:25:37.402 INFO:teuthology.orchestra.run.smithi195.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-10T10:25:37.427 INFO:teuthology.orchestra.run.smithi195.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-10T10:25:37.452 INFO:teuthology.orchestra.run.smithi195.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-10T10:25:37.478 INFO:teuthology.orchestra.run.smithi195.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-10T10:25:37.503 INFO:teuthology.orchestra.run.smithi195.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-10T10:25:37.511 INFO:teuthology.orchestra.run.smithi148.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 43 kB/s | 11 kB 00:00 2024-05-10T10:25:37.528 INFO:teuthology.orchestra.run.smithi195.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-10T10:25:37.554 INFO:teuthology.orchestra.run.smithi195.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-05-10T10:25:37.579 INFO:teuthology.orchestra.run.smithi195.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-10T10:25:37.604 INFO:teuthology.orchestra.run.smithi195.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-10T10:25:37.620 INFO:teuthology.orchestra.run.smithi148.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-05-10T10:25:37.629 INFO:teuthology.orchestra.run.smithi195.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-10T10:25:37.663 INFO:teuthology.orchestra.run.smithi195.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-10T10:25:37.689 INFO:teuthology.orchestra.run.smithi195.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 237 kB/s | 20 kB 00:00 2024-05-10T10:25:37.714 INFO:teuthology.orchestra.run.smithi195.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-10T10:25:37.728 INFO:teuthology.orchestra.run.smithi148.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-05-10T10:25:37.756 INFO:teuthology.orchestra.run.smithi195.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-10T10:25:37.829 INFO:teuthology.orchestra.run.smithi148.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-10T10:25:37.929 INFO:teuthology.orchestra.run.smithi148.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-10T10:25:38.481 INFO:teuthology.orchestra.run.smithi195.stdout:(149/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-05-10T10:25:38.948 INFO:teuthology.orchestra.run.smithi195.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.9 MB/s | 2.3 MB 00:01 2024-05-10T10:25:39.015 INFO:teuthology.orchestra.run.smithi195.stdout:(151/156): librados2-19.0.0-3449.gd39fa088.el9. 2.6 MB/s | 3.4 MB 00:01 2024-05-10T10:25:39.438 INFO:teuthology.orchestra.run.smithi148.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 767 kB/s | 2.0 MB 00:02 2024-05-10T10:25:39.472 INFO:teuthology.orchestra.run.smithi148.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 678 kB/s | 1.0 MB 00:01 2024-05-10T10:25:39.572 INFO:teuthology.orchestra.run.smithi148.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-05-10T10:25:39.680 INFO:teuthology.orchestra.run.smithi148.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 378 kB/s | 79 kB 00:00 2024-05-10T10:25:39.707 INFO:teuthology.orchestra.run.smithi195.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 55 kB/s | 42 kB 00:00 2024-05-10T10:25:39.781 INFO:teuthology.orchestra.run.smithi148.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 277 kB/s | 58 kB 00:00 2024-05-10T10:25:39.824 INFO:teuthology.orchestra.run.smithi195.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.4 MB/s | 1.8 MB 00:01 2024-05-10T10:25:39.889 INFO:teuthology.orchestra.run.smithi148.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-05-10T10:25:39.974 INFO:teuthology.orchestra.run.smithi195.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 259 kB/s | 248 kB 00:00 2024-05-10T10:25:40.000 INFO:teuthology.orchestra.run.smithi148.stdout:(134/156): libarrow-9.0.0-11.el9.x86_64.rpm 786 kB/s | 4.4 MB 00:05 2024-05-10T10:25:40.106 INFO:teuthology.orchestra.run.smithi148.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 415 kB/s | 90 kB 00:00 2024-05-10T10:25:40.132 INFO:teuthology.orchestra.run.smithi148.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 215 kB/s | 28 kB 00:00 2024-05-10T10:25:40.165 INFO:teuthology.orchestra.run.smithi148.stdout:(137/156): python3-pecan-1.4.2-3.el9.noarch.rpm 562 kB/s | 272 kB 00:00 2024-05-10T10:25:40.207 INFO:teuthology.orchestra.run.smithi148.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-10T10:25:40.308 INFO:teuthology.orchestra.run.smithi148.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-05-10T10:25:40.325 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-05-10T10:25:40.366 INFO:teuthology.orchestra.run.smithi148.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 295 kB/s | 59 kB 00:00 2024-05-10T10:25:40.420 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T10:25:40.441 INFO:teuthology.orchestra.run.smithi148.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 609 kB/s | 188 kB 00:00 2024-05-10T10:25:40.458 INFO:teuthology.orchestra.run.smithi148.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 339 kB/s | 51 kB 00:00 2024-05-10T10:25:40.466 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T10:25:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T10:25:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T10:25:40.588 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T10:25:40.651 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T10:25:40.667 INFO:teuthology.orchestra.run.smithi148.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 398 kB/s | 90 kB 00:00 2024-05-10T10:25:40.784 INFO:teuthology.orchestra.run.smithi148.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-05-10T10:25:40.826 INFO:teuthology.orchestra.run.smithi148.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 500 kB/s | 230 kB 00:00 2024-05-10T10:25:40.982 INFO:teuthology.orchestra.run.smithi195.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 1.3 MB 00:01 2024-05-10T10:25:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T10:25:41.106 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:25:41.109 INFO:teuthology.orchestra.run.smithi148.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 589 kB/s | 192 kB 00:00 2024-05-10T10:25:41.141 INFO:teuthology.orchestra.run.smithi195.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 2.4 MB 00:01 2024-05-10T10:25:41.144 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:25:41.144 INFO:teuthology.orchestra.run.smithi195.stdout:Total 11 MB/s | 226 MB 00:21 2024-05-10T10:25:41.228 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:25:41.257 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:25:41.326 INFO:teuthology.orchestra.run.smithi148.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 492 kB/s | 427 kB 00:00 2024-05-10T10:25:41.348 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:25:41.378 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:25:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T10:25:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T10:25:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:25:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:25:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:25:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:25:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:25:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:25:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:25:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:25:42.343 INFO:teuthology.orchestra.run.smithi148.stdout:(148/156): librados2-19.0.0-3449.gd39fa088.el9. 2.8 MB/s | 3.4 MB 00:01 2024-05-10T10:25:42.353 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-05-10T10:25:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:25:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:25:42.504 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-05-10T10:25:42.504 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-05-10T10:25:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:42.585 INFO:teuthology.orchestra.run.smithi148.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 9.6 MB/s | 2.3 MB 00:00 2024-05-10T10:25:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:42.644 INFO:teuthology.orchestra.run.smithi148.stdout:(150/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 2.4 MB/s | 3.1 MB 00:01 2024-05-10T10:25:42.727 INFO:teuthology.orchestra.run.smithi148.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 13 MB/s | 1.8 MB 00:00 2024-05-10T10:25:42.753 INFO:teuthology.orchestra.run.smithi148.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 384 kB/s | 42 kB 00:00 2024-05-10T10:25:42.778 INFO:teuthology.orchestra.run.smithi148.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-05-10T10:25:42.826 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:42.887 INFO:teuthology.orchestra.run.smithi148.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-10T10:25:42.928 INFO:teuthology.orchestra.run.smithi148.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-05-10T10:25:42.935 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T10:25:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T10:25:43.077 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T10:25:43.112 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T10:25:43.176 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T10:25:43.204 INFO:teuthology.orchestra.run.smithi148.stdout:(156/156): thrift-0.15.0-2.el9.x86_64.rpm 681 kB/s | 1.6 MB 00:02 2024-05-10T10:25:43.210 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:25:43.210 INFO:teuthology.orchestra.run.smithi148.stdout:Total 9.1 MB/s | 226 MB 00:24 2024-05-10T10:25:43.212 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T10:25:43.255 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T10:25:43.289 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T10:25:43.316 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T10:25:43.384 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T10:25:43.437 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T10:25:43.469 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T10:25:43.504 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T10:25:43.579 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T10:25:43.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T10:25:43.658 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:25:43.687 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T10:25:43.755 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T10:25:43.801 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T10:25:43.875 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T10:25:44.015 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T10:25:44.079 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T10:25:44.383 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-05-10T10:25:44.533 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-05-10T10:25:44.533 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-05-10T10:25:44.592 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T10:25:44.665 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-05-10T10:25:44.665 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-05-10T10:25:44.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T10:25:44.712 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T10:25:44.747 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T10:25:44.784 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T10:25:44.828 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T10:25:44.977 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T10:25:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T10:25:45.092 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T10:25:45.165 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T10:25:45.197 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T10:25:45.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T10:25:45.270 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T10:25:45.308 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T10:25:45.351 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T10:25:45.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T10:25:45.528 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T10:25:45.582 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T10:25:45.612 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T10:25:45.653 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T10:25:45.748 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T10:25:45.795 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T10:25:45.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T10:25:45.994 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T10:25:46.177 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T10:25:46.268 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T10:25:46.396 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T10:25:46.435 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T10:25:46.472 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T10:25:46.506 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T10:25:46.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T10:25:46.576 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T10:25:46.606 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T10:25:46.641 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T10:25:46.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T10:25:46.707 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T10:25:46.742 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T10:25:46.810 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-05-10T10:25:46.811 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-05-10T10:25:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T10:25:46.891 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T10:25:46.928 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T10:25:46.975 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T10:25:47.074 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T10:25:47.129 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T10:25:47.841 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T10:25:47.886 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T10:25:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T10:25:48.018 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T10:25:48.068 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T10:25:48.158 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T10:25:48.196 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T10:25:48.373 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T10:25:48.402 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T10:25:48.486 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T10:25:49.130 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T10:25:49.243 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T10:25:50.483 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T10:25:50.556 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T10:25:50.584 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T10:25:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T10:25:50.931 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T10:25:50.972 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:25:51.043 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-05-10T10:25:51.063 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:25:51.094 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T10:25:51.120 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T10:25:51.130 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T10:25:51.159 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T10:25:51.224 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T10:25:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T10:25:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T10:25:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T10:25:51.610 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T10:25:51.634 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:25:51.684 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T10:25:51.704 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:25:51.727 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T10:25:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:25:51.912 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:25:51.974 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:25:52.066 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:25:52.095 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:25:52.273 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T10:25:52.325 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T10:25:52.353 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:25:52.696 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:25:52.721 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:25:52.775 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:25:52.797 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:25:52.819 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:25:53.032 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:25:53.158 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-05-10T10:25:53.200 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:25:53.259 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T10:25:53.301 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T10:25:53.352 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:25:53.368 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T10:25:53.404 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:25:53.409 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T10:25:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T10:25:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T10:25:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:53.811 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T10:25:53.920 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T10:25:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:25:53.968 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:25:53.998 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:25:54.023 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T10:25:54.070 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T10:25:54.083 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:25:54.112 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:25:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T10:25:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T10:25:54.203 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:25:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T10:25:54.233 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:25:54.272 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T10:25:54.305 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T10:25:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T10:25:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T10:25:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T10:25:54.455 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T10:25:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T10:25:54.486 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T10:25:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:25:54.529 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T10:25:54.539 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:25:54.585 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:25:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:25:54.605 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T10:25:54.625 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:25:54.640 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T10:25:54.675 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:25:54.676 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T10:25:54.676 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T10:25:54.676 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:25:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:25:54.687 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:25:54.709 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:25:54.711 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:25:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T10:25:54.732 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:25:54.781 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T10:25:54.837 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T10:25:54.908 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T10:25:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:25:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T10:25:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T10:25:55.141 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:25:55.282 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:25:55.343 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:25:55.475 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T10:25:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T10:25:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T10:25:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:25:55.789 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T10:25:55.826 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T10:25:55.869 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T10:25:55.886 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T10:25:55.979 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T10:25:56.023 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T10:25:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T10:25:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T10:25:56.079 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T10:25:56.133 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T10:25:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T10:25:56.176 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T10:25:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T10:25:56.219 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T10:25:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T10:25:56.251 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T10:25:56.279 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T10:25:56.283 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T10:25:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T10:25:56.347 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T10:25:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T10:25:56.401 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T10:25:56.408 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T10:25:56.441 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T10:25:56.467 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T10:25:56.476 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T10:25:56.551 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T10:25:56.587 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T10:25:56.611 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T10:25:56.630 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:25:56.648 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T10:25:56.659 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T10:25:56.678 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T10:25:56.720 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T10:25:56.728 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T10:25:56.773 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T10:25:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T10:25:56.847 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T10:25:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T10:25:56.920 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T10:25:56.952 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T10:25:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T10:25:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T10:25:57.042 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T10:25:57.076 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T10:25:57.204 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T10:25:57.234 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T10:25:57.254 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T10:25:57.288 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T10:25:57.328 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T10:25:57.359 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T10:25:57.389 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T10:25:57.423 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T10:25:57.457 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T10:25:57.498 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T10:25:57.533 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T10:25:57.564 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T10:25:57.619 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T10:25:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T10:25:57.664 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T10:25:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T10:25:57.694 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T10:25:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T10:25:57.741 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T10:25:57.765 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T10:25:57.821 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T10:25:57.839 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T10:25:57.888 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T10:25:57.982 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T10:25:58.035 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T10:25:58.089 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T10:25:58.162 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T10:25:58.202 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T10:25:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T10:25:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T10:25:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T10:25:58.373 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T10:25:58.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T10:25:58.550 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T10:25:58.587 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T10:25:58.603 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T10:25:58.617 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T10:25:58.651 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T10:25:58.658 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T10:25:58.685 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T10:25:58.761 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T10:25:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T10:25:58.810 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T10:25:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T10:25:58.850 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T10:25:58.883 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T10:25:58.908 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T10:25:58.930 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T10:25:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T10:25:59.004 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T10:25:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T10:25:59.157 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T10:25:59.182 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T10:25:59.185 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T10:25:59.202 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T10:25:59.224 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T10:25:59.258 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T10:25:59.280 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T10:25:59.298 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T10:25:59.328 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T10:25:59.363 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T10:25:59.396 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T10:25:59.437 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T10:25:59.472 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T10:25:59.557 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T10:25:59.602 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T10:25:59.642 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T10:25:59.689 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T10:25:59.786 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T10:25:59.843 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T10:25:59.940 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T10:26:00.058 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T10:26:00.572 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T10:26:00.624 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T10:26:00.658 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T10:26:00.755 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T10:26:00.807 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T10:26:00.910 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T10:26:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T10:26:01.121 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T10:26:01.158 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T10:26:01.254 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T10:26:01.310 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T10:26:01.389 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T10:26:01.417 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T10:26:01.450 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T10:26:01.775 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T10:26:01.822 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:26:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T10:26:01.919 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:26:01.951 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T10:26:01.988 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T10:26:02.030 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T10:26:02.470 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T10:26:02.508 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:26:02.579 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:26:02.601 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T10:26:03.299 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T10:26:03.386 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T10:26:03.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T10:26:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T10:26:03.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T10:26:03.810 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:26:03.900 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:26:03.932 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T10:26:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T10:26:04.454 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T10:26:04.480 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:26:04.534 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:26:04.557 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T10:26:04.877 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:26:04.906 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:26:05.459 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:26:05.501 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:26:05.592 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:26:05.592 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T10:26:05.592 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T10:26:05.592 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:05.627 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-05-10T10:26:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:26:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:26:06.295 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:06.295 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T10:26:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:06.848 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:26:06.878 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:26:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:26:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:26:07.538 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:26:07.538 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T10:26:07.538 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T10:26:07.538 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:07.583 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:26:07.797 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:26:07.824 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:26:07.824 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:07.824 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T10:26:07.824 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:26:07.824 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:26:07.824 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:08.047 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:26:08.070 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:26:08.071 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:08.071 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T10:26:08.071 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:26:08.075 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:26:08.075 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:08.515 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:26:08.538 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:26:08.538 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:08.538 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T10:26:08.538 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:26:08.538 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:26:08.538 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:08.637 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T10:26:08.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T10:26:08.757 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T10:26:08.944 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T10:26:08.990 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T10:26:09.071 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T10:26:09.160 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T10:26:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T10:26:09.360 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T10:26:09.392 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T10:26:09.419 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T10:26:09.864 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:26:09.893 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:26:09.925 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T10:26:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T10:26:10.016 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:26:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:26:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:26:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:26:12.068 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:26:12.178 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:26:12.337 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T10:26:12.370 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:26:12.393 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:26:12.393 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:12.394 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T10:26:12.394 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:26:12.394 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:26:12.394 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:12.437 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:26:12.448 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:26:12.546 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T10:26:12.600 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T10:26:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T10:26:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:26:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:26:13.580 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:13.580 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T10:26:13.580 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:26:13.580 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:26:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:26:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:26:13.647 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:13.647 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T10:26:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:26:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:26:13.714 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:13.714 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T10:26:13.714 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:26:13.714 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:26:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:26:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:26:14.040 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:14.041 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T10:26:14.041 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:26:14.041 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:26:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-05-10T10:26:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:17.474 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:26:17.490 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:26:17.490 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:17.490 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T10:26:17.491 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:18.696 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T10:26:18.737 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T10:26:18.778 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T10:26:18.810 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T10:26:18.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T10:26:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:26:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:26:18.979 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:18.979 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T10:26:18.979 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:26:18.979 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:26:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:19.018 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T10:26:19.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T10:26:19.113 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T10:26:19.139 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:26:19.139 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-05-10T10:26:19.139 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-05-10T10:26:19.139 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-05-10T10:26:19.139 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-05-10T10:26:19.140 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-05-10T10:26:19.140 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-05-10T10:26:19.140 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-05-10T10:26:19.140 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T10:26:19.189 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T10:26:19.210 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:26:19.234 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:26:19.235 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:19.315 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:26:19.331 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:26:19.332 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:19.332 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T10:26:19.332 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:19.678 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:26:19.702 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:26:19.702 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:19.702 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T10:26:19.702 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:26:19.702 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:26:19.702 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:19.771 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:19.812 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:19.818 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T10:26:19.832 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:19.875 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T10:26:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T10:26:20.115 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T10:26:20.164 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T10:26:20.242 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T10:26:20.331 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T10:26:20.470 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T10:26:20.531 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T10:26:20.564 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T10:26:20.590 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T10:26:20.657 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T10:26:20.742 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T10:26:20.791 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T10:26:20.834 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:26:20.861 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:26:20.861 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:20.862 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T10:26:20.862 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:26:20.862 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:26:20.862 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:20.894 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T10:26:20.928 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T10:26:20.954 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T10:26:20.955 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:26:21.020 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:26:21.047 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:26:21.048 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:26:21.071 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T10:26:21.080 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T10:26:21.080 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:26:21.093 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:26:21.117 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:26:21.117 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:21.117 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T10:26:21.117 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:26:21.117 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:26:21.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:21.133 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T10:26:21.161 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:26:21.165 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:26:21.190 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T10:26:21.195 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:21.195 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:21.375 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:21.376 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:26:21.461 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:26:21.534 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:21.563 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:26:21.585 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:26:21.585 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:21.585 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T10:26:21.585 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:26:21.585 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:26:21.585 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:21.699 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T10:26:21.756 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T10:26:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T10:26:21.997 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T10:26:22.045 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T10:26:22.123 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T10:26:22.209 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:26:22.212 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T10:26:22.238 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:26:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T10:26:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T10:26:22.419 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T10:26:22.447 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T10:26:22.884 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:26:22.912 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:26:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T10:26:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T10:26:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:26:23.208 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:26:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:26:23.359 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:26:23.525 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T10:26:23.549 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:26:23.573 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:26:23.574 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:23.574 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T10:26:23.574 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:26:23.574 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:26:23.574 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:23.625 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:26:23.638 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:26:23.734 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T10:26:23.791 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T10:26:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:26:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:26:24.721 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T10:26:24.753 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:26:24.777 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:26:24.777 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:24.777 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T10:26:24.777 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:26:24.777 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:26:24.777 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:24.820 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:26:24.843 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:26:24.843 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:24.843 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T10:26:24.843 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:24.896 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:26:24.919 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:26:24.919 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:24.919 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T10:26:24.919 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:26:24.919 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:26:24.919 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:25.081 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:26:25.104 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:26:25.216 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:26:25.221 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:26:25.245 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:26:25.245 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:25.245 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T10:26:25.245 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:26:25.245 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:26:25.245 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T10:26:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:26:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:26:25.447 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:25.447 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T10:26:25.447 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:26:25.447 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:26:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:26:25.510 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:26:25.608 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T10:26:25.664 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T10:26:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T10:26:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:26:26.658 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:26:26.658 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:26.658 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T10:26:26.658 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:26:26.658 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:26:26.659 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:26.701 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:26:26.724 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:26:26.724 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:26.725 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T10:26:26.725 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:26.769 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:26:26.792 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:26:26.792 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:26.792 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T10:26:26.792 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:26:26.792 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:26:26.792 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:27.103 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:26:27.127 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:26:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:26:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T10:26:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:26:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:26:27.127 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:27.202 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:26:27.203 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T10:26:27.204 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T10:26:27.205 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T10:26:27.206 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T10:26:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T10:26:27.208 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T10:26:27.209 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T10:26:27.210 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T10:26:27.210 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T10:26:27.210 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T10:26:27.210 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T10:26:27.210 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T10:26:27.211 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T10:26:27.212 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T10:26:27.213 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T10:26:27.214 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T10:26:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T10:26:27.216 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T10:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-05-10T10:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T10:26:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T10:26:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T10:26:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T10:26:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T10:26:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T10:26:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T10:26:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T10:26:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T10:26:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T10:26:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T10:26:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:26:29.802 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-05-10T10:26:29.876 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T10:26:29.916 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T10:26:29.965 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T10:26:29.997 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T10:26:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T10:26:30.209 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T10:26:30.215 DEBUG:teuthology.parallel:result is None 2024-05-10T10:26:30.246 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T10:26:30.276 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T10:26:30.306 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T10:26:30.344 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T10:26:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:31.774 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T10:26:31.814 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T10:26:31.845 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T10:26:31.858 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T10:26:31.887 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T10:26:31.914 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T10:26:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T10:26:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T10:26:32.054 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T10:26:32.083 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T10:26:32.108 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T10:26:32.109 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T10:26:32.109 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:26:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T10:26:32.182 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T10:26:32.201 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:26:32.220 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T10:26:32.234 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T10:26:32.234 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:26:32.283 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T10:26:32.302 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:26:32.327 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T10:26:32.333 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:32.333 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:32.521 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:32.521 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:26:32.589 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:26:32.663 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:26:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T10:26:33.877 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T10:26:33.924 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T10:26:34.026 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T10:26:34.063 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T10:26:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T10:26:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:26:34.174 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:26:34.207 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T10:26:34.207 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:26:34.292 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:26:34.316 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T10:26:34.322 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:34.322 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:34.526 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:26:34.527 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:26:34.620 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:26:34.692 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:38.193 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:26:38.194 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T10:26:38.195 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T10:26:38.196 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T10:26:38.196 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T10:26:38.196 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T10:26:38.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T10:26:38.210 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T10:26:38.211 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T10:26:38.212 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T10:26:38.213 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T10:26:38.214 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T10:26:38.215 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T10:26:38.216 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T10:26:38.217 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T10:26:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T10:26:38.219 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T10:26:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T10:26:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T10:26:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T10:26:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T10:26:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T10:26:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T10:26:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T10:26:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T10:26:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T10:26:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T10:26:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T10:26:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.551 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.552 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.553 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T10:26:40.554 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.555 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T10:26:40.556 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T10:26:40.557 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T10:26:40.558 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T10:26:40.559 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T10:26:40.560 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T10:26:40.561 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:40.562 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:26:40.563 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-05-10T10:26:40.976 DEBUG:teuthology.parallel:result is None 2024-05-10T10:26:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:26:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T10:26:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T10:26:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T10:26:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T10:26:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T10:26:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T10:26:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T10:26:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T10:26:43.010 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T10:26:43.011 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T10:26:43.012 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T10:26:43.012 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T10:26:43.012 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T10:26:43.012 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T10:26:43.012 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-05-10T10:26:43.012 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-05-10T10:26:43.444 DEBUG:teuthology.parallel:result is None 2024-05-10T10:26:43.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:26:43.586 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T10:26:43.612 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T10:26:43.613 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T10:26:43.613 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T10:26:43.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:26:43.750 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T10:26:43.774 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T10:26:43.774 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T10:26:43.774 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T10:26:43.776 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:26:43.911 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T10:26:43.937 INFO:teuthology.orchestra.run.smithi195.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T10:26:43.937 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T10:26:43.937 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T10:26:43.939 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T10:26:43.939 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:26:43.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T10:26:43.972 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:26:43.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T10:26:44.002 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:26:44.002 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T10:26:44.033 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T10:26:44.034 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:26:44.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T10:26:44.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T10:26:44.133 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:26:44.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T10:26:44.161 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T10:26:44.228 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:26:44.228 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T10:26:44.258 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T10:26:44.327 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T10:26:44.327 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:26:44.327 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T10:26:44.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T10:26:44.419 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:26:44.419 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T10:26:44.445 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T10:26:44.511 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:26:44.512 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T10:26:44.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T10:26:44.609 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T10:26:44.610 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:26:44.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T10:26:44.635 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T10:26:44.700 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:26:44.700 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T10:26:44.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T10:26:44.794 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:26:44.794 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T10:26:44.823 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T10:26:44.892 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-10T10:26:44.997 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T10:26:44.997 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:26:44.997 INFO:tasks.cephadm:Cluster fsid is ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:26:44.997 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-10T10:26:44.998 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.112', 'mon.b': '172.21.15.148', 'mon.c': '172.21.15.195'} 2024-05-10T10:26:44.998 INFO:tasks.cephadm:First mon is mon.a on smithi112 2024-05-10T10:26:44.998 INFO:tasks.cephadm:First mgr is x 2024-05-10T10:26:44.998 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-10T10:26:44.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-05-10T10:26:45.024 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-05-10T10:26:45.051 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname $(hostname -s) 2024-05-10T10:26:45.081 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-10T10:26:45.081 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:26:45.218 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3449-gd39fa088', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3449.gd39fa088'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-10 00:52:03.112070', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-63896-from-wip-rishabh-testing-20240501.193033', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-10T10:26:45.360 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-63896-from-wip-rishabh-testing-20240501.193033, sha1 d39fa08867836ac070c47cb66c782fa559fd480b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:26:45.362 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm 2024-05-10T10:26:45.362 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:45.765 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 782738 May 10 10:26 /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:45.765 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:46.291 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 782738 May 10 10:26 /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:46.291 DEBUG:teuthology.orchestra.run.smithi195:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:46.805 INFO:teuthology.orchestra.run.smithi195.stdout:-rw-r--r--. 1 ubuntu ubuntu 782738 May 10 10:26 /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:46.806 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:46.825 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:46.845 DEBUG:teuthology.orchestra.run.smithi195:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T10:26:46.887 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b on all hosts... 2024-05-10T10:26:46.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b pull 2024-05-10T10:26:46.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b pull 2024-05-10T10:26:46.892 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b pull 2024-05-10T10:26:48.432 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T10:26:48.535 INFO:teuthology.orchestra.run.smithi195.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T10:26:48.571 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T10:27:21.431 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:27:21.432 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T10:27:21.432 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5", 2024-05-10T10:27:21.432 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-05-10T10:27:21.432 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f" 2024-05-10T10:27:21.432 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:27:21.432 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:27:24.840 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-05-10T10:27:24.841 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T10:27:24.841 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5", 2024-05-10T10:27:24.841 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-05-10T10:27:24.841 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f" 2024-05-10T10:27:24.841 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-05-10T10:27:24.841 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-05-10T10:27:25.705 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-10T10:27:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T10:27:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: "image_id": "ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5", 2024-05-10T10:27:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: "repo_digests": [ 2024-05-10T10:27:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f" 2024-05-10T10:27:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-05-10T10:27:25.706 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-10T10:27:25.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-05-10T10:27:25.764 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-05-10T10:27:25.798 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph 2024-05-10T10:27:25.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-05-10T10:27:25.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-05-10T10:27:25.885 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /etc/ceph 2024-05-10T10:27:25.909 INFO:tasks.cephadm:Writing seed config... 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-10T10:27:25.911 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-10T10:27:25.912 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-10T10:27:25.913 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-10T10:27:25.914 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-10T10:27:25.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:27:25.914 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-10T10:27:25.932 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ccd347ac-0eb7-11ef-bc98-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-10T10:27:25.932 DEBUG:teuthology.orchestra.run.smithi112:mon.a> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a.service 2024-05-10T10:27:25.974 DEBUG:teuthology.orchestra.run.smithi112:mgr.x> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.x.service 2024-05-10T10:27:26.018 INFO:tasks.cephadm:Bootstrapping... 2024-05-10T10:27:26.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b -v bootstrap --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:27:26.209 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:27:26.210 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b', '-v', 'bootstrap', '--fsid', 'ccd347ac-0eb7-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-05-10T10:27:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-10T10:27:26.210 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-05-10T10:27:26.235 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-10T10:27:26.235 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-05-10T10:27:26.235 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-05-10T10:27:26.242 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T10:27:26.242 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T10:27:26.248 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T10:27:26.248 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-10T10:27:26.255 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-05-10T10:27:26.261 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-05-10T10:27:26.261 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-05-10T10:27:26.261 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-05-10T10:27:26.286 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-10T10:27:26.286 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-10T10:27:26.286 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-05-10T10:27:26.286 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-05-10T10:27:26.293 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T10:27:26.293 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T10:27:26.299 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T10:27:26.299 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-10T10:27:26.305 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-05-10T10:27:26.312 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-05-10T10:27:26.312 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-05-10T10:27:26.312 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-05-10T10:27:26.312 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:26.312 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 139699316178368 on /run/cephadm/ccd347ac-0eb7-11ef-bc98-c7b262605968.lock 2024-05-10T10:27:26.313 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 139699316178368 acquired on /run/cephadm/ccd347ac-0eb7-11ef-bc98-c7b262605968.lock 2024-05-10T10:27:26.313 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-05-10T10:27:26.313 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-05-10T10:27:26.313 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-05-10T10:27:26.316 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-05-10T10:27:26.316 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-05-10T10:27:26.319 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-10T10:27:26.319 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-10T10:27:26.319 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1779sec hoplimit 64 pref medium 2024-05-10T10:27:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-10T10:27:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-10T10:27:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T10:27:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-10T10:27:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-05-10T10:27:26.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T10:27:26.323 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-05-10T10:27:26.323 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-05-10T10:27:26.323 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-10T10:27:26.323 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-10T10:27:26.324 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T10:27:26.858 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 2024-05-10T10:27:26.858 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T10:27:26.858 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-05-10T10:27:26.858 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-10T10:27:26.858 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:5401d9dfdf7b5b1eb30ca8a50398217fe5e68f9c8026e432d9728743b160edef 2024-05-10T10:27:26.858 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 2024-05-10T10:27:26.859 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-10T10:27:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev) 2024-05-10T10:27:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev) 2024-05-10T10:27:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-05-10T10:27:28.628 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-05-10T10:27:28.628 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-05-10T10:27:29.433 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCQ9j1mDc1rOBAADrLyDKrlCadCxNU+wxpkkw== 2024-05-10T10:27:30.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCR9j1mvHHWLBAACnW6FyEA6tMOo7ovTaVmkA== 2024-05-10T10:27:31.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCS9j1mkhHIKhAA/+hPwhuWu+qy4GlUC4t6cw== 2024-05-10T10:27:31.415 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-05-10T10:27:32.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T10:27:32.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-10T10:27:32.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:32.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T10:27:32.345 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for a [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T10:27:32.346 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-05-10T10:27:32.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:32.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T10:27:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:27:32.346 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-05-10T10:27:33.342 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.a on 2024-05-10T10:27:33.548 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-10T10:27:33.688 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-10T10:27:33.862 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968.target. 2024-05-10T10:27:33.862 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968.target. 2024-05-10T10:27:34.130 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a 2024-05-10T10:27:34.130 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-10T10:27:34.305 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968.target.wants/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@.service. 2024-05-10T10:27:34.370 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 systemd[1]: Starting Ceph mon.a for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:27:34.645 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 podman[29702]: 2024-05-10 10:27:34.541249653 +0000 UTC m=+0.096341969 container create 4be866111497449861252da1fa4e5c0345430896d616e7c0db6b15c6cee010bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5) 2024-05-10T10:27:34.646 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 podman[29702]: 2024-05-10 10:27:34.462368751 +0000 UTC m=+0.017461086 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:27:34.833 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T10:27:34.833 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-10T10:27:34.840 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T10:27:34.840 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-10T10:27:34.840 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-10T10:27:34.840 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T10:27:34.840 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-05-10T10:27:34.841 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-05-10T10:27:34.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 podman[29702]: 2024-05-10 10:27:34.751820953 +0000 UTC m=+0.306913270 container init 4be866111497449861252da1fa4e5c0345430896d616e7c0db6b15c6cee010bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS) 2024-05-10T10:27:34.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 podman[29702]: 2024-05-10 10:27:34.755207971 +0000 UTC m=+0.310300292 container start 4be866111497449861252da1fa4e5c0345430896d616e7c0db6b15c6cee010bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-10T10:27:34.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 ceph-mon[29728]: mkfs ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:34.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 ceph-mon[29728]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T10:27:34.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 bash[29702]: 4be866111497449861252da1fa4e5c0345430896d616e7c0db6b15c6cee010bf 2024-05-10T10:27:34.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:34 smithi112 systemd[1]: Started Ceph mon.a for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: monmap epoch 1 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: last_changed 2024-05-10T10:27:31.745764+0000 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: min_mon_release 19 (squid) 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: election_strategy: 1 2024-05-10T10:27:35.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:27:35.910 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: fsmap 2024-05-10T10:27:35.910 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T10:27:35.910 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: mgrmap e1: no daemons active 2024-05-10T10:27:35.910 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:35 smithi112 ceph-mon[29728]: from='client.? 172.21.15.112:0/2447757131' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-10T10:27:35.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.515895s) 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-05-10T10:27:35.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.236+0000 7f77bcf23700 1 Processor -- start 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.236+0000 7f77bcf23700 1 -- start start 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.236+0000 7f77bcf23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b810bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.236+0000 7f77bcf23700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77b807ce50 con 0x7f77b81096e0 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b810bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b810bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54138/0 (socket says 172.21.15.112:54138) 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b659c700 1 -- 172.21.15.112:0/687564259 learned_addr learned my addr 172.21.15.112:0/687564259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b659c700 1 -- 172.21.15.112:0/687564259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b810c010 con 0x7f77b81096e0 2024-05-10T10:27:35.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b659c700 1 --2- 172.21.15.112:0/687564259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b810bad0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77a80098a0 tx=0x7f77a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e22eaa6f452526 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b559a700 1 -- 172.21.15.112:0/687564259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77a8004160 con 0x7f77b81096e0 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b559a700 1 -- 172.21.15.112:0/687564259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a8004300 con 0x7f77b81096e0 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.237+0000 7f77b559a700 1 -- 172.21.15.112:0/687564259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77a80036a0 con 0x7f77b81096e0 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.238+0000 7f77bcf23700 1 -- 172.21.15.112:0/687564259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 msgr2=0x7f77b810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.238+0000 7f77bcf23700 1 --2- 172.21.15.112:0/687564259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b810bad0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77a80098a0 tx=0x7f77a8009bb0 comp rx=0 tx=0).stop 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.238+0000 7f77bcf23700 1 -- 172.21.15.112:0/687564259 shutdown_connections 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.238+0000 7f77bcf23700 1 --2- 172.21.15.112:0/687564259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b810bad0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.238+0000 7f77bcf23700 1 -- 172.21.15.112:0/687564259 >> 172.21.15.112:0/687564259 conn(0x7f77b81030f0 msgr2=0x7f77b8105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.238+0000 7f77bcf23700 1 -- 172.21.15.112:0/687564259 shutdown_connections 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.238+0000 7f77bcf23700 1 -- 172.21.15.112:0/687564259 wait complete. 2024-05-10T10:27:35.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.239+0000 7f77bcf23700 1 Processor -- start 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.239+0000 7f77bcf23700 1 -- start start 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.239+0000 7f77bcf23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.239+0000 7f77bcf23700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77b810c480 con 0x7f77b81096e0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.239+0000 7f77b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.239+0000 7f77b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54150/0 (socket says 172.21.15.112:54150) 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.239+0000 7f77b659c700 1 -- 172.21.15.112:0/2447757131 learned_addr learned my addr 172.21.15.112:0/2447757131 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77b659c700 1 -- 172.21.15.112:0/2447757131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b819c890 con 0x7f77b81096e0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77b659c700 1 --2- 172.21.15.112:0/2447757131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b819c350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009fb0 tx=0x7f77a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77a77fe700 1 -- 172.21.15.112:0/2447757131 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77a8003de0 con 0x7f77b81096e0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77a77fe700 1 -- 172.21.15.112:0/2447757131 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a8011630 con 0x7f77b81096e0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b819cb80 con 0x7f77b81096e0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77a77fe700 1 -- 172.21.15.112:0/2447757131 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77a8011930 con 0x7f77b81096e0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b819f7c0 con 0x7f77b81096e0 2024-05-10T10:27:35.921 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77a77fe700 1 -- 172.21.15.112:0/2447757131 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77a8024480 con 0x7f77b81096e0 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.240+0000 7f77a77fe700 1 -- 172.21.15.112:0/2447757131 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77a801a7a0 con 0x7f77b81096e0 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.241+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b8064b00 con 0x7f77b81096e0 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.243+0000 7f77a77fe700 1 -- 172.21.15.112:0/2447757131 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f77a801aa40 con 0x7f77b81096e0 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.313+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f77b802e720 con 0x7f77b81096e0 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.314+0000 7f77a77fe700 1 -- 172.21.15.112:0/2447757131 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f77a801ac20 con 0x7f77b81096e0 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.315+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 msgr2=0x7f77b819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.315+0000 7f77bcf23700 1 --2- 172.21.15.112:0/2447757131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b819c350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009fb0 tx=0x7f77a8004750 comp rx=0 tx=0).stop 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.315+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 shutdown_connections 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.315+0000 7f77bcf23700 1 --2- 172.21.15.112:0/2447757131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77b81096e0 0x7f77b819c350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.315+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 >> 172.21.15.112:0/2447757131 conn(0x7f77b81030f0 msgr2=0x7f77b8105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.315+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 shutdown_connections 2024-05-10T10:27:35.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:35.315+0000 7f77bcf23700 1 -- 172.21.15.112:0/2447757131 wait complete. 2024-05-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-05-10T10:27:35.923 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:36.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.301+0000 7f970842e700 1 Processor -- start 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.301+0000 7f970842e700 1 -- start start 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.301+0000 7f970842e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f97001084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.301+0000 7f970842e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9700108a70 con 0x7f9700108080 2024-05-10T10:27:36.908 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.301+0000 7f97061ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f97001084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.301+0000 7f97061ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f97001084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59646/0 (socket says 172.21.15.112:59646) 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.301+0000 7f97061ca700 1 -- 172.21.15.112:0/109417101 learned_addr learned my addr 172.21.15.112:0/109417101 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f97061ca700 1 -- 172.21.15.112:0/109417101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9700108da0 con 0x7f9700108080 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f97061ca700 1 --2- 172.21.15.112:0/109417101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f97001084a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00bb40 tx=0x7f96fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbc6588d66efe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f97051c8700 1 -- 172.21.15.112:0/109417101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96fc00dbe0 con 0x7f9700108080 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f97051c8700 1 -- 172.21.15.112:0/109417101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc00dd80 con 0x7f9700108080 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f97051c8700 1 -- 172.21.15.112:0/109417101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96fc004180 con 0x7f9700108080 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f970842e700 1 -- 172.21.15.112:0/109417101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 msgr2=0x7f97001084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f970842e700 1 --2- 172.21.15.112:0/109417101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f97001084a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00bb40 tx=0x7f96fc00be50 comp rx=0 tx=0).stop 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f970842e700 1 -- 172.21.15.112:0/109417101 shutdown_connections 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f970842e700 1 --2- 172.21.15.112:0/109417101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f97001084a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f970842e700 1 -- 172.21.15.112:0/109417101 >> 172.21.15.112:0/109417101 conn(0x7f9700102920 msgr2=0x7f9700104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:36.909 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f970842e700 1 -- 172.21.15.112:0/109417101 shutdown_connections 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.302+0000 7f970842e700 1 -- 172.21.15.112:0/109417101 wait complete. 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f970842e700 1 Processor -- start 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f970842e700 1 -- start start 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f970842e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f970019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f970842e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9700109210 con 0x7f9700108080 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f97061ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f970019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f97061ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f970019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59662/0 (socket says 172.21.15.112:59662) 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f97061ca700 1 -- 172.21.15.112:0/641449652 learned_addr learned my addr 172.21.15.112:0/641449652 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.303+0000 7f97061ca700 1 -- 172.21.15.112:0/641449652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970019a5d0 con 0x7f9700108080 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f97061ca700 1 --2- 172.21.15.112:0/641449652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f970019a090 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b340 tx=0x7f96fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96fc029070 con 0x7f9700108080 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc0044e0 con 0x7f9700108080 2024-05-10T10:27:36.910 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970019a860 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96fc0105a0 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970019d550 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f96fc02c070 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.304+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96fc021070 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.305+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9700064b00 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.307+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f96fc015070 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.375+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f970002e660 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.378+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f96fc00dce0 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.379+0000 7f96f77fe700 1 -- 172.21.15.112:0/641449652 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f96fc010bb0 con 0x7f9700108080 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.380+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 msgr2=0x7f970019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.380+0000 7f970842e700 1 --2- 172.21.15.112:0/641449652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f970019a090 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b340 tx=0x7f96fc004750 comp rx=0 tx=0).stop 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.380+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 shutdown_connections 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.380+0000 7f970842e700 1 --2- 172.21.15.112:0/641449652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9700108080 0x7f970019a090 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:36.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.380+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 >> 172.21.15.112:0/641449652 conn(0x7f9700102920 msgr2=0x7f9700190d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:36.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.380+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 shutdown_connections 2024-05-10T10:27:36.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:36.380+0000 7f970842e700 1 -- 172.21.15.112:0/641449652 wait complete. 2024-05-10T10:27:36.912 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-05-10T10:27:37.023 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:36 smithi112 ceph-mon[29728]: from='client.? 172.21.15.112:0/641449652' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T10:27:37.023 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:36 smithi112 ceph-mon[29728]: from='client.? 172.21.15.112:0/641449652' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-10T10:27:37.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.314+0000 7f95fe159700 1 Processor -- start 2024-05-10T10:27:37.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.314+0000 7f95fe159700 1 -- start start 2024-05-10T10:27:37.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.314+0000 7f95fe159700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f8107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.314+0000 7f95fe159700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81085b0 con 0x7f95f8107bc0 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f8107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f8107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59666/0 (socket says 172.21.15.112:59666) 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f77fe700 1 -- 172.21.15.112:0/1067481702 learned_addr learned my addr 172.21.15.112:0/1067481702 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f77fe700 1 -- 172.21.15.112:0/1067481702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f8108dc0 con 0x7f95f8107bc0 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f77fe700 1 --2- 172.21.15.112:0/1067481702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f8107fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009e80 tx=0x7f95e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993919e282c2cbaf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f67fc700 1 -- 172.21.15.112:0/1067481702 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e8017070 con 0x7f95f8107bc0 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f67fc700 1 -- 172.21.15.112:0/1067481702 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f95e800b8c0 con 0x7f95f8107bc0 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.315+0000 7f95f67fc700 1 -- 172.21.15.112:0/1067481702 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e8004420 con 0x7f95f8107bc0 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 -- 172.21.15.112:0/1067481702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 msgr2=0x7f95f8107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 --2- 172.21.15.112:0/1067481702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f8107fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009e80 tx=0x7f95e800b330 comp rx=0 tx=0).stop 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 -- 172.21.15.112:0/1067481702 shutdown_connections 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 --2- 172.21.15.112:0/1067481702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f8107fe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:37.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 -- 172.21.15.112:0/1067481702 >> 172.21.15.112:0/1067481702 conn(0x7f95f8103200 msgr2=0x7f95f8105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 -- 172.21.15.112:0/1067481702 shutdown_connections 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 -- 172.21.15.112:0/1067481702 wait complete. 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.316+0000 7f95fe159700 1 Processor -- start 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95fe159700 1 -- start start 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95fe159700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95fe159700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95f810bff0 con 0x7f95f8107bc0 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59670/0 (socket says 172.21.15.112:59670) 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f77fe700 1 -- 172.21.15.112:0/2222448237 learned_addr learned my addr 172.21.15.112:0/2222448237 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f77fe700 1 -- 172.21.15.112:0/2222448237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f819cff0 con 0x7f95f8107bc0 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f77fe700 1 --2- 172.21.15.112:0/2222448237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f819cab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f95e8004000 tx=0x7f95e8003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f4ff9700 1 -- 172.21.15.112:0/2222448237 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e8020020 con 0x7f95f8107bc0 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f4ff9700 1 -- 172.21.15.112:0/2222448237 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f95e80128a0 con 0x7f95f8107bc0 2024-05-10T10:27:37.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f819d280 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.317+0000 7f95f4ff9700 1 -- 172.21.15.112:0/2222448237 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e8017040 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.318+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f819ff70 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.318+0000 7f95f4ff9700 1 -- 172.21.15.112:0/2222448237 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f95e8023020 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.318+0000 7f95f4ff9700 1 -- 172.21.15.112:0/2222448237 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f95e800f050 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.318+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f8064b00 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.321+0000 7f95f4ff9700 1 -- 172.21.15.112:0/2222448237 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f95e802a020 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.388+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f95f802e6e0 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.388+0000 7f95f4ff9700 1 -- 172.21.15.112:0/2222448237 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f95e800b860 con 0x7f95f8107bc0 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.389+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 msgr2=0x7f95f819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.389+0000 7f95fe159700 1 --2- 172.21.15.112:0/2222448237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f819cab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f95e8004000 tx=0x7f95e8003ad0 comp rx=0 tx=0).stop 2024-05-10T10:27:37.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.389+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 shutdown_connections 2024-05-10T10:27:37.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.389+0000 7f95fe159700 1 --2- 172.21.15.112:0/2222448237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f8107bc0 0x7f95f819cab0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:37.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.389+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 >> 172.21.15.112:0/2222448237 conn(0x7f95f8103200 msgr2=0x7f95f8104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:37.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.389+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 shutdown_connections 2024-05-10T10:27:37.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:37.389+0000 7f95fe159700 1 -- 172.21.15.112:0/2222448237 wait complete. 2024-05-10T10:27:37.939 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-05-10T10:27:37.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:37 smithi112 ceph-mon[29728]: from='client.? 172.21.15.112:0/2222448237' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:27:38.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:37 smithi112 systemd[1]: Stopping Ceph mon.a for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:27:38.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a[29724]: 2024-05-10T10:27:38.206+0000 7f077a688700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:27:38.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a[29724]: 2024-05-10T10:27:38.206+0000 7f077a688700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-10T10:27:38.648 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 podman[29974]: 2024-05-10 10:27:38.339383776 +0000 UTC m=+0.238689277 container died 4be866111497449861252da1fa4e5c0345430896d616e7c0db6b15c6cee010bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:27:38.648 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 podman[29974]: 2024-05-10 10:27:38.647567291 +0000 UTC m=+0.546872796 container cleanup 4be866111497449861252da1fa4e5c0345430896d616e7c0db6b15c6cee010bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:27:38.957 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 bash[29974]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a 2024-05-10T10:27:38.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 podman[29988]: 2024-05-10 10:27:38.856567249 +0000 UTC m=+0.514529667 container remove 4be866111497449861252da1fa4e5c0345430896d616e7c0db6b15c6cee010bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:27:39.297 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-10T10:27:39.297 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 systemd[1]: Stopped Ceph mon.a for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:27:39.297 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:38 smithi112 systemd[1]: Starting Ceph mon.a for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:27:39.297 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 podman[30080]: 2024-05-10 10:27:39.093199463 +0000 UTC m=+0.018458666 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:27:39.297 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 podman[30080]: 2024-05-10 10:27:39.190471233 +0000 UTC m=+0.115730420 container create 41fabb039a5b7ad4eb696899bf0e1d3b195e4ecba2544fbdd001eac4f48a1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:27:39.488 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-10T10:27:39.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 podman[30080]: 2024-05-10 10:27:39.384726585 +0000 UTC m=+0.309985768 container init 41fabb039a5b7ad4eb696899bf0e1d3b195e4ecba2544fbdd001eac4f48a1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:27:39.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 podman[30080]: 2024-05-10 10:27:39.387807554 +0000 UTC m=+0.313066741 container start 41fabb039a5b7ad4eb696899bf0e1d3b195e4ecba2544fbdd001eac4f48a1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev), process ceph-mon, pid 2 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: pidfile_write: ignore empty --pid-file 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: load: jerasure load: lrc 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: RocksDB version: 7.9.2 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Git sha 0 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Compile date 2024-05-10 01:38:55 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: DB SUMMARY 2024-05-10T10:27:39.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: DB Session ID: 5RSXY0QW14DLBOB8I6UI 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: CURRENT file: CURRENT 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103560 ; 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.error_if_exists: 0 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.create_if_missing: 0 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.paranoid_checks: 1 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T10:27:39.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.env: 0x565103883cc0 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.info_log: 0x565105017da0 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.statistics: (nil) 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.use_fsync: 0 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_log_file_size: 0 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.allow_fallocate: 1 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T10:27:39.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.use_direct_reads: 0 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.db_log_dir: 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.wal_dir: 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T10:27:39.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.write_buffer_manager: 0x565104fe90e0 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.unordered_write: 0 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T10:27:39.585 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T10:27:39.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.row_cache: None 2024-05-10T10:27:39.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.wal_filter: None 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.two_write_queues: 0 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.wal_compression: 0 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.atomic_flush: 0 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T10:27:39.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.log_readahead_size: 0 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_background_jobs: 2 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_background_compactions: -1 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_subcompactions: 1 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T10:27:39.588 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_open_files: -1 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_background_flushes: -1 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Compression algorithms supported: 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T10:27:39.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kZSTD supported: 0 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kXpressCompression supported: 0 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kZlibCompression supported: 1 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kSnappyCompression supported: 1 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kLZ4Compression supported: 1 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: kBZip2Compression supported: 0 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.merge_operator: 2024-05-10T10:27:39.590 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_filter: None 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x565105017f60) 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_type: 0 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_index_type: 0 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_shortening: 1 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T10:27:39.591 INFO:journalctl@ceph.mon.a.smithi112.stdout: checksum: 4 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: no_block_cache: 0 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache: 0x56510501b1f0 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_options: 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: capacity : 536870912 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_shard_bits : 4 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: strict_capacity_limit : 0 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_compressed: (nil) 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: persistent_cache: (nil) 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size: 4096 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size_deviation: 10 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_restart_interval: 16 2024-05-10T10:27:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_block_restart_interval: 1 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: metadata_block_size: 4096 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: partition_filters: 0 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: use_delta_encoding: 1 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: filter_policy: bloomfilter 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: whole_key_filtering: 1 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: verify_compression: 0 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: format_version: 5 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_index_compression: 1 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_align: 0 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: max_auto_readahead_size: 262144 2024-05-10T10:27:39.593 INFO:journalctl@ceph.mon.a.smithi112.stdout: prepopulate_block_cache: 0 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout: initial_auto_readahead_size: 8192 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression: NoCompression 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.num_levels: 7 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T10:27:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T10:27:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T10:27:39.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T10:27:39.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T10:27:39.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T10:27:39.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T10:27:39.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T10:27:39.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T10:27:39.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T10:27:39.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T10:27:39.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.inplace_update_support: 0 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T10:27:39.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.bloom_locality: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.max_successive_merges: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.ttl: 2592000 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T10:27:39.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.enable_blob_files: false 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.min_blob_size: 0 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T10:27:39.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T10:27:39.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-10T10:27:39.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-10T10:27:39.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4a23ddf1-1706-4c25-bf1a-5a2397b5b2af 2024-05-10T10:27:39.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715336859419134, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-10T10:27:39.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-10T10:27:39.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715336859420149, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96793, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13784, "raw_average_key_size": 49, "raw_value_size": 89251, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715336859, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4a23ddf1-1706-4c25-bf1a-5a2397b5b2af", "db_session_id": "5RSXY0QW14DLBOB8I6UI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-10T10:27:39.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715336859420228, "job": 1, "event": "recovery_finished"} 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565105138000 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: DB pointer 0x565105124000 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** DB Stats ** 2024-05-10T10:27:39.604 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T10:27:39.605 INFO:journalctl@ceph.mon.a.smithi112.stdout: L0 2/0 98.38 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: Sum 2/0 98.38 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T10:27:39.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative compaction: 0.00 GB write, 30.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval compaction: 0.00 GB write, 30.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache BinnedLRUCache@0x56510501b1f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: starting mon.a rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???) e1 preinit fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:39.607 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???).mds e1 new map 2024-05-10T10:27:39.608 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???).mds e1 print_map 2024-05-10T10:27:39.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: e1 2024-05-10T10:27:39.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T10:27:39.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T10:27:39.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: legacy client fscid: -1 2024-05-10T10:27:39.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-10T10:27:39.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: No filesystems configured 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: monmap epoch 1 2024-05-10T10:27:39.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: last_changed 2024-05-10T10:27:31.745764+0000 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: min_mon_release 19 (squid) 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: election_strategy: 1 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: fsmap 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T10:27:39.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 ceph-mon[30100]: mgrmap e1: no daemons active 2024-05-10T10:27:39.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 bash[30080]: 41fabb039a5b7ad4eb696899bf0e1d3b195e4ecba2544fbdd001eac4f48a1236 2024-05-10T10:27:39.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:39 smithi112 systemd[1]: Started Ceph mon.a for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:27:40.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.005+0000 7fe616e83700 1 Processor -- start 2024-05-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.005+0000 7fe616e83700 1 -- start start 2024-05-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.005+0000 7fe616e83700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61010bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.005+0000 7fe616e83700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe61007cd90 con 0x7fe6101097a0 2024-05-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe614c1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61010bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe614c1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61010bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59680/0 (socket says 172.21.15.112:59680) 2024-05-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe614c1f700 1 -- 172.21.15.112:0/804192335 learned_addr learned my addr 172.21.15.112:0/804192335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe614c1f700 1 -- 172.21.15.112:0/804192335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61010c0d0 con 0x7fe6101097a0 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe614c1f700 1 --2- 172.21.15.112:0/804192335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61010bb90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009e80 tx=0x7fe60c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc12ab632b48c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe6077fe700 1 -- 172.21.15.112:0/804192335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c017070 con 0x7fe6101097a0 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe6077fe700 1 -- 172.21.15.112:0/804192335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe60c00b8c0 con 0x7fe6101097a0 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.006+0000 7fe6077fe700 1 -- 172.21.15.112:0/804192335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c004420 con 0x7fe6101097a0 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.007+0000 7fe616e83700 1 -- 172.21.15.112:0/804192335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 msgr2=0x7fe61010bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.007+0000 7fe616e83700 1 --2- 172.21.15.112:0/804192335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61010bb90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009e80 tx=0x7fe60c00b330 comp rx=0 tx=0).stop 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.007+0000 7fe616e83700 1 -- 172.21.15.112:0/804192335 shutdown_connections 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.007+0000 7fe616e83700 1 --2- 172.21.15.112:0/804192335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61010bb90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.007+0000 7fe616e83700 1 -- 172.21.15.112:0/804192335 >> 172.21.15.112:0/804192335 conn(0x7fe6101030f0 msgr2=0x7fe610105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.007+0000 7fe616e83700 1 -- 172.21.15.112:0/804192335 shutdown_connections 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.007+0000 7fe616e83700 1 -- 172.21.15.112:0/804192335 wait complete. 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe616e83700 1 Processor -- start 2024-05-10T10:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe616e83700 1 -- start start 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe616e83700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe616e83700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe61010cf00 con 0x7fe6101097a0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe614c1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe614c1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59692/0 (socket says 172.21.15.112:59692) 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe614c1f700 1 -- 172.21.15.112:0/3471245164 learned_addr learned my addr 172.21.15.112:0/3471245164 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.008+0000 7fe614c1f700 1 -- 172.21.15.112:0/3471245164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61019d000 con 0x7fe6101097a0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe614c1f700 1 --2- 172.21.15.112:0/3471245164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61019cac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009e50 tx=0x7fe60c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe605ffb700 1 -- 172.21.15.112:0/3471245164 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c020020 con 0x7fe6101097a0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe605ffb700 1 -- 172.21.15.112:0/3471245164 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe60c012430 con 0x7fe6101097a0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61019d290 con 0x7fe6101097a0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe605ffb700 1 -- 172.21.15.112:0/3471245164 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c017040 con 0x7fe6101097a0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61019ff80 con 0x7fe6101097a0 2024-05-10T10:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe605ffb700 1 -- 172.21.15.112:0/3471245164 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe60c023020 con 0x7fe6101097a0 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.009+0000 7fe605ffb700 1 -- 172.21.15.112:0/3471245164 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe60c00f050 con 0x7fe6101097a0 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.010+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe610064b00 con 0x7fe6101097a0 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.012+0000 7fe605ffb700 1 -- 172.21.15.112:0/3471245164 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe60c02a020 con 0x7fe6101097a0 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.078+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe6101a0150 con 0x7fe6101097a0 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.080+0000 7fe605ffb700 1 -- 172.21.15.112:0/3471245164 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe60c00b860 con 0x7fe6101097a0 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.081+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 msgr2=0x7fe61019cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.081+0000 7fe616e83700 1 --2- 172.21.15.112:0/3471245164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61019cac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009e50 tx=0x7fe60c00bd50 comp rx=0 tx=0).stop 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.081+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 shutdown_connections 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.081+0000 7fe616e83700 1 --2- 172.21.15.112:0/3471245164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6101097a0 0x7fe61019cac0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.081+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 >> 172.21.15.112:0/3471245164 conn(0x7fe6101030f0 msgr2=0x7fe610103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.081+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 shutdown_connections 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:40.081+0000 7fe616e83700 1 -- 172.21.15.112:0/3471245164 wait complete. 2024-05-10T10:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-05-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-10T10:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-10T10:27:40.914 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.x 2024-05-10T10:27:40.915 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-10T10:27:41.064 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968.target.wants/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@.service. 2024-05-10T10:27:41.421 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:41 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3471245164' entity='client.admin' 2024-05-10T10:27:41.627 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T10:27:41.627 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-10T10:27:41.635 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T10:27:41.635 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-10T10:27:41.635 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-10T10:27:41.635 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T10:27:41.643 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T10:27:41.643 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-10T10:27:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T10:27:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-10T10:27:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-10T10:27:41.652 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-10T10:27:41.652 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-05-10T10:27:41.652 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-05-10T10:27:41.695 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:41 smithi112 systemd[1]: Started Ceph mgr.x for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:27:41.987 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:41.693+0000 7f29840b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T10:27:41.987 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:41.880+0000 7f29840b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:27:41.988 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:41.986+0000 7f29840b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T10:27:42.560 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:42 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1601464631' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T10:27:42.561 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:42 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:42.312+0000 7f29840b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T10:27:42.561 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:42 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:42.400+0000 7f29840b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T10:27:42.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-05-10T10:27:42.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:42.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T10:27:42.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T10:27:42.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:42.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-10T10:27:42.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T10:27:42.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:42.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T10:27:42.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T10:27:42.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:42.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T10:27:34.800712+0000", 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.115+0000 7fd07c64e700 1 Processor -- start 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.115+0000 7fd07c64e700 1 -- start start 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.115+0000 7fd07c64e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd074107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.115+0000 7fd07c64e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd074108570 con 0x7fd074107b80 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd07a3ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd074107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd07a3ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd074107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59730/0 (socket says 172.21.15.112:59730) 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd07a3ea700 1 -- 172.21.15.112:0/2934084463 learned_addr learned my addr 172.21.15.112:0/2934084463 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:42.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd07a3ea700 1 -- 172.21.15.112:0/2934084463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd074108d80 con 0x7fd074107b80 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd07a3ea700 1 --2- 172.21.15.112:0/2934084463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd074107fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd07000bb40 tx=0x7fd07000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7933e184959fa9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd0793e8700 1 -- 172.21.15.112:0/2934084463 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd07000dbe0 con 0x7fd074107b80 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd0793e8700 1 -- 172.21.15.112:0/2934084463 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd0700044a0 con 0x7fd074107b80 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.116+0000 7fd0793e8700 1 -- 172.21.15.112:0/2934084463 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd070003e30 con 0x7fd074107b80 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.117+0000 7fd07c64e700 1 -- 172.21.15.112:0/2934084463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 msgr2=0x7fd074107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.117+0000 7fd07c64e700 1 --2- 172.21.15.112:0/2934084463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd074107fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd07000bb40 tx=0x7fd07000be50 comp rx=0 tx=0).stop 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.117+0000 7fd07c64e700 1 -- 172.21.15.112:0/2934084463 shutdown_connections 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.117+0000 7fd07c64e700 1 --2- 172.21.15.112:0/2934084463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd074107fa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.117+0000 7fd07c64e700 1 -- 172.21.15.112:0/2934084463 >> 172.21.15.112:0/2934084463 conn(0x7fd074103140 msgr2=0x7fd074105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.117+0000 7fd07c64e700 1 -- 172.21.15.112:0/2934084463 shutdown_connections 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.117+0000 7fd07c64e700 1 -- 172.21.15.112:0/2934084463 wait complete. 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07c64e700 1 Processor -- start 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07c64e700 1 -- start start 2024-05-10T10:27:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07c64e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd07419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07c64e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd07407bf90 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07a3ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd07419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07a3ea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd07419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59746/0 (socket says 172.21.15.112:59746) 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07a3ea700 1 -- 172.21.15.112:0/1601464631 learned_addr learned my addr 172.21.15.112:0/1601464631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.118+0000 7fd07a3ea700 1 -- 172.21.15.112:0/1601464631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07419ade0 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd07a3ea700 1 --2- 172.21.15.112:0/1601464631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd07419a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd070003bc0 tx=0x7fd070003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd06b7fe700 1 -- 172.21.15.112:0/1601464631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd070027070 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd06b7fe700 1 -- 172.21.15.112:0/1601464631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd0700109f0 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07419b070 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07419dd60 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd06b7fe700 1 -- 172.21.15.112:0/1601464631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd070022990 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd06b7fe700 1 -- 172.21.15.112:0/1601464631 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd07002a070 con 0x7fd074107b80 2024-05-10T10:27:42.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.119+0000 7fd06b7fe700 1 -- 172.21.15.112:0/1601464631 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd07001f070 con 0x7fd074107b80 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.120+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058005360 con 0x7fd074107b80 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.122+0000 7fd06b7fe700 1 -- 172.21.15.112:0/1601464631 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd070031030 con 0x7fd074107b80 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.194+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd058005920 con 0x7fd074107b80 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.195+0000 7fd06b7fe700 1 -- 172.21.15.112:0/1601464631 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd070015070 con 0x7fd074107b80 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.195+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 msgr2=0x7fd07419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.195+0000 7fd07c64e700 1 --2- 172.21.15.112:0/1601464631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd07419a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd070003bc0 tx=0x7fd070003f80 comp rx=0 tx=0).stop 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.196+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 shutdown_connections 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.196+0000 7fd07c64e700 1 --2- 172.21.15.112:0/1601464631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd074107b80 0x7fd07419a8a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.196+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 >> 172.21.15.112:0/1601464631 conn(0x7fd074103140 msgr2=0x7fd074190ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.196+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 shutdown_connections 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:42.196+0000 7fd07c64e700 1 -- 172.21.15.112:0/1601464631 wait complete. 2024-05-10T10:27:42.847 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-05-10T10:27:43.933 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:43.595+0000 7f29840b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T10:27:43.933 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:43.764+0000 7f29840b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T10:27:43.933 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:43.932+0000 7f29840b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T10:27:44.510 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:44.218+0000 7f29840b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T10:27:44.511 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:44.310+0000 7f29840b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T10:27:44.511 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:44.509+0000 7f29840b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T10:27:44.829 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:44.698+0000 7f29840b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:27:45.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:45 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/256245875' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T10:27:45.424 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:45.312+0000 7f29840b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T10:27:45.830 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:45.543+0000 7f29840b0200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T10:27:46.329 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:45.972+0000 7f29840b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T10:27:46.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:46.071+0000 7f29840b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T10:27:46.795 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:46.438+0000 7f29840b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T10:27:46.796 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:46.794+0000 7f29840b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T10:27:47.079 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:46.887+0000 7f29840b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T10:27:47.437 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:47.131+0000 7f29840b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T10:27:47.438 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:47.264+0000 7f29840b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T10:27:47.438 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:47.360+0000 7f29840b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T10:27:47.778 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:47.684+0000 7f29840b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T10:27:47.779 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:47.777+0000 7f29840b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T10:27:48.657 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:48.384+0000 7f29840b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T10:27:48.658 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:48.603+0000 7f29840b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T10:27:48.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:48.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-10T10:27:48.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-10T10:27:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T10:27:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T10:27:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:48.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T10:27:48.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T10:27:48.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T10:27:34.800712+0000", 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-10T10:27:48.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.255+0000 7f96bb0b6700 1 Processor -- start 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.255+0000 7f96bb0b6700 1 -- start start 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.255+0000 7f96bb0b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b410baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.255+0000 7f96bb0b6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96b407ce30 con 0x7f96b4109700 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b8e52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b410baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b8e52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b410baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59754/0 (socket says 172.21.15.112:59754) 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b8e52700 1 -- 172.21.15.112:0/268958215 learned_addr learned my addr 172.21.15.112:0/268958215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b8e52700 1 -- 172.21.15.112:0/268958215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b410c030 con 0x7f96b4109700 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b8e52700 1 --2- 172.21.15.112:0/268958215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b410baf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96a80098a0 tx=0x7f96a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc41c29911999b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b37fe700 1 -- 172.21.15.112:0/268958215 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8004160 con 0x7f96b4109700 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b37fe700 1 -- 172.21.15.112:0/268958215 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f96a8004300 con 0x7f96b4109700 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.256+0000 7f96b37fe700 1 -- 172.21.15.112:0/268958215 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a80119e0 con 0x7f96b4109700 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.257+0000 7f96bb0b6700 1 -- 172.21.15.112:0/268958215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 msgr2=0x7f96b410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:48.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.257+0000 7f96bb0b6700 1 --2- 172.21.15.112:0/268958215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b410baf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96a80098a0 tx=0x7f96a8009bb0 comp rx=0 tx=0).stop 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.257+0000 7f96bb0b6700 1 -- 172.21.15.112:0/268958215 shutdown_connections 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.257+0000 7f96bb0b6700 1 --2- 172.21.15.112:0/268958215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b410baf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.257+0000 7f96bb0b6700 1 -- 172.21.15.112:0/268958215 >> 172.21.15.112:0/268958215 conn(0x7f96b41030f0 msgr2=0x7f96b4105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.257+0000 7f96bb0b6700 1 -- 172.21.15.112:0/268958215 shutdown_connections 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.257+0000 7f96bb0b6700 1 -- 172.21.15.112:0/268958215 wait complete. 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.258+0000 7f96bb0b6700 1 Processor -- start 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.258+0000 7f96bb0b6700 1 -- start start 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.258+0000 7f96bb0b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.258+0000 7f96bb0b6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96b410cf20 con 0x7f96b4109700 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.258+0000 7f96b8e52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.258+0000 7f96b8e52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59760/0 (socket says 172.21.15.112:59760) 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.258+0000 7f96b8e52700 1 -- 172.21.15.112:0/256245875 learned_addr learned my addr 172.21.15.112:0/256245875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.259+0000 7f96b8e52700 1 -- 172.21.15.112:0/256245875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b419cfb0 con 0x7f96b4109700 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.259+0000 7f96b8e52700 1 --2- 172.21.15.112:0/256245875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b419ca70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96a80116a0 tx=0x7f96a80046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.259+0000 7f96b1ffb700 1 -- 172.21.15.112:0/256245875 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8003b70 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.259+0000 7f96b1ffb700 1 -- 172.21.15.112:0/256245875 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f96a801a8a0 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.259+0000 7f96b1ffb700 1 -- 172.21.15.112:0/256245875 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8024890 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.259+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b419d240 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.259+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b419ff30 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.260+0000 7f96b1ffb700 1 -- 172.21.15.112:0/256245875 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f96a8022020 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.260+0000 7f96b1ffb700 1 -- 172.21.15.112:0/256245875 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96a80193c0 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.260+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b4064b00 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.262+0000 7f96b1ffb700 1 -- 172.21.15.112:0/256245875 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f96a80094f0 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.335+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f96b41a0100 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.336+0000 7f96b1ffb700 1 -- 172.21.15.112:0/256245875 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f96a8004300 con 0x7f96b4109700 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.337+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 msgr2=0x7f96b419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.337+0000 7f96bb0b6700 1 --2- 172.21.15.112:0/256245875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b419ca70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96a80116a0 tx=0x7f96a80046a0 comp rx=0 tx=0).stop 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.337+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 shutdown_connections 2024-05-10T10:27:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.337+0000 7f96bb0b6700 1 --2- 172.21.15.112:0/256245875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b4109700 0x7f96b419ca70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:48.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.337+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 >> 172.21.15.112:0/256245875 conn(0x7f96b41030f0 msgr2=0x7f96b4103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:48.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.337+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 shutdown_connections 2024-05-10T10:27:48.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:45.337+0000 7f96bb0b6700 1 -- 172.21.15.112:0/256245875 wait complete. 2024-05-10T10:27:48.897 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: Activating manager daemon x 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: mgrmap e2: x(active, starting, since 0.00316528s) 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: Manager daemon x is now available 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T10:27:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' 2024-05-10T10:27:49.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' 2024-05-10T10:27:49.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:48 smithi112 ceph-mon[30100]: from='mgr.14100 172.21.15.112:0/3815308950' entity='mgr.x' 2024-05-10T10:27:50.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:50 smithi112 ceph-mon[30100]: mgrmap e3: x(active, since 1.0077s) 2024-05-10T10:27:52.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:51 smithi112 ceph-mon[30100]: mgrmap e4: x(active, since 2s) 2024-05-10T10:27:55.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:54 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/171081203' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T10:27:57.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:57.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-10T10:27:57.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:27:57.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T10:27:57.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T10:27:57.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T10:27:57.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:57.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T10:27:57.419 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T10:27:57.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T10:27:34.800712+0000", 2024-05-10T10:27:57.421 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.411+0000 7f4e92411700 1 Processor -- start 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.411+0000 7f4e92411700 1 -- start start 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.411+0000 7f4e92411700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c109100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e92411700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1096d0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49342/0 (socket says 172.21.15.112:49342) 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8bfff700 1 -- 172.21.15.112:0/596990996 learned_addr learned my addr 172.21.15.112:0/596990996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8bfff700 1 -- 172.21.15.112:0/596990996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c109e40 con 0x7f4e8c108ce0 2024-05-10T10:27:57.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8bfff700 1 --2- 172.21.15.112:0/596990996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c109100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0098a0 tx=0x7f4e7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2fe48531c92127 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8affd700 1 -- 172.21.15.112:0/596990996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c004160 con 0x7f4e8c108ce0 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8affd700 1 -- 172.21.15.112:0/596990996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4e7c004300 con 0x7f4e8c108ce0 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.412+0000 7f4e8affd700 1 -- 172.21.15.112:0/596990996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c0119e0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.413+0000 7f4e92411700 1 -- 172.21.15.112:0/596990996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 msgr2=0x7f4e8c109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.413+0000 7f4e92411700 1 --2- 172.21.15.112:0/596990996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c109100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0098a0 tx=0x7f4e7c009bb0 comp rx=0 tx=0).stop 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.413+0000 7f4e92411700 1 -- 172.21.15.112:0/596990996 shutdown_connections 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.413+0000 7f4e92411700 1 --2- 172.21.15.112:0/596990996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c109100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.413+0000 7f4e92411700 1 -- 172.21.15.112:0/596990996 >> 172.21.15.112:0/596990996 conn(0x7f4e8c1042a0 msgr2=0x7f4e8c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.413+0000 7f4e92411700 1 -- 172.21.15.112:0/596990996 shutdown_connections 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.413+0000 7f4e92411700 1 -- 172.21.15.112:0/596990996 wait complete. 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.414+0000 7f4e92411700 1 Processor -- start 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.414+0000 7f4e92411700 1 -- start start 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.414+0000 7f4e92411700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:57.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.414+0000 7f4e92411700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c10d070 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.414+0000 7f4e8bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.414+0000 7f4e8bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c19ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49354/0 (socket says 172.21.15.112:49354) 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.414+0000 7f4e8bfff700 1 -- 172.21.15.112:0/171081203 learned_addr learned my addr 172.21.15.112:0/171081203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.415+0000 7f4e8bfff700 1 -- 172.21.15.112:0/171081203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c19cf50 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.415+0000 7f4e8bfff700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c19ca10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0119b0 tx=0x7f4e7c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.415+0000 7f4e897fa700 1 -- 172.21.15.112:0/171081203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c003af0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.415+0000 7f4e897fa700 1 -- 172.21.15.112:0/171081203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4e7c01a8a0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.415+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c19d1e0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.415+0000 7f4e897fa700 1 -- 172.21.15.112:0/171081203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c024830 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.415+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c19fed0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.416+0000 7f4e897fa700 1 -- 172.21.15.112:0/171081203 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f4e7c022020 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.416+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8c10d1f0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.416+0000 7f4e897fa700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f4e7403e420 0x7f4e740408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.416+0000 7f4e897fa700 1 -- 172.21.15.112:0/171081203 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e7c0536f0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.416+0000 7f4e8b7fe700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f4e7403e420 0x7f4e740408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.416+0000 7f4e8b7fe700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f4e7403e420 0x7f4e740408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e80006fd0 tx=0x7f4e80006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.421+0000 7f4e897fa700 1 -- 172.21.15.112:0/171081203 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4e7c0533d0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.717+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4e8c1a0520 con 0x7f4e8c108ce0 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.717+0000 7f4e897fa700 1 -- 172.21.15.112:0/171081203 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4e7c01aae0 con 0x7f4e8c108ce0 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.719+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f4e7403e420 msgr2=0x7f4e740408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.719+0000 7f4e92411700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f4e7403e420 0x7f4e740408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e80006fd0 tx=0x7f4e80006e40 comp rx=0 tx=0).stop 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.719+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 msgr2=0x7f4e8c19ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.719+0000 7f4e92411700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c19ca10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0119b0 tx=0x7f4e7c011a90 comp rx=0 tx=0).stop 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.719+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 shutdown_connections 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.720+0000 7f4e92411700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f4e7403e420 0x7f4e740408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.720+0000 7f4e92411700 1 --2- 172.21.15.112:0/171081203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e8c108ce0 0x7f4e8c19ca10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:57.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.720+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 >> 172.21.15.112:0/171081203 conn(0x7f4e8c1042a0 msgr2=0x7f4e8c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:57.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.720+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 shutdown_connections 2024-05-10T10:27:57.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:54.720+0000 7f4e92411700 1 -- 172.21.15.112:0/171081203 wait complete. 2024-05-10T10:27:57.426 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-05-10T10:27:58.388 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:58 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2933067907' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T10:27:58.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:58.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T10:27:58.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:27:58.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-10T10:27:58.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T10:27:58.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T10:27:58.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.825+0000 7f7501ab0700 1 Processor -- start 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.825+0000 7f7501ab0700 1 -- start start 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f7501ab0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc109140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f7501ab0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc109710 con 0x7f74fc108d20 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74fb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74fb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc109140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38984/0 (socket says 172.21.15.112:38984) 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74fb7fe700 1 -- 172.21.15.112:0/3400281155 learned_addr learned my addr 172.21.15.112:0/3400281155 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74fb7fe700 1 -- 172.21.15.112:0/3400281155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc109e80 con 0x7f74fc108d20 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74fb7fe700 1 --2- 172.21.15.112:0/3400281155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc109140 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0098a0 tx=0x7f74ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78286bd96b158d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74faffd700 1 -- 172.21.15.112:0/3400281155 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ec004160 con 0x7f74fc108d20 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74faffd700 1 -- 172.21.15.112:0/3400281155 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f74ec004300 con 0x7f74fc108d20 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.826+0000 7f74faffd700 1 -- 172.21.15.112:0/3400281155 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ec0119e0 con 0x7f74fc108d20 2024-05-10T10:27:58.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.827+0000 7f7501ab0700 1 -- 172.21.15.112:0/3400281155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 msgr2=0x7f74fc109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.827+0000 7f7501ab0700 1 --2- 172.21.15.112:0/3400281155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc109140 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0098a0 tx=0x7f74ec009bb0 comp rx=0 tx=0).stop 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.827+0000 7f7501ab0700 1 -- 172.21.15.112:0/3400281155 shutdown_connections 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.827+0000 7f7501ab0700 1 --2- 172.21.15.112:0/3400281155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc109140 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.827+0000 7f7501ab0700 1 -- 172.21.15.112:0/3400281155 >> 172.21.15.112:0/3400281155 conn(0x7f74fc104300 msgr2=0x7f74fc106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.827+0000 7f7501ab0700 1 -- 172.21.15.112:0/3400281155 shutdown_connections 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.827+0000 7f7501ab0700 1 -- 172.21.15.112:0/3400281155 wait complete. 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f7501ab0700 1 Processor -- start 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f7501ab0700 1 -- start start 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f7501ab0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f7501ab0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc10d0b0 con 0x7f74fc108d20 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f74fb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f74fb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38996/0 (socket says 172.21.15.112:38996) 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f74fb7fe700 1 -- 172.21.15.112:0/2933067907 learned_addr learned my addr 172.21.15.112:0/2933067907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:58.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f74fb7fe700 1 -- 172.21.15.112:0/2933067907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc19d010 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.828+0000 7f74fb7fe700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc19cad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006b20 tx=0x7f74ec011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.829+0000 7f74f97fa700 1 -- 172.21.15.112:0/2933067907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ec003a60 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.829+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74fc19d2a0 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.829+0000 7f74f97fa700 1 -- 172.21.15.112:0/2933067907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f74ec003c00 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.829+0000 7f74f97fa700 1 -- 172.21.15.112:0/2933067907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74ec024780 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.829+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74fc19ff90 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.830+0000 7f74f97fa700 1 -- 172.21.15.112:0/2933067907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f74ec022020 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.830+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc10d230 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.830+0000 7f74f97fa700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f74e803e470 0x7f74e8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.830+0000 7f74f97fa700 1 -- 172.21.15.112:0/2933067907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f74ec0535f0 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.830+0000 7f74f3fff700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f74e803e470 0x7f74e8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.830+0000 7f74f3fff700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f74e803e470 0x7f74e8040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006fd0 tx=0x7f74e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:57.835+0000 7f74f97fa700 1 -- 172.21.15.112:0/2933067907 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f74ec053350 con 0x7f74fc108d20 2024-05-10T10:27:58.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.038+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f74fc064b00 con 0x7f74fc108d20 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.039+0000 7f74f97fa700 1 -- 172.21.15.112:0/2933067907 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f74ec01aba0 con 0x7f74fc108d20 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.041+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f74e803e470 msgr2=0x7f74e8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.041+0000 7f7501ab0700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f74e803e470 0x7f74e8040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006fd0 tx=0x7f74e4006e40 comp rx=0 tx=0).stop 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.041+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 msgr2=0x7f74fc19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.041+0000 7f7501ab0700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc19cad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006b20 tx=0x7f74ec011a60 comp rx=0 tx=0).stop 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.041+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 shutdown_connections 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.041+0000 7f7501ab0700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f74e803e470 0x7f74e8040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.041+0000 7f7501ab0700 1 --2- 172.21.15.112:0/2933067907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74fc108d20 0x7f74fc19cad0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.042+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 >> 172.21.15.112:0/2933067907 conn(0x7f74fc104300 msgr2=0x7f74fc104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.042+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 shutdown_connections 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:58.042+0000 7f7501ab0700 1 -- 172.21.15.112:0/2933067907 wait complete. 2024-05-10T10:27:58.594 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-05-10T10:27:59.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:27:59 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2160977055' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-10T10:27:59.580 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:59 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: ignoring --setuser ceph since I am not root 2024-05-10T10:27:59.580 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:59 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: ignoring --setgroup ceph since I am not root 2024-05-10T10:27:59.580 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:59 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:59.502+0000 7f6078c0f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.015+0000 7f0295e53700 1 Processor -- start 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.015+0000 7f0295e53700 1 -- start start 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.015+0000 7f0295e53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029007d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.015+0000 7f0295e53700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f029007d7c0 con 0x7f029007edd0 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.015+0000 7f028f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029007d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.015+0000 7f028f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029007d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39004/0 (socket says 172.21.15.112:39004) 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.015+0000 7f028f7fe700 1 -- 172.21.15.112:0/256993797 learned_addr learned my addr 172.21.15.112:0/256993797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.016+0000 7f028f7fe700 1 -- 172.21.15.112:0/256993797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029007d940 con 0x7f029007edd0 2024-05-10T10:27:59.911 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.016+0000 7f028f7fe700 1 --2- 172.21.15.112:0/256993797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029007d280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02800098a0 tx=0x7f0280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c873e0ed4108b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.016+0000 7f028e7fc700 1 -- 172.21.15.112:0/256993797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0280004160 con 0x7f029007edd0 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.016+0000 7f028e7fc700 1 -- 172.21.15.112:0/256993797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0280004300 con 0x7f029007edd0 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.016+0000 7f028e7fc700 1 -- 172.21.15.112:0/256993797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02800119e0 con 0x7f029007edd0 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 -- 172.21.15.112:0/256993797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 msgr2=0x7f029007d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 --2- 172.21.15.112:0/256993797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029007d280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02800098a0 tx=0x7f0280009bb0 comp rx=0 tx=0).stop 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 -- 172.21.15.112:0/256993797 shutdown_connections 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 --2- 172.21.15.112:0/256993797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029007d280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 -- 172.21.15.112:0/256993797 >> 172.21.15.112:0/256993797 conn(0x7f02901042a0 msgr2=0x7f02901066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 -- 172.21.15.112:0/256993797 shutdown_connections 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 -- 172.21.15.112:0/256993797 wait complete. 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 Processor -- start 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.017+0000 7f0295e53700 1 -- start start 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f0295e53700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f0295e53700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f029010f130 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39018/0 (socket says 172.21.15.112:39018) 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028f7fe700 1 -- 172.21.15.112:0/2160977055 learned_addr learned my addr 172.21.15.112:0/2160977055 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028f7fe700 1 -- 172.21.15.112:0/2160977055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029019f130 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028f7fe700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029019ebf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02800119b0 tx=0x7f0280011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02800039f0 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0280003b90 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028001a620 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029019f3c0 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.018+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a20b0 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.019+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f028001a8d0 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.019+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290064b00 con 0x7f029007edd0 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.019+0000 7f028cff9700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f027803e420 0x7f02780408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:27:59.913 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.019+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0280053d10 con 0x7f029007edd0 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.020+0000 7f028effd700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f027803e420 0x7f02780408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.020+0000 7f028effd700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f027803e420 0x7f02780408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0284006fd0 tx=0x7f0284006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.024+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f028002c350 con 0x7f029007edd0 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.280+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f02901a2280 con 0x7f029007edd0 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.332+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f02901a2280 con 0x7f029007edd0 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.333+0000 7f028cff9700 1 -- 172.21.15.112:0/2160977055 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f0280024430 con 0x7f029007edd0 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f027803e420 msgr2=0x7f02780408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f027803e420 0x7f02780408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0284006fd0 tx=0x7f0284006e40 comp rx=0 tx=0).stop 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 msgr2=0x7f029019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029019ebf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02800119b0 tx=0x7f0280011a90 comp rx=0 tx=0).stop 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 shutdown_connections 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f027803e420 0x7f02780408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 --2- 172.21.15.112:0/2160977055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f029007edd0 0x7f029019ebf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:27:59.914 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.334+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 >> 172.21.15.112:0/2160977055 conn(0x7f02901042a0 msgr2=0x7f0290104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:27:59.915 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.335+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 shutdown_connections 2024-05-10T10:27:59.915 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:27:59.335+0000 7f0295e53700 1 -- 172.21.15.112:0/2160977055 wait complete. 2024-05-10T10:28:00.030 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:59 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:59.690+0000 7f6078c0f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:28:00.030 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:27:59 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:27:59.793+0000 7f6078c0f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T10:28:00.322 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:00 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:00.111+0000 7f6078c0f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T10:28:00.322 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:00 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:00.201+0000 7f6078c0f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T10:28:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:00 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2160977055' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-10T10:28:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:00 smithi112 ceph-mon[30100]: mgrmap e5: x(active, since 10s) 2024-05-10T10:28:01.260 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.346+0000 7feeaa982700 1 Processor -- start 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.346+0000 7feeaa982700 1 -- start start 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.346+0000 7feeaa982700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea4109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:01.261 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.346+0000 7feeaa982700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feea41096f0 con 0x7feea4108d00 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.346+0000 7feea3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea4109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.347+0000 7feea3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea4109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39046/0 (socket says 172.21.15.112:39046) 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.347+0000 7feea3fff700 1 -- 172.21.15.112:0/3558132575 learned_addr learned my addr 172.21.15.112:0/3558132575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.347+0000 7feea3fff700 1 -- 172.21.15.112:0/3558132575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea4109e60 con 0x7feea4108d00 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.347+0000 7feea3fff700 1 --2- 172.21.15.112:0/3558132575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea4109120 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee940098a0 tx=0x7fee94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f58f71af63306a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.347+0000 7feea2ffd700 1 -- 172.21.15.112:0/3558132575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee94004160 con 0x7feea4108d00 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.347+0000 7feea2ffd700 1 -- 172.21.15.112:0/3558132575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fee94004300 con 0x7feea4108d00 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.347+0000 7feea2ffd700 1 -- 172.21.15.112:0/3558132575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee940119e0 con 0x7feea4108d00 2024-05-10T10:28:01.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 -- 172.21.15.112:0/3558132575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 msgr2=0x7feea4109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 --2- 172.21.15.112:0/3558132575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea4109120 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee940098a0 tx=0x7fee94009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 -- 172.21.15.112:0/3558132575 shutdown_connections 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 --2- 172.21.15.112:0/3558132575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea4109120 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 -- 172.21.15.112:0/3558132575 >> 172.21.15.112:0/3558132575 conn(0x7feea41042e0 msgr2=0x7feea4106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 -- 172.21.15.112:0/3558132575 shutdown_connections 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 -- 172.21.15.112:0/3558132575 wait complete. 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 Processor -- start 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.348+0000 7feeaa982700 1 -- start start 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feeaa982700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feeaa982700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feea410d090 con 0x7feea4108d00 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feea3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feea3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39050/0 (socket says 172.21.15.112:39050) 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feea3fff700 1 -- 172.21.15.112:0/4008494866 learned_addr learned my addr 172.21.15.112:0/4008494866 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:01.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feea3fff700 1 -- 172.21.15.112:0/4008494866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea419cfc0 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feea3fff700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea419ca80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee94006b20 tx=0x7fee94011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.349+0000 7feea17fa700 1 -- 172.21.15.112:0/4008494866 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee94003a60 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.350+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea419d250 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.350+0000 7feea17fa700 1 -- 172.21.15.112:0/4008494866 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fee94003c00 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.350+0000 7feea17fa700 1 -- 172.21.15.112:0/4008494866 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee94024780 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.350+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea419ff40 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.351+0000 7feea17fa700 1 -- 172.21.15.112:0/4008494866 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fee94022020 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.351+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea410d210 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.351+0000 7feea17fa700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 0x7fee90040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.351+0000 7feea37fe700 1 -- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 msgr2=0x7fee90040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1796554484 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.351+0000 7feea17fa700 1 -- 172.21.15.112:0/4008494866 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee9401a430 con 0x7feea4108d00 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.351+0000 7feea37fe700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 0x7fee90040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:28:01.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.356+0000 7feea17fa700 1 -- 172.21.15.112:0/4008494866 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fee9400e260 con 0x7feea4108d00 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.552+0000 7feea37fe700 1 -- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 msgr2=0x7fee90040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1796554484 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.552+0000 7feea37fe700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 0x7fee90040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.658+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7feea41a0590 con 0x7feea4108d00 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.658+0000 7feea17fa700 1 -- 172.21.15.112:0/4008494866 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fee94054310 con 0x7feea4108d00 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 msgr2=0x7fee90040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 0x7fee90040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 msgr2=0x7feea419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea419ca80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee94006b20 tx=0x7fee94011a60 comp rx=0 tx=0).stop 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 shutdown_connections 2024-05-10T10:28:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7fee9003e470 0x7fee90040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:01.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 --2- 172.21.15.112:0/4008494866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea4108d00 0x7feea419ca80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:01.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 >> 172.21.15.112:0/4008494866 conn(0x7feea41042e0 msgr2=0x7feea4104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:01.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 shutdown_connections 2024-05-10T10:28:01.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:00.661+0000 7feeaa982700 1 -- 172.21.15.112:0/4008494866 wait complete. 2024-05-10T10:28:01.266 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-05-10T10:28:01.266 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-05-10T10:28:01.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:01 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4008494866' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T10:28:01.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:01 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:01.414+0000 7f6078c0f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T10:28:01.785 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:01 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:01.611+0000 7f6078c0f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T10:28:01.785 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:01 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:01.783+0000 7f6078c0f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T10:28:02.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:02 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:02.072+0000 7f6078c0f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T10:28:02.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:02 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:02.193+0000 7f6078c0f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T10:28:02.830 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:02 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:02.397+0000 7f6078c0f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T10:28:02.830 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:02 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:02.591+0000 7f6078c0f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:28:03.329 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:03 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:03.193+0000 7f6078c0f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T10:28:03.829 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:03 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:03.415+0000 7f6078c0f200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T10:28:04.308 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:03 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:03.840+0000 7f6078c0f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T10:28:04.309 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:03 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:03.938+0000 7f6078c0f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T10:28:04.309 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:04 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:04.307+0000 7f6078c0f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T10:28:05.007 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:04 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:04.666+0000 7f6078c0f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T10:28:05.008 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:04 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:04.759+0000 7f6078c0f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T10:28:05.329 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:05 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:05.006+0000 7f6078c0f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T10:28:05.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:05 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:05.141+0000 7f6078c0f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T10:28:05.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:05 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:05.237+0000 7f6078c0f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T10:28:05.829 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:05 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:05.563+0000 7f6078c0f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T10:28:05.830 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:05 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:05.657+0000 7f6078c0f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T10:28:06.537 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:06.261+0000 7f6078c0f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T10:28:06.537 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:06.480+0000 7f6078c0f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: Active manager daemon x restarted 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: Activating manager daemon x 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: osdmap e2: 0 total, 0 up, 0 in 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: mgrmap e6: x(active, starting, since 0.00338461s) 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T10:28:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: Manager daemon x is now available 2024-05-10T10:28:06.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:06.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:06.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:06.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:06.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:06 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T10:28:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:07 smithi112 ceph-mon[30100]: Found migration_current of "None". Setting to last migration. 2024-05-10T10:28:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:07 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T10:28:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:07 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:07 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:07 smithi112 ceph-mon[30100]: mgrmap e7: x(active, since 1.00824s) 2024-05-10T10:28:08.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-10T10:28:08.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-10T10:28:08.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T10:28:08.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-10T10:28:08.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171c770700 1 Processor -- start 2024-05-10T10:28:08.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171c770700 1 -- start start 2024-05-10T10:28:08.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171c770700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171410bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:08.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171c770700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f171407ce30 con 0x7f17141097d0 2024-05-10T10:28:08.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171a50c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171410bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:08.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171a50c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171410bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39066/0 (socket says 172.21.15.112:39066) 2024-05-10T10:28:08.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171a50c700 1 -- 172.21.15.112:0/268514534 learned_addr learned my addr 172.21.15.112:0/268514534 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:08.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.632+0000 7f171a50c700 1 -- 172.21.15.112:0/268514534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171410c100 con 0x7f17141097d0 2024-05-10T10:28:08.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.633+0000 7f171a50c700 1 --2- 172.21.15.112:0/268514534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171410bbc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f17100098a0 tx=0x7f1710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c7ee313cae83ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:08.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.633+0000 7f171950a700 1 -- 172.21.15.112:0/268514534 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1710004160 con 0x7f17141097d0 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.633+0000 7f171950a700 1 -- 172.21.15.112:0/268514534 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1710004300 con 0x7f17141097d0 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.633+0000 7f171950a700 1 -- 172.21.15.112:0/268514534 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17100119e0 con 0x7f17141097d0 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.633+0000 7f171c770700 1 -- 172.21.15.112:0/268514534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 msgr2=0x7f171410bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.633+0000 7f171c770700 1 --2- 172.21.15.112:0/268514534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171410bbc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f17100098a0 tx=0x7f1710009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 -- 172.21.15.112:0/268514534 shutdown_connections 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 --2- 172.21.15.112:0/268514534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171410bbc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 -- 172.21.15.112:0/268514534 >> 172.21.15.112:0/268514534 conn(0x7f1714103200 msgr2=0x7f1714105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:08.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 -- 172.21.15.112:0/268514534 shutdown_connections 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 -- 172.21.15.112:0/268514534 wait complete. 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 Processor -- start 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 -- start start 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171c770700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f171410cf80 con 0x7f17141097d0 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.634+0000 7f171a50c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f171a50c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39078/0 (socket says 172.21.15.112:39078) 2024-05-10T10:28:08.077 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f171a50c700 1 -- 172.21.15.112:0/856082966 learned_addr learned my addr 172.21.15.112:0/856082966 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:08.078 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f171a50c700 1 -- 172.21.15.112:0/856082966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171419d2e0 con 0x7f17141097d0 2024-05-10T10:28:08.078 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f171a50c700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171419cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17100116a0 tx=0x7f17100046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:08.078 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1710003b70 con 0x7f17141097d0 2024-05-10T10:28:08.078 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f171001a8a0 con 0x7f17141097d0 2024-05-10T10:28:08.078 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171419d570 con 0x7f17141097d0 2024-05-10T10:28:08.078 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1710024890 con 0x7f17141097d0 2024-05-10T10:28:08.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.635+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17141a0220 con 0x7f17141097d0 2024-05-10T10:28:08.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.636+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f1710022020 con 0x7f17141097d0 2024-05-10T10:28:08.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.636+0000 7f170b7fe700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:08.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.636+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 --> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1700041090 con 0x7f170003e4c0 2024-05-10T10:28:08.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.636+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f17100538c0 con 0x7f17141097d0 2024-05-10T10:28:08.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.636+0000 7f1719d0b700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 msgr2=0x7f1700040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1796554484 2024-05-10T10:28:08.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.636+0000 7f1719d0b700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:28:08.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.837+0000 7f1719d0b700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 msgr2=0x7f1700040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1796554484 2024-05-10T10:28:08.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:01.837+0000 7f1719d0b700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T10:28:08.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:02.238+0000 7f1719d0b700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 msgr2=0x7f1700040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1796554484 2024-05-10T10:28:08.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:02.238+0000 7f1719d0b700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T10:28:08.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:03.039+0000 7f1719d0b700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 msgr2=0x7f1700040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1796554484 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:03.039+0000 7f1719d0b700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:04.640+0000 7f1719d0b700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 msgr2=0x7f1700040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1796554484 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:04.640+0000 7f1719d0b700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:06.483+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f171002d3f0 con 0x7f17141097d0 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:06.484+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 msgr2=0x7f1700040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:06.484+0000 7f170b7fe700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.487+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f1710055310 con 0x7f17141097d0 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.488+0000 7f170b7fe700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f1700041db0 0x7f17000441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:08.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.488+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1710007d80 con 0x7f1700041db0 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.488+0000 7f1719d0b700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f1700041db0 0x7f17000441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.489+0000 7f1719d0b700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f1700041db0 0x7f17000441a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1704003a10 tx=0x7f17040092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.489+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1710007d80 con 0x7f1700041db0 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.497+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f171406a570 con 0x7f1700041db0 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f170b7fe700 1 -- 172.21.15.112:0/856082966 <== mgr.14118 v2:172.21.15.112:6800/2836147288 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f171406a570 con 0x7f1700041db0 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f1700041db0 msgr2=0x7f17000441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f1700041db0 0x7f17000441a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1704003a10 tx=0x7f17040092b0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 msgr2=0x7f171419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171419cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17100116a0 tx=0x7f17100046a0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 shutdown_connections 2024-05-10T10:28:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f1700041db0 0x7f17000441a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17141097d0 0x7f171419cda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 --2- 172.21.15.112:0/856082966 >> [v2:172.21.15.112:6800/1796554484,v1:172.21.15.112:6801/1796554484] conn(0x7f170003e4c0 0x7f1700040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 >> 172.21.15.112:0/856082966 conn(0x7f1714103200 msgr2=0x7f1714103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 shutdown_connections 2024-05-10T10:28:08.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:07.498+0000 7f171c770700 1 -- 172.21.15.112:0/856082966 wait complete. 2024-05-10T10:28:08.084 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-05-10T10:28:08.084 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-05-10T10:28:08.716 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:08 smithi112 ceph-mon[30100]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T10:28:08.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:08 smithi112 ceph-mon[30100]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T10:28:08.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:08 smithi112 ceph-mon[30100]: [10/May/2024:10:28:07] ENGINE Bus STARTING 2024-05-10T10:28:08.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:08 smithi112 ceph-mon[30100]: [10/May/2024:10:28:07] ENGINE Serving on https://172.21.15.112:7150 2024-05-10T10:28:08.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:08 smithi112 ceph-mon[30100]: [10/May/2024:10:28:07] ENGINE Serving on http://172.21.15.112:8765 2024-05-10T10:28:08.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:08 smithi112 ceph-mon[30100]: [10/May/2024:10:28:07] ENGINE Bus STARTED 2024-05-10T10:28:08.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:08 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:09.271 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.484+0000 7fd81c83a700 1 Processor -- start 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.484+0000 7fd81c83a700 1 -- start start 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81c83a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81810bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81c83a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd81807ce30 con 0x7fd818109720 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81810bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81810bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44390/0 (socket says 172.21.15.112:44390) 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81659c700 1 -- 172.21.15.112:0/3459427446 learned_addr learned my addr 172.21.15.112:0/3459427446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81659c700 1 -- 172.21.15.112:0/3459427446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81810c050 con 0x7fd818109720 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81659c700 1 --2- 172.21.15.112:0/3459427446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81810bb10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd8080098a0 tx=0x7fd808009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b33c3f5ba54a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.485+0000 7fd81559a700 1 -- 172.21.15.112:0/3459427446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd808004160 con 0x7fd818109720 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81559a700 1 -- 172.21.15.112:0/3459427446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd808004300 con 0x7fd818109720 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81559a700 1 -- 172.21.15.112:0/3459427446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8080119e0 con 0x7fd818109720 2024-05-10T10:28:09.272 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81c83a700 1 -- 172.21.15.112:0/3459427446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 msgr2=0x7fd81810bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81c83a700 1 --2- 172.21.15.112:0/3459427446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81810bb10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd8080098a0 tx=0x7fd808009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81c83a700 1 -- 172.21.15.112:0/3459427446 shutdown_connections 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81c83a700 1 --2- 172.21.15.112:0/3459427446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81810bb10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81c83a700 1 -- 172.21.15.112:0/3459427446 >> 172.21.15.112:0/3459427446 conn(0x7fd8181030f0 msgr2=0x7fd818105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81c83a700 1 -- 172.21.15.112:0/3459427446 shutdown_connections 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.486+0000 7fd81c83a700 1 -- 172.21.15.112:0/3459427446 wait complete. 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81c83a700 1 Processor -- start 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81c83a700 1 -- start start 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81c83a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81c83a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd81810cf40 con 0x7fd818109720 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:09.273 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44402/0 (socket says 172.21.15.112:44402) 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81659c700 1 -- 172.21.15.112:0/1360457450 learned_addr learned my addr 172.21.15.112:0/1360457450 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.487+0000 7fd81659c700 1 -- 172.21.15.112:0/1360457450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81819d000 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.488+0000 7fd81659c700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81819cac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd8080119e0 tx=0x7fd8080046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.488+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd808003bf0 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.488+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd80801a8a0 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.488+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8080248c0 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.488+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81819d290 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.488+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81819ff80 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.489+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fd808022020 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.489+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd818064b00 con 0x7fd818109720 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.489+0000 7fd8077fe700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd80003e310 0x7fd8000407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:09.274 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.489+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd808053970 con 0x7fd818109720 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.489+0000 7fd815d9b700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd80003e310 0x7fd8000407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.489+0000 7fd815d9b700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd80003e310 0x7fd8000407d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd80c006fd0 tx=0x7fd80c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.494+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd80800b3e0 con 0x7fd818109720 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.538+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd808024c70 con 0x7fd818109720 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.715+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd818104340 con 0x7fd80003e310 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.720+0000 7fd8077fe700 1 -- 172.21.15.112:0/1360457450 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd818104340 con 0x7fd80003e310 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd80003e310 msgr2=0x7fd8000407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd80003e310 0x7fd8000407d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd80c006fd0 tx=0x7fd80c006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 msgr2=0x7fd81819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81819cac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd8080119e0 tx=0x7fd8080046a0 comp rx=0 tx=0).stop 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 shutdown_connections 2024-05-10T10:28:09.275 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd80003e310 0x7fd8000407d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:09.276 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 --2- 172.21.15.112:0/1360457450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd818109720 0x7fd81819cac0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:09.276 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 >> 172.21.15.112:0/1360457450 conn(0x7fd8181030f0 msgr2=0x7fd818103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:09.276 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 shutdown_connections 2024-05-10T10:28:09.276 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:08.723+0000 7fd81c83a700 1 -- 172.21.15.112:0/1360457450 wait complete. 2024-05-10T10:28:09.665 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:09 smithi112 ceph-mon[30100]: mgrmap e8: x(active, since 2s) 2024-05-10T10:28:09.665 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:09 smithi112 ceph-mon[30100]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:09.665 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:09 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:09.665 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:09 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:10.476 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd5429de700 1 Processor -- start 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd5429de700 1 -- start start 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd5429de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd5429de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1085a0 con 0x7fd53c107bb0 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd53bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd53bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44412/0 (socket says 172.21.15.112:44412) 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd53bfff700 1 -- 172.21.15.112:0/4032457755 learned_addr learned my addr 172.21.15.112:0/4032457755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.683+0000 7fd53bfff700 1 -- 172.21.15.112:0/4032457755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c108db0 con 0x7fd53c107bb0 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd53bfff700 1 --2- 172.21.15.112:0/4032457755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c107fd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0098a0 tx=0x7fd52c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795742e43a760843 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd53affd700 1 -- 172.21.15.112:0/4032457755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd52c004160 con 0x7fd53c107bb0 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd53affd700 1 -- 172.21.15.112:0/4032457755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd52c004300 con 0x7fd53c107bb0 2024-05-10T10:28:10.477 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd53affd700 1 -- 172.21.15.112:0/4032457755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd52c0119e0 con 0x7fd53c107bb0 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd5429de700 1 -- 172.21.15.112:0/4032457755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 msgr2=0x7fd53c107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd5429de700 1 --2- 172.21.15.112:0/4032457755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c107fd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0098a0 tx=0x7fd52c009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd5429de700 1 -- 172.21.15.112:0/4032457755 shutdown_connections 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd5429de700 1 --2- 172.21.15.112:0/4032457755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c107fd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd5429de700 1 -- 172.21.15.112:0/4032457755 >> 172.21.15.112:0/4032457755 conn(0x7fd53c103150 msgr2=0x7fd53c105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd5429de700 1 -- 172.21.15.112:0/4032457755 shutdown_connections 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.684+0000 7fd5429de700 1 -- 172.21.15.112:0/4032457755 wait complete. 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.685+0000 7fd5429de700 1 Processor -- start 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.685+0000 7fd5429de700 1 -- start start 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.685+0000 7fd5429de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.685+0000 7fd5429de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c10bfe0 con 0x7fd53c107bb0 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.685+0000 7fd53bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.685+0000 7fd53bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44426/0 (socket says 172.21.15.112:44426) 2024-05-10T10:28:10.478 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.685+0000 7fd53bfff700 1 -- 172.21.15.112:0/343152396 learned_addr learned my addr 172.21.15.112:0/343152396 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.686+0000 7fd53bfff700 1 -- 172.21.15.112:0/343152396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c19d380 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.686+0000 7fd53bfff700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c19ce40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd52c006b20 tx=0x7fd52c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.686+0000 7fd5397fa700 1 -- 172.21.15.112:0/343152396 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd52c003a60 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.686+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c19d610 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.686+0000 7fd5397fa700 1 -- 172.21.15.112:0/343152396 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd52c003c00 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.686+0000 7fd5397fa700 1 -- 172.21.15.112:0/343152396 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd52c024780 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.686+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c1a0300 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.687+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53c10c160 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.687+0000 7fd5397fa700 1 -- 172.21.15.112:0/343152396 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd52c022020 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.687+0000 7fd5397fa700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd52403e470 0x7fd524040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.687+0000 7fd5397fa700 1 -- 172.21.15.112:0/343152396 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd52c053af0 con 0x7fd53c107bb0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.687+0000 7fd53b7fe700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd52403e470 0x7fd524040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.688+0000 7fd53b7fe700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd52403e470 0x7fd524040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd530006fd0 tx=0x7fd530006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:10.479 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.692+0000 7fd5397fa700 1 -- 172.21.15.112:0/343152396 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd52c0212c0 con 0x7fd53c107bb0 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.894+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd53c02e580 con 0x7fd52403e470 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.895+0000 7fd5397fa700 1 -- 172.21.15.112:0/343152396 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd53c02e580 con 0x7fd52403e470 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd52403e470 msgr2=0x7fd524040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd52403e470 0x7fd524040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd530006fd0 tx=0x7fd530006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 msgr2=0x7fd53c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c19ce40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd52c006b20 tx=0x7fd52c011a60 comp rx=0 tx=0).stop 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 shutdown_connections 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fd52403e470 0x7fd524040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 --2- 172.21.15.112:0/343152396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53c107bb0 0x7fd53c19ce40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 >> 172.21.15.112:0/343152396 conn(0x7fd53c103150 msgr2=0x7fd53c103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 shutdown_connections 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:09.897+0000 7fd5429de700 1 -- 172.21.15.112:0/343152396 wait complete. 2024-05-10T10:28:10.480 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-05-10T10:28:10.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:10 smithi112 ceph-mon[30100]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:11.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:11 smithi112 ceph-mon[30100]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:11.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:11 smithi112 ceph-mon[30100]: Generating ssh key... 2024-05-10T10:28:11.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:11 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:11.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:11 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:11.758 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: Generating public/private rsa key pair. 2024-05-10T10:28:11.758 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: Your identification has been saved in /tmp/tmpg8aj7ir0/key. 2024-05-10T10:28:11.758 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: Your public key has been saved in /tmp/tmpg8aj7ir0/key.pub. 2024-05-10T10:28:11.758 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: The key fingerprint is: 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: SHA256:4w0hAoF/FPzGr++CdlWl/8PEP2Df/O9r1g3KWW18Ay8 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: The key's randomart image is: 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: +---[RSA 3072]----+ 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | .oo.. | 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: |. .o . | 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | . ..o. . o | 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | . ..+. .o . | 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | . . .S. . .oo | 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | .o+ .E=o=| 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | . o. ...B+=*| 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | o + + +=*| 2024-05-10T10:28:11.759 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: | . . +o o+O| 2024-05-10T10:28:11.760 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: +----[SHA256]-----+ 2024-05-10T10:28:12.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0b9bf700 1 Processor -- start 2024-05-10T10:28:12.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0b9bf700 1 -- start start 2024-05-10T10:28:12.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0b9bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0410bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:12.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0b9bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0407ce10 con 0x7fcd04109760 2024-05-10T10:28:12.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0975b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0410bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:12.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0975b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0410bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44440/0 (socket says 172.21.15.112:44440) 2024-05-10T10:28:12.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0975b700 1 -- 172.21.15.112:0/3682147356 learned_addr learned my addr 172.21.15.112:0/3682147356 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:12.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.936+0000 7fcd0975b700 1 -- 172.21.15.112:0/3682147356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0410c090 con 0x7fcd04109760 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0975b700 1 --2- 172.21.15.112:0/3682147356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0410bb50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd000098a0 tx=0x7fcd00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad395ea46e3221d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fccf7fff700 1 -- 172.21.15.112:0/3682147356 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd00004160 con 0x7fcd04109760 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fccf7fff700 1 -- 172.21.15.112:0/3682147356 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcd00004300 con 0x7fcd04109760 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fccf7fff700 1 -- 172.21.15.112:0/3682147356 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd000119e0 con 0x7fcd04109760 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/3682147356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 msgr2=0x7fcd0410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0b9bf700 1 --2- 172.21.15.112:0/3682147356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0410bb50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd000098a0 tx=0x7fcd00009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/3682147356 shutdown_connections 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0b9bf700 1 --2- 172.21.15.112:0/3682147356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0410bb50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/3682147356 >> 172.21.15.112:0/3682147356 conn(0x7fcd041031b0 msgr2=0x7fcd041055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/3682147356 shutdown_connections 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.937+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/3682147356 wait complete. 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.938+0000 7fcd0b9bf700 1 Processor -- start 2024-05-10T10:28:12.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.938+0000 7fcd0b9bf700 1 -- start start 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.938+0000 7fcd0b9bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.938+0000 7fcd0b9bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0410cf30 con 0x7fcd04109760 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.938+0000 7fcd0975b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.938+0000 7fcd0975b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44442/0 (socket says 172.21.15.112:44442) 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.938+0000 7fcd0975b700 1 -- 172.21.15.112:0/596030713 learned_addr learned my addr 172.21.15.112:0/596030713 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.939+0000 7fcd0975b700 1 -- 172.21.15.112:0/596030713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0419cf50 con 0x7fcd04109760 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.939+0000 7fcd0975b700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0419ca10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009130 tx=0x7fcd00011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.939+0000 7fccf67fc700 1 -- 172.21.15.112:0/596030713 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd00003b20 con 0x7fcd04109760 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.939+0000 7fccf67fc700 1 -- 172.21.15.112:0/596030713 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcd0001a8a0 con 0x7fcd04109760 2024-05-10T10:28:12.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.939+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0419d1e0 con 0x7fcd04109760 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.939+0000 7fccf67fc700 1 -- 172.21.15.112:0/596030713 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd00024810 con 0x7fcd04109760 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.939+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0419fed0 con 0x7fcd04109760 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.940+0000 7fccf67fc700 1 -- 172.21.15.112:0/596030713 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcd00022020 con 0x7fcd04109760 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.940+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd04064b00 con 0x7fcd04109760 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.940+0000 7fccf67fc700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fccf003e470 0x7fccf0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.940+0000 7fccf67fc700 1 -- 172.21.15.112:0/596030713 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd00053910 con 0x7fcd04109760 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.940+0000 7fcd08f5a700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fccf003e470 0x7fccf0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.941+0000 7fcd08f5a700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fccf003e470 0x7fccf0040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fccf8006fd0 tx=0x7fccf8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:10.945+0000 7fccf67fc700 1 -- 172.21.15.112:0/596030713 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcd00005a20 con 0x7fcd04109760 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.153+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fcd04104360 con 0x7fccf003e470 2024-05-10T10:28:12.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.455+0000 7fccf67fc700 1 -- 172.21.15.112:0/596030713 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd04104360 con 0x7fccf003e470 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fccf003e470 msgr2=0x7fccf0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fccf003e470 0x7fccf0040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fccf8006fd0 tx=0x7fccf8006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 msgr2=0x7fcd0419ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0419ca10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009130 tx=0x7fcd00011a60 comp rx=0 tx=0).stop 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 shutdown_connections 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fccf003e470 0x7fccf0040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 --2- 172.21.15.112:0/596030713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd04109760 0x7fcd0419ca10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 >> 172.21.15.112:0/596030713 conn(0x7fcd041031b0 msgr2=0x7fcd04103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 shutdown_connections 2024-05-10T10:28:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:11.457+0000 7fcd0b9bf700 1 -- 172.21.15.112:0/596030713 wait complete. 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2hdUSu47XMzAMSNGOxfjVNEZMTm9TCpY4AeqDUzinCSXRTX4CTKj81XooOOwCdTepbszvHg+c5xeebnFClN7Oiwp2tdNGEoV9kT5HdVam3Nv5kVU2V8QtlDcsRXoFmgtGc6egp5RmasNY8PVC3nUFb2J6t+SfW784nyNMtspReiTbJutcYIhJbp2UMWRIGLf15pC5Ie5b5aThlEnEvjZhZnx0cB13XTS3M1EiJQW6KTFmEcJvy5ywshpxxVetigNohJhiqZGlqD1QTvdhAbW8/4T5oTf116kIBHsL2fOq3hYQr7OKL58WqfOHsERY6ugg37W99L4eCbnWoWdk721cbcgVNboERr/TLW09mRfnT8cS/mjHn2GAccafoKTMwaFjOTHa0r8ibn/b3DnF4MEOwlHnsDwnGx4jXSQXYZi/TeHoziNVWujvDQx1Dk67ccJ6cwk/9aesLIeylzY9UAzOVMrz/0D5Q98JUxiDgeWFqQV8gmIXdLFVbRxtAM5InWE= ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.474+0000 7fc5032a0700 1 Processor -- start 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.474+0000 7fc5032a0700 1 -- start start 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.474+0000 7fc5032a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.474+0000 7fc5032a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc109b00 con 0x7fc4fc109110 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc50103c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc50103c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44444/0 (socket says 172.21.15.112:44444) 2024-05-10T10:28:13.214 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc50103c700 1 -- 172.21.15.112:0/1051863765 learned_addr learned my addr 172.21.15.112:0/1051863765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc50103c700 1 -- 172.21.15.112:0/1051863765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc107d70 con 0x7fc4fc109110 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc50103c700 1 --2- 172.21.15.112:0/1051863765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc109530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40098a0 tx=0x7fc4f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0abf7d6f1c697f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc4fbfff700 1 -- 172.21.15.112:0/1051863765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4f4004160 con 0x7fc4fc109110 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc4fbfff700 1 -- 172.21.15.112:0/1051863765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc4f4004300 con 0x7fc4fc109110 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.475+0000 7fc4fbfff700 1 -- 172.21.15.112:0/1051863765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4f40119e0 con 0x7fc4fc109110 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.476+0000 7fc5032a0700 1 -- 172.21.15.112:0/1051863765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 msgr2=0x7fc4fc109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.476+0000 7fc5032a0700 1 --2- 172.21.15.112:0/1051863765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc109530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40098a0 tx=0x7fc4f4009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.476+0000 7fc5032a0700 1 -- 172.21.15.112:0/1051863765 shutdown_connections 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.476+0000 7fc5032a0700 1 --2- 172.21.15.112:0/1051863765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc109530 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.476+0000 7fc5032a0700 1 -- 172.21.15.112:0/1051863765 >> 172.21.15.112:0/1051863765 conn(0x7fc4fc1028e0 msgr2=0x7fc4fc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.476+0000 7fc5032a0700 1 -- 172.21.15.112:0/1051863765 shutdown_connections 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.476+0000 7fc5032a0700 1 -- 172.21.15.112:0/1051863765 wait complete. 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc5032a0700 1 Processor -- start 2024-05-10T10:28:13.215 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc5032a0700 1 -- start start 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc5032a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc107740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc5032a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10bfb0 con 0x7fc4fc109110 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc50103c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc50103c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc107740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44452/0 (socket says 172.21.15.112:44452) 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc50103c700 1 -- 172.21.15.112:0/601465300 learned_addr learned my addr 172.21.15.112:0/601465300 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc50103c700 1 -- 172.21.15.112:0/601465300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc19f790 con 0x7fc4fc109110 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc50103c700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc107740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40119b0 tx=0x7fc4f4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc4fa7fc700 1 -- 172.21.15.112:0/601465300 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4f4003af0 con 0x7fc4fc109110 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc4fa7fc700 1 -- 172.21.15.112:0/601465300 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc4f401a8a0 con 0x7fc4fc109110 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc19f960 con 0x7fc4fc109110 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc4fa7fc700 1 -- 172.21.15.112:0/601465300 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4f4024830 con 0x7fc4fc109110 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.477+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc19fd00 con 0x7fc4fc109110 2024-05-10T10:28:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.478+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc10c130 con 0x7fc4fc109110 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.478+0000 7fc4fa7fc700 1 -- 172.21.15.112:0/601465300 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc4f4022020 con 0x7fc4fc109110 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.479+0000 7fc4fa7fc700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc4e403e470 0x7fc4e4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.479+0000 7fc4fa7fc700 1 -- 172.21.15.112:0/601465300 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc4f4053a30 con 0x7fc4fc109110 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.479+0000 7fc50083b700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc4e403e470 0x7fc4e4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.479+0000 7fc50083b700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc4e403e470 0x7fc4e4040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006fd0 tx=0x7fc4f0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.484+0000 7fc4fa7fc700 1 -- 172.21.15.112:0/601465300 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc4f4019de0 con 0x7fc4fc109110 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.692+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc4fc103a60 con 0x7fc4e403e470 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.692+0000 7fc4fa7fc700 1 -- 172.21.15.112:0/601465300 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc4fc103a60 con 0x7fc4e403e470 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc4e403e470 msgr2=0x7fc4e4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc4e403e470 0x7fc4e4040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006fd0 tx=0x7fc4f0006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 msgr2=0x7fc4fc107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc107740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40119b0 tx=0x7fc4f4011a90 comp rx=0 tx=0).stop 2024-05-10T10:28:13.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 shutdown_connections 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc4e403e470 0x7fc4e4040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 --2- 172.21.15.112:0/601465300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4fc109110 0x7fc4fc107740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 >> 172.21.15.112:0/601465300 conn(0x7fc4fc1028e0 msgr2=0x7fc4fc103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 shutdown_connections 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:12.694+0000 7fc5032a0700 1 -- 172.21.15.112:0/601465300 wait complete. 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-05-10T10:28:13.218 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-05-10T10:28:13.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:13 smithi112 ceph-mon[30100]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:14.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:14 smithi112 ceph-mon[30100]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:15.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:15 smithi112 ceph-mon[30100]: Deploying cephadm binary to smithi112 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.633+0000 7f38f1692700 1 Processor -- start 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.633+0000 7f38f1692700 1 -- start start 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.633+0000 7f38f1692700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.633+0000 7f38f1692700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec108540 con 0x7f38ec107b50 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.633+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.634+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44460/0 (socket says 172.21.15.112:44460) 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.634+0000 7f38eaffd700 1 -- 172.21.15.112:0/283907179 learned_addr learned my addr 172.21.15.112:0/283907179 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.634+0000 7f38eaffd700 1 -- 172.21.15.112:0/283907179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec108d50 con 0x7f38ec107b50 2024-05-10T10:28:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.634+0000 7f38eaffd700 1 --2- 172.21.15.112:0/283907179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec107f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bb40 tx=0x7f38dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ebda6ab4f4548a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.634+0000 7f38e9ffb700 1 -- 172.21.15.112:0/283907179 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38dc00dbe0 con 0x7f38ec107b50 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.634+0000 7f38e9ffb700 1 -- 172.21.15.112:0/283907179 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f38dc0044a0 con 0x7f38ec107b50 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.634+0000 7f38e9ffb700 1 -- 172.21.15.112:0/283907179 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38dc003e30 con 0x7f38ec107b50 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 -- 172.21.15.112:0/283907179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 msgr2=0x7f38ec107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 --2- 172.21.15.112:0/283907179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec107f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bb40 tx=0x7f38dc00be50 comp rx=0 tx=0).stop 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 -- 172.21.15.112:0/283907179 shutdown_connections 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 --2- 172.21.15.112:0/283907179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec107f70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 -- 172.21.15.112:0/283907179 >> 172.21.15.112:0/283907179 conn(0x7f38ec1030f0 msgr2=0x7f38ec105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 -- 172.21.15.112:0/283907179 shutdown_connections 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 -- 172.21.15.112:0/283907179 wait complete. 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 Processor -- start 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.635+0000 7f38f1692700 1 -- start start 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38f1692700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:18.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38f1692700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec07bf90 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44476/0 (socket says 172.21.15.112:44476) 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38eaffd700 1 -- 172.21.15.112:0/3519529824 learned_addr learned my addr 172.21.15.112:0/3519529824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38eaffd700 1 -- 172.21.15.112:0/3519529824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec19ae40 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38eaffd700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec19a900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bfd0 tx=0x7f38dc003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38d3fff700 1 -- 172.21.15.112:0/3519529824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38dc027070 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38d3fff700 1 -- 172.21.15.112:0/3519529824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f38dc010a40 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec19b0d0 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38d3fff700 1 -- 172.21.15.112:0/3519529824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38dc022ad0 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.636+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec19ddc0 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.637+0000 7f38d3fff700 1 -- 172.21.15.112:0/3519529824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f38dc02a070 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.637+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec064b00 con 0x7f38ec107b50 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.638+0000 7f38d3fff700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f38d403e470 0x7f38d4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:18.673 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.638+0000 7f38d3fff700 1 -- 172.21.15.112:0/3519529824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f38dc01f070 con 0x7f38ec107b50 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.638+0000 7f38ea7fc700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f38d403e470 0x7f38d4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.638+0000 7f38ea7fc700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f38d403e470 0x7f38d4040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38e0006fd0 tx=0x7f38e0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.643+0000 7f38d3fff700 1 -- 172.21.15.112:0/3519529824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f38dc015070 con 0x7f38ec107b50 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:13.848+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}) -- 0x7f38ec0008d0 con 0x7f38d403e470 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.020+0000 7f38d3fff700 1 -- 172.21.15.112:0/3519529824 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f38ec0008d0 con 0x7f38d403e470 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f38d403e470 msgr2=0x7f38d4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f38d403e470 0x7f38d4040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38e0006fd0 tx=0x7f38e0006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 msgr2=0x7f38ec19a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec19a900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bfd0 tx=0x7f38dc003f80 comp rx=0 tx=0).stop 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 shutdown_connections 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f38d403e470 0x7f38d4040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 --2- 172.21.15.112:0/3519529824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ec107b50 0x7f38ec19a900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 >> 172.21.15.112:0/3519529824 conn(0x7f38ec1030f0 msgr2=0x7f38ec191520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 shutdown_connections 2024-05-10T10:28:18.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:18.024+0000 7f38f1692700 1 -- 172.21.15.112:0/3519529824 wait complete. 2024-05-10T10:28:18.675 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mon service... 2024-05-10T10:28:19.311 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:19 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:19.311 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:19 smithi112 ceph-mon[30100]: Added host smithi112 2024-05-10T10:28:19.311 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:19 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-10T10:28:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.077+0000 7fe36965c700 1 Processor -- start 2024-05-10T10:28:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.077+0000 7fe36965c700 1 -- start start 2024-05-10T10:28:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.077+0000 7fe36965c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe364107f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.077+0000 7fe36965c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe364108550 con 0x7fe364107b60 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.077+0000 7fe362ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe364107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.077+0000 7fe362ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe364107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33022/0 (socket says 172.21.15.112:33022) 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.077+0000 7fe362ffd700 1 -- 172.21.15.112:0/1691687998 learned_addr learned my addr 172.21.15.112:0/1691687998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.078+0000 7fe362ffd700 1 -- 172.21.15.112:0/1691687998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe364108d60 con 0x7fe364107b60 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.078+0000 7fe362ffd700 1 --2- 172.21.15.112:0/1691687998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe364107f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3540098a0 tx=0x7fe354009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3933730ddcc0ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.078+0000 7fe361ffb700 1 -- 172.21.15.112:0/1691687998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe354004160 con 0x7fe364107b60 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.078+0000 7fe361ffb700 1 -- 172.21.15.112:0/1691687998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe354004300 con 0x7fe364107b60 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.078+0000 7fe361ffb700 1 -- 172.21.15.112:0/1691687998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3540119e0 con 0x7fe364107b60 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 -- 172.21.15.112:0/1691687998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 msgr2=0x7fe364107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 --2- 172.21.15.112:0/1691687998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe364107f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3540098a0 tx=0x7fe354009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 -- 172.21.15.112:0/1691687998 shutdown_connections 2024-05-10T10:28:20.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 --2- 172.21.15.112:0/1691687998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe364107f80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 -- 172.21.15.112:0/1691687998 >> 172.21.15.112:0/1691687998 conn(0x7fe364103100 msgr2=0x7fe364105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 -- 172.21.15.112:0/1691687998 shutdown_connections 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 -- 172.21.15.112:0/1691687998 wait complete. 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 Processor -- start 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.079+0000 7fe36965c700 1 -- start start 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe36965c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe36419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe36965c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe36410bf90 con 0x7fe364107b60 2024-05-10T10:28:20.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe362ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe36419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe362ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe36419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33026/0 (socket says 172.21.15.112:33026) 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe362ffd700 1 -- 172.21.15.112:0/2585111217 learned_addr learned my addr 172.21.15.112:0/2585111217 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe362ffd700 1 -- 172.21.15.112:0/2585111217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36419cfe0 con 0x7fe364107b60 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe362ffd700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe36419caa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe3540119b0 tx=0x7fe354011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe34bfff700 1 -- 172.21.15.112:0/2585111217 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe354003b70 con 0x7fe364107b60 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36419d270 con 0x7fe364107b60 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe34bfff700 1 -- 172.21.15.112:0/2585111217 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe35401a8a0 con 0x7fe364107b60 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe34bfff700 1 -- 172.21.15.112:0/2585111217 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe354024830 con 0x7fe364107b60 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.080+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36419ff60 con 0x7fe364107b60 2024-05-10T10:28:20.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.081+0000 7fe34bfff700 1 -- 172.21.15.112:0/2585111217 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe354022020 con 0x7fe364107b60 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.081+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36410c110 con 0x7fe364107b60 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.081+0000 7fe34bfff700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fe34c03e420 0x7fe34c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.082+0000 7fe34bfff700 1 -- 172.21.15.112:0/2585111217 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe354053ac0 con 0x7fe364107b60 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.082+0000 7fe3627fc700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fe34c03e420 0x7fe34c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.082+0000 7fe3627fc700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fe34c03e420 0x7fe34c0408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe358006fd0 tx=0x7fe358006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.087+0000 7fe34bfff700 1 -- 172.21.15.112:0/2585111217 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe35400e260 con 0x7fe364107b60 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.309+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe36402e840 con 0x7fe34c03e420 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.317+0000 7fe34bfff700 1 -- 172.21.15.112:0/2585111217 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe36402e840 con 0x7fe34c03e420 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fe34c03e420 msgr2=0x7fe34c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fe34c03e420 0x7fe34c0408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe358006fd0 tx=0x7fe358006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 msgr2=0x7fe36419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe36419caa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe3540119b0 tx=0x7fe354011a90 comp rx=0 tx=0).stop 2024-05-10T10:28:20.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 shutdown_connections 2024-05-10T10:28:20.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fe34c03e420 0x7fe34c0408e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:20.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 --2- 172.21.15.112:0/2585111217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe364107b60 0x7fe36419caa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:20.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.319+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 >> 172.21.15.112:0/2585111217 conn(0x7fe364103100 msgr2=0x7fe364103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:20.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.320+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 shutdown_connections 2024-05-10T10:28:20.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:19.320+0000 7fe36965c700 1 -- 172.21.15.112:0/2585111217 wait complete. 2024-05-10T10:28:20.069 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mgr service... 2024-05-10T10:28:20.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:20 smithi112 ceph-mon[30100]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:20.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:20 smithi112 ceph-mon[30100]: Saving service mon spec with placement count:5 2024-05-10T10:28:20.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:20 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5dca2a700 1 Processor -- start 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5dca2a700 1 -- start start 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5dca2a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d810bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5dca2a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d807ce70 con 0x7ff5d81096f0 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5d659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d810bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5d659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d810bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33042/0 (socket says 172.21.15.112:33042) 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5d659c700 1 -- 172.21.15.112:0/3846069983 learned_addr learned my addr 172.21.15.112:0/3846069983 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.821+0000 7ff5d659c700 1 -- 172.21.15.112:0/3846069983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d810c020 con 0x7ff5d81096f0 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5d659c700 1 --2- 172.21.15.112:0/3846069983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d810bae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098a0 tx=0x7ff5c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c69955cbdf52946 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:21.666 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5d559a700 1 -- 172.21.15.112:0/3846069983 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c8004160 con 0x7ff5d81096f0 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5d559a700 1 -- 172.21.15.112:0/3846069983 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff5c8004300 con 0x7ff5d81096f0 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5d559a700 1 -- 172.21.15.112:0/3846069983 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c80119e0 con 0x7ff5d81096f0 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5dca2a700 1 -- 172.21.15.112:0/3846069983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 msgr2=0x7ff5d810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5dca2a700 1 --2- 172.21.15.112:0/3846069983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d810bae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098a0 tx=0x7ff5c8009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5dca2a700 1 -- 172.21.15.112:0/3846069983 shutdown_connections 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5dca2a700 1 --2- 172.21.15.112:0/3846069983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d810bae0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5dca2a700 1 -- 172.21.15.112:0/3846069983 >> 172.21.15.112:0/3846069983 conn(0x7ff5d81031c0 msgr2=0x7ff5d81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5dca2a700 1 -- 172.21.15.112:0/3846069983 shutdown_connections 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.822+0000 7ff5dca2a700 1 -- 172.21.15.112:0/3846069983 wait complete. 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.823+0000 7ff5dca2a700 1 Processor -- start 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.823+0000 7ff5dca2a700 1 -- start start 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.823+0000 7ff5dca2a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:21.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.823+0000 7ff5dca2a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d810cf20 con 0x7ff5d81096f0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.823+0000 7ff5d659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5d659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33044/0 (socket says 172.21.15.112:33044) 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5d659c700 1 -- 172.21.15.112:0/1019283228 learned_addr learned my addr 172.21.15.112:0/1019283228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5d659c700 1 -- 172.21.15.112:0/1019283228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d819cf40 con 0x7ff5d81096f0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5d659c700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d819ca00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80119b0 tx=0x7ff5c8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5c77fe700 1 -- 172.21.15.112:0/1019283228 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c80039f0 con 0x7ff5d81096f0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5c77fe700 1 -- 172.21.15.112:0/1019283228 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff5c8003b90 con 0x7ff5d81096f0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5c77fe700 1 -- 172.21.15.112:0/1019283228 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5c801a620 con 0x7ff5d81096f0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d819d1d0 con 0x7ff5d81096f0 2024-05-10T10:28:21.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.824+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d819fec0 con 0x7ff5d81096f0 2024-05-10T10:28:21.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.825+0000 7ff5c77fe700 1 -- 172.21.15.112:0/1019283228 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff5c8024430 con 0x7ff5d81096f0 2024-05-10T10:28:21.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.825+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d8064b00 con 0x7ff5d81096f0 2024-05-10T10:28:21.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.825+0000 7ff5c77fe700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff5c003e420 0x7ff5c00408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:21.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.825+0000 7ff5c77fe700 1 -- 172.21.15.112:0/1019283228 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5c8053bc0 con 0x7ff5d81096f0 2024-05-10T10:28:21.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.825+0000 7ff5d5d9b700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff5c003e420 0x7ff5c00408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:21.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.826+0000 7ff5d5d9b700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff5c003e420 0x7ff5c00408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc006fd0 tx=0x7ff5cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:21.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:20.833+0000 7ff5c77fe700 1 -- 172.21.15.112:0/1019283228 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff5c8024720 con 0x7ff5d81096f0 2024-05-10T10:28:21.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.052+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff5d81042a0 con 0x7ff5c003e420 2024-05-10T10:28:21.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.056+0000 7ff5c77fe700 1 -- 172.21.15.112:0/1019283228 <== mgr.14118 v2:172.21.15.112:6800/2836147288 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff5d81042a0 con 0x7ff5c003e420 2024-05-10T10:28:21.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff5c003e420 msgr2=0x7ff5c00408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:21.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff5c003e420 0x7ff5c00408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc006fd0 tx=0x7ff5cc006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:21.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 msgr2=0x7ff5d819ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:21.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d819ca00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80119b0 tx=0x7ff5c8011a90 comp rx=0 tx=0).stop 2024-05-10T10:28:21.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 shutdown_connections 2024-05-10T10:28:21.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff5c003e420 0x7ff5c00408e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:21.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 --2- 172.21.15.112:0/1019283228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d81096f0 0x7ff5d819ca00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:21.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 >> 172.21.15.112:0/1019283228 conn(0x7ff5d81031c0 msgr2=0x7ff5d8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:21.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 shutdown_connections 2024-05-10T10:28:21.672 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:21.059+0000 7ff5dca2a700 1 -- 172.21.15.112:0/1019283228 wait complete. 2024-05-10T10:28:22.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:22 smithi112 ceph-mon[30100]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:22.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:22 smithi112 ceph-mon[30100]: Saving service mgr spec with placement count:2 2024-05-10T10:28:22.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:22 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:23.366 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:23 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:23.366 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:23 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:23.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:23 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/987155955' entity='client.admin' 2024-05-10T10:28:23.374 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.560+0000 7fad5bd1f700 1 Processor -- start 2024-05-10T10:28:23.374 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.560+0000 7fad5bd1f700 1 -- start start 2024-05-10T10:28:23.374 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.560+0000 7fad5bd1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5410baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:23.374 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.560+0000 7fad5bd1f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad5407ce30 con 0x7fad54109700 2024-05-10T10:28:23.374 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad59abb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5410baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad59abb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5410baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33046/0 (socket says 172.21.15.112:33046) 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad59abb700 1 -- 172.21.15.112:0/3164782123 learned_addr learned my addr 172.21.15.112:0/3164782123 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad59abb700 1 -- 172.21.15.112:0/3164782123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5410c030 con 0x7fad54109700 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad59abb700 1 --2- 172.21.15.112:0/3164782123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5410baf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad500098a0 tx=0x7fad50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434e10d6672fab7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad58ab9700 1 -- 172.21.15.112:0/3164782123 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad50004160 con 0x7fad54109700 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad58ab9700 1 -- 172.21.15.112:0/3164782123 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad50004300 con 0x7fad54109700 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.561+0000 7fad58ab9700 1 -- 172.21.15.112:0/3164782123 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad50011960 con 0x7fad54109700 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.562+0000 7fad5bd1f700 1 -- 172.21.15.112:0/3164782123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 msgr2=0x7fad5410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.562+0000 7fad5bd1f700 1 --2- 172.21.15.112:0/3164782123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5410baf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad500098a0 tx=0x7fad50009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.562+0000 7fad5bd1f700 1 -- 172.21.15.112:0/3164782123 shutdown_connections 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.562+0000 7fad5bd1f700 1 --2- 172.21.15.112:0/3164782123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5410baf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.562+0000 7fad5bd1f700 1 -- 172.21.15.112:0/3164782123 >> 172.21.15.112:0/3164782123 conn(0x7fad541030f0 msgr2=0x7fad54105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:23.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.562+0000 7fad5bd1f700 1 -- 172.21.15.112:0/3164782123 shutdown_connections 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.562+0000 7fad5bd1f700 1 -- 172.21.15.112:0/3164782123 wait complete. 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad5bd1f700 1 Processor -- start 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad5bd1f700 1 -- start start 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad5bd1f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad5bd1f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad5410cfd0 con 0x7fad54109700 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad59abb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad59abb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33052/0 (socket says 172.21.15.112:33052) 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad59abb700 1 -- 172.21.15.112:0/987155955 learned_addr learned my addr 172.21.15.112:0/987155955 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad59abb700 1 -- 172.21.15.112:0/987155955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5419cff0 con 0x7fad54109700 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.563+0000 7fad59abb700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5419cab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad50004660 tx=0x7fad50011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.564+0000 7fad4affd700 1 -- 172.21.15.112:0/987155955 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad50003aa0 con 0x7fad54109700 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.564+0000 7fad4affd700 1 -- 172.21.15.112:0/987155955 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad5001b8d0 con 0x7fad54109700 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.564+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5419d280 con 0x7fad54109700 2024-05-10T10:28:23.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.564+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5419ff70 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.564+0000 7fad4affd700 1 -- 172.21.15.112:0/987155955 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad50023820 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.565+0000 7fad4affd700 1 -- 172.21.15.112:0/987155955 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fad50023a70 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.565+0000 7fad4affd700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fad4003e420 0x7fad400408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.565+0000 7fad4affd700 1 -- 172.21.15.112:0/987155955 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad50053eb0 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.565+0000 7fad592ba700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fad4003e420 0x7fad400408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.565+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad54064b00 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.566+0000 7fad592ba700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fad4003e420 0x7fad400408e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad44006fd0 tx=0x7fad44006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.570+0000 7fad4affd700 1 -- 172.21.15.112:0/987155955 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fad5001f030 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.788+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fad541a0390 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.791+0000 7fad4affd700 1 -- 172.21.15.112:0/987155955 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fad50019e20 con 0x7fad54109700 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fad4003e420 msgr2=0x7fad400408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fad4003e420 0x7fad400408e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad44006fd0 tx=0x7fad44006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 msgr2=0x7fad5419cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:23.377 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5419cab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad50004660 tx=0x7fad50011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:23.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 shutdown_connections 2024-05-10T10:28:23.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fad4003e420 0x7fad400408e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:23.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 --2- 172.21.15.112:0/987155955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad54109700 0x7fad5419cab0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:23.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.793+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 >> 172.21.15.112:0/987155955 conn(0x7fad541030f0 msgr2=0x7fad54103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:23.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.794+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 shutdown_connections 2024-05-10T10:28:23.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:22.794+0000 7fad5bd1f700 1 -- 172.21.15.112:0/987155955 wait complete. 2024-05-10T10:28:25.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.745+0000 7ff61df4b700 1 Processor -- start 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.745+0000 7ff61df4b700 1 -- start start 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.745+0000 7ff61df4b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff618109070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.745+0000 7ff61df4b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff618109640 con 0x7ff618108c50 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff618109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff618109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33060/0 (socket says 172.21.15.112:33060) 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6177fe700 1 -- 172.21.15.112:0/283796812 learned_addr learned my addr 172.21.15.112:0/283796812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6177fe700 1 -- 172.21.15.112:0/283796812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff618109e50 con 0x7ff618108c50 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6177fe700 1 --2- 172.21.15.112:0/283796812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff618109070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6160cbdc2e4e2db9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6167fc700 1 -- 172.21.15.112:0/283796812 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff608004160 con 0x7ff618108c50 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6167fc700 1 -- 172.21.15.112:0/283796812 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff608004300 con 0x7ff618108c50 2024-05-10T10:28:25.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.746+0000 7ff6167fc700 1 -- 172.21.15.112:0/283796812 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff608011960 con 0x7ff618108c50 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.747+0000 7ff61df4b700 1 -- 172.21.15.112:0/283796812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 msgr2=0x7ff618109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.747+0000 7ff61df4b700 1 --2- 172.21.15.112:0/283796812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff618109070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff608009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.747+0000 7ff61df4b700 1 -- 172.21.15.112:0/283796812 shutdown_connections 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.747+0000 7ff61df4b700 1 --2- 172.21.15.112:0/283796812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff618109070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.747+0000 7ff61df4b700 1 -- 172.21.15.112:0/283796812 >> 172.21.15.112:0/283796812 conn(0x7ff618104210 msgr2=0x7ff618106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.747+0000 7ff61df4b700 1 -- 172.21.15.112:0/283796812 shutdown_connections 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.747+0000 7ff61df4b700 1 -- 172.21.15.112:0/283796812 wait complete. 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff61df4b700 1 Processor -- start 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff61df4b700 1 -- start start 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff61df4b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff61819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff61df4b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff61810d080 con 0x7ff618108c50 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff61819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff61819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33070/0 (socket says 172.21.15.112:33070) 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff6177fe700 1 -- 172.21.15.112:0/122147649 learned_addr learned my addr 172.21.15.112:0/122147649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff6177fe700 1 -- 172.21.15.112:0/122147649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61819d020 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.748+0000 7ff6177fe700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff61819cae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff608011670 tx=0x7ff608011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.749+0000 7ff614ff9700 1 -- 172.21.15.112:0/122147649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff60801f070 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.749+0000 7ff614ff9700 1 -- 172.21.15.112:0/122147649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff608003b20 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.749+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61819d2b0 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.749+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61819ffa0 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.749+0000 7ff614ff9700 1 -- 172.21.15.112:0/122147649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff608019580 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.750+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f8005360 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.750+0000 7ff614ff9700 1 -- 172.21.15.112:0/122147649 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff608022070 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.750+0000 7ff614ff9700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff60403e470 0x7ff604040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.750+0000 7ff614ff9700 1 -- 172.21.15.112:0/122147649 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff608053a50 con 0x7ff618108c50 2024-05-10T10:28:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.750+0000 7ff616ffd700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff60403e470 0x7ff604040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.751+0000 7ff616ffd700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff60403e470 0x7ff604040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.755+0000 7ff614ff9700 1 -- 172.21.15.112:0/122147649 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff6080172b0 con 0x7ff618108c50 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.983+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff5f8005650 con 0x7ff618108c50 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.986+0000 7ff614ff9700 1 -- 172.21.15.112:0/122147649 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff6080198a0 con 0x7ff618108c50 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff60403e470 msgr2=0x7ff604040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff60403e470 0x7ff604040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 msgr2=0x7ff61819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff61819cae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff608011670 tx=0x7ff608011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 shutdown_connections 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ff60403e470 0x7ff604040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 --2- 172.21.15.112:0/122147649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff618108c50 0x7ff61819cae0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 >> 172.21.15.112:0/122147649 conn(0x7ff618104210 msgr2=0x7ff618104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 shutdown_connections 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:23.989+0000 7ff61df4b700 1 -- 172.21.15.112:0/122147649 wait complete. 2024-05-10T10:28:25.172 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-05-10T10:28:25.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:24 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/122147649' entity='client.admin' 2024-05-10T10:28:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:24 smithi112 ceph-mon[30100]: from='mgr.14118 172.21.15.112:0/2513048400' entity='mgr.x' 2024-05-10T10:28:26.178 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:25 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1645951148' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-10T10:28:26.178 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: ignoring --setuser ceph since I am not root 2024-05-10T10:28:26.178 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: ignoring --setgroup ceph since I am not root 2024-05-10T10:28:26.178 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:26.176+0000 7f3e5f095200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T10:28:26.475 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:26.369+0000 7f3e5f095200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:28:26.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.573+0000 7fc5ba29e700 1 Processor -- start 2024-05-10T10:28:26.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.573+0000 7fc5ba29e700 1 -- start start 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.573+0000 7fc5ba29e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b410bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.573+0000 7fc5ba29e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b407ce30 con 0x7fc5b4109740 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b410bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b410bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47952/0 (socket says 172.21.15.112:47952) 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b3fff700 1 -- 172.21.15.112:0/377976316 learned_addr learned my addr 172.21.15.112:0/377976316 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b3fff700 1 -- 172.21.15.112:0/377976316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b410c070 con 0x7fc5b4109740 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b3fff700 1 --2- 172.21.15.112:0/377976316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b410bb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40098a0 tx=0x7fc5a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd27571a6f30e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b2ffd700 1 -- 172.21.15.112:0/377976316 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a4004160 con 0x7fc5b4109740 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b2ffd700 1 -- 172.21.15.112:0/377976316 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5a4004300 con 0x7fc5b4109740 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.574+0000 7fc5b2ffd700 1 -- 172.21.15.112:0/377976316 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a4011960 con 0x7fc5b4109740 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 -- 172.21.15.112:0/377976316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 msgr2=0x7fc5b410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:26.577 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 --2- 172.21.15.112:0/377976316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b410bb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40098a0 tx=0x7fc5a4009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 -- 172.21.15.112:0/377976316 shutdown_connections 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 --2- 172.21.15.112:0/377976316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b410bb30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 -- 172.21.15.112:0/377976316 >> 172.21.15.112:0/377976316 conn(0x7fc5b41031b0 msgr2=0x7fc5b41055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 -- 172.21.15.112:0/377976316 shutdown_connections 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 -- 172.21.15.112:0/377976316 wait complete. 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.575+0000 7fc5ba29e700 1 Processor -- start 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5ba29e700 1 -- start start 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5ba29e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5ba29e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b410d010 con 0x7fc5b4109740 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47962/0 (socket says 172.21.15.112:47962) 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5b3fff700 1 -- 172.21.15.112:0/1645951148 learned_addr learned my addr 172.21.15.112:0/1645951148 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5b3fff700 1 -- 172.21.15.112:0/1645951148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b419cfa0 con 0x7fc5b4109740 2024-05-10T10:28:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5b3fff700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b419ca60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4003e50 tx=0x7fc5a4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a4003a60 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b419d230 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.576+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5a401b8d0 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.577+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a40237d0 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.577+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b419ff20 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.577+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc5a402c3f0 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.578+0000 7fc5b17fa700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc5a003e420 0x7fc5a00408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.578+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b4064b00 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.578+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc5a4054010 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.578+0000 7fc5b37fe700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc5a003e420 0x7fc5a00408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.578+0000 7fc5b37fe700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc5a003e420 0x7fc5a00408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc59c006fd0 tx=0x7fc59c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.583+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc5a401f030 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.844+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc5b41a00f0 con 0x7fc5b4109740 2024-05-10T10:28:26.579 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.989+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc5a40202b0 con 0x7fc5b4109740 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.991+0000 7fc5b17fa700 1 -- 172.21.15.112:0/1645951148 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fc5a4052ac0 con 0x7fc5b4109740 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc5a003e420 msgr2=0x7fc5a00408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc5a003e420 0x7fc5a00408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc59c006fd0 tx=0x7fc59c006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 msgr2=0x7fc5b419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b419ca60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4003e50 tx=0x7fc5a4011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 shutdown_connections 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7fc5a003e420 0x7fc5a00408e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 --2- 172.21.15.112:0/1645951148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109740 0x7fc5b419ca60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 >> 172.21.15.112:0/1645951148 conn(0x7fc5b41031b0 msgr2=0x7fc5b4103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 shutdown_connections 2024-05-10T10:28:26.580 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:25.992+0000 7fc5ba29e700 1 -- 172.21.15.112:0/1645951148 wait complete. 2024-05-10T10:28:26.793 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:26.474+0000 7f3e5f095200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T10:28:27.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:26 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1645951148' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-10T10:28:27.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:26 smithi112 ceph-mon[30100]: mgrmap e9: x(active, since 19s) 2024-05-10T10:28:27.080 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:26.793+0000 7f3e5f095200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T10:28:27.080 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:26.881+0000 7f3e5f095200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T10:28:28.075 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:27 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2849470667' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T10:28:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.014+0000 7f40fff0f700 1 Processor -- start 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.014+0000 7f40fff0f700 1 -- start start 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fff0f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f810bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fff0f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40f807ce70 con 0x7f40f81096f0 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fdcab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f810bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fdcab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f810bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47980/0 (socket says 172.21.15.112:47980) 2024-05-10T10:28:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fdcab700 1 -- 172.21.15.112:0/1967922349 learned_addr learned my addr 172.21.15.112:0/1967922349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fdcab700 1 -- 172.21.15.112:0/1967922349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f810c020 con 0x7f40f81096f0 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fdcab700 1 --2- 172.21.15.112:0/1967922349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f810bae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40f40098a0 tx=0x7f40f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595e69752542febc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fcca9700 1 -- 172.21.15.112:0/1967922349 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40f4004160 con 0x7f40f81096f0 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fcca9700 1 -- 172.21.15.112:0/1967922349 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40f4004300 con 0x7f40f81096f0 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.015+0000 7f40fcca9700 1 -- 172.21.15.112:0/1967922349 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40f4011960 con 0x7f40f81096f0 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.016+0000 7f40fff0f700 1 -- 172.21.15.112:0/1967922349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 msgr2=0x7f40f810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.016+0000 7f40fff0f700 1 --2- 172.21.15.112:0/1967922349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f810bae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40f40098a0 tx=0x7f40f4009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.016+0000 7f40fff0f700 1 -- 172.21.15.112:0/1967922349 shutdown_connections 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.016+0000 7f40fff0f700 1 --2- 172.21.15.112:0/1967922349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f810bae0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.016+0000 7f40fff0f700 1 -- 172.21.15.112:0/1967922349 >> 172.21.15.112:0/1967922349 conn(0x7f40f8103140 msgr2=0x7f40f8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.016+0000 7f40fff0f700 1 -- 172.21.15.112:0/1967922349 shutdown_connections 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.016+0000 7f40fff0f700 1 -- 172.21.15.112:0/1967922349 wait complete. 2024-05-10T10:28:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fff0f700 1 Processor -- start 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fff0f700 1 -- start start 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fff0f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f8108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fff0f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40f810cfd0 con 0x7f40f81096f0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fdcab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f8108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fdcab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f8108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47990/0 (socket says 172.21.15.112:47990) 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fdcab700 1 -- 172.21.15.112:0/2849470667 learned_addr learned my addr 172.21.15.112:0/2849470667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fdcab700 1 -- 172.21.15.112:0/2849470667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f81094b0 con 0x7f40f81096f0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.017+0000 7f40fdcab700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f8108f70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40f4004660 tx=0x7f40f4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.018+0000 7f40eeffd700 1 -- 172.21.15.112:0/2849470667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40f4003aa0 con 0x7f40f81096f0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.018+0000 7f40eeffd700 1 -- 172.21.15.112:0/2849470667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40f401b8d0 con 0x7f40f81096f0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.018+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f81076a0 con 0x7f40f81096f0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.018+0000 7f40eeffd700 1 -- 172.21.15.112:0/2849470667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40f4023820 con 0x7f40f81096f0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.018+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f8107be0 con 0x7f40f81096f0 2024-05-10T10:28:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.019+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f8064b00 con 0x7f40f81096f0 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.019+0000 7f40eeffd700 1 -- 172.21.15.112:0/2849470667 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f40f401ba80 con 0x7f40f81096f0 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.019+0000 7f40eeffd700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 0x7f40e4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.019+0000 7f40eeffd700 1 -- 172.21.15.112:0/2849470667 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40f4053ef0 con 0x7f40f81096f0 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.019+0000 7f40fd4aa700 1 -- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 msgr2=0x7f40e4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2836147288 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.019+0000 7f40fd4aa700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 0x7f40e4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.024+0000 7f40eeffd700 1 -- 172.21.15.112:0/2849470667 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f40f401f030 con 0x7f40f81096f0 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.220+0000 7f40fd4aa700 1 -- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 msgr2=0x7f40e4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2836147288 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.220+0000 7f40fd4aa700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 0x7f40e4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.320+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f40f8108480 con 0x7f40f81096f0 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.321+0000 7f40eeffd700 1 -- 172.21.15.112:0/2849470667 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f40f40202b0 con 0x7f40f81096f0 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.327+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 msgr2=0x7f40e4040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.327+0000 7f40fff0f700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 0x7f40e4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.327+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 msgr2=0x7f40f8108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.327+0000 7f40fff0f700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f8108f70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40f4004660 tx=0x7f40f4011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.327+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 shutdown_connections 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.327+0000 7f40fff0f700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7f40e403e470 0x7f40e4040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.327+0000 7f40fff0f700 1 --2- 172.21.15.112:0/2849470667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f81096f0 0x7f40f8108f70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.328+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 >> 172.21.15.112:0/2849470667 conn(0x7f40f8103140 msgr2=0x7f40f8103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.329+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 shutdown_connections 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:27.329+0000 7f40fff0f700 1 -- 172.21.15.112:0/2849470667 wait complete. 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-05-10T10:28:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-05-10T10:28:28.329 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:28.121+0000 7f3e5f095200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T10:28:28.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:28.293+0000 7f3e5f095200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T10:28:28.745 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:28.468+0000 7f3e5f095200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T10:28:29.062 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:28.766+0000 7f3e5f095200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T10:28:29.062 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:28.859+0000 7f3e5f095200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T10:28:29.062 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:29.060+0000 7f3e5f095200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T10:28:29.329 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:29.254+0000 7f3e5f095200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:28:30.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:29.853+0000 7f3e5f095200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T10:28:30.330 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:30.075+0000 7f3e5f095200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T10:28:30.829 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:30.504+0000 7f3e5f095200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T10:28:30.830 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:30.603+0000 7f3e5f095200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T10:28:31.329 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:30.981+0000 7f3e5f095200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T10:28:31.685 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:31.340+0000 7f3e5f095200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T10:28:31.685 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:31.435+0000 7f3e5f095200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T10:28:32.079 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:31.684+0000 7f3e5f095200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T10:28:32.080 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:31.822+0000 7f3e5f095200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T10:28:32.080 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:31.920+0000 7f3e5f095200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T10:28:32.580 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:32.251+0000 7f3e5f095200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T10:28:32.580 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:32.346+0000 7f3e5f095200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T10:28:33.233 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:32.956+0000 7f3e5f095200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T10:28:33.233 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:28:33 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:28:33.176+0000 7f3e5f095200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: Active manager daemon x restarted 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: Activating manager daemon x 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: osdmap e3: 0 total, 0 up, 0 in 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: mgrmap e10: x(active, starting, since 0.00409063s) 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: Manager daemon x is now available 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T10:28:34.534 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T10:28:34.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:34 smithi112 ceph-mon[30100]: [10/May/2024:10:28:33] ENGINE Bus STARTING 2024-05-10T10:28:34.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:34 smithi112 ceph-mon[30100]: [10/May/2024:10:28:34] ENGINE Serving on http://172.21.15.112:8765 2024-05-10T10:28:34.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:34 smithi112 ceph-mon[30100]: mgrmap e11: x(active, since 1.00607s) 2024-05-10T10:28:35.102 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-10T10:28:35.102 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-10T10:28:35.102 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T10:28:35.102 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-10T10:28:35.102 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.537+0000 7ffa488d2700 1 Processor -- start 2024-05-10T10:28:35.103 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa488d2700 1 -- start start 2024-05-10T10:28:35.103 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa488d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa44107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:35.103 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa488d2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44108570 con 0x7ffa44107b80 2024-05-10T10:28:35.103 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa44107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:35.103 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa44107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47994/0 (socket says 172.21.15.112:47994) 2024-05-10T10:28:35.103 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa4259c700 1 -- 172.21.15.112:0/2083321529 learned_addr learned my addr 172.21.15.112:0/2083321529 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa4259c700 1 -- 172.21.15.112:0/2083321529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa44108d80 con 0x7ffa44107b80 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.538+0000 7ffa4259c700 1 --2- 172.21.15.112:0/2083321529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa44107fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffa340098a0 tx=0x7ffa34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e270e4093afda2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.539+0000 7ffa4159a700 1 -- 172.21.15.112:0/2083321529 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa34004160 con 0x7ffa44107b80 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.539+0000 7ffa4159a700 1 -- 172.21.15.112:0/2083321529 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa34004300 con 0x7ffa44107b80 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.539+0000 7ffa4159a700 1 -- 172.21.15.112:0/2083321529 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa34011960 con 0x7ffa44107b80 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.539+0000 7ffa488d2700 1 -- 172.21.15.112:0/2083321529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 msgr2=0x7ffa44107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.539+0000 7ffa488d2700 1 --2- 172.21.15.112:0/2083321529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa44107fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffa340098a0 tx=0x7ffa34009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:35.104 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.540+0000 7ffa488d2700 1 -- 172.21.15.112:0/2083321529 shutdown_connections 2024-05-10T10:28:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.540+0000 7ffa488d2700 1 --2- 172.21.15.112:0/2083321529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa44107fa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.540+0000 7ffa488d2700 1 -- 172.21.15.112:0/2083321529 >> 172.21.15.112:0/2083321529 conn(0x7ffa44103120 msgr2=0x7ffa44105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.540+0000 7ffa488d2700 1 -- 172.21.15.112:0/2083321529 shutdown_connections 2024-05-10T10:28:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.540+0000 7ffa488d2700 1 -- 172.21.15.112:0/2083321529 wait complete. 2024-05-10T10:28:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa488d2700 1 Processor -- start 2024-05-10T10:28:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa488d2700 1 -- start start 2024-05-10T10:28:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa488d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa4419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa488d2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4410bfb0 con 0x7ffa44107b80 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa4419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa4419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48002/0 (socket says 172.21.15.112:48002) 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa4259c700 1 -- 172.21.15.112:0/1336298171 learned_addr learned my addr 172.21.15.112:0/1336298171 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa4259c700 1 -- 172.21.15.112:0/1336298171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4419bf20 con 0x7ffa44107b80 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.541+0000 7ffa4259c700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa4419b9e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa34011670 tx=0x7ffa34011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.542+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa3401f070 con 0x7ffa44107b80 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.542+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa34003b20 con 0x7ffa44107b80 2024-05-10T10:28:35.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.542+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4419c1b0 con 0x7ffa44107b80 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.542+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa34019580 con 0x7ffa44107b80 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.542+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4419eea0 con 0x7ffa44107b80 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.543+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7ffa34022070 con 0x7ffa44107b80 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.543+0000 7ffa337fe700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.543+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 --> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa4406a570 con 0x7ffa2c03e4c0 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.543+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffa34053a90 con 0x7ffa44107b80 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.543+0000 7ffa41d9b700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 msgr2=0x7ffa2c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2836147288 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.543+0000 7ffa41d9b700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.744+0000 7ffa41d9b700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 msgr2=0x7ffa2c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2836147288 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:28.744+0000 7ffa41d9b700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:29.145+0000 7ffa41d9b700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 msgr2=0x7ffa2c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2836147288 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:29.145+0000 7ffa41d9b700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T10:28:35.107 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:29.946+0000 7ffa41d9b700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 msgr2=0x7ffa2c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2836147288 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:29.946+0000 7ffa41d9b700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:31.547+0000 7ffa41d9b700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 msgr2=0x7ffa2c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2836147288 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:31.548+0000 7ffa41d9b700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:33.180+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7ffa3400bd00 con 0x7ffa44107b80 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:33.180+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 msgr2=0x7ffa2c040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:33.180+0000 7ffa337fe700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.181+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7ffa340555a0 con 0x7ffa44107b80 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.182+0000 7ffa337fe700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa2c041d50 0x7ffa2c044140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.182+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa4406a570 con 0x7ffa2c041d50 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.182+0000 7ffa41d9b700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa2c041d50 0x7ffa2c044140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.183+0000 7ffa41d9b700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa2c041d50 0x7ffa2c044140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffa38003a10 tx=0x7ffa380092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.183+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ffa4406a570 con 0x7ffa2c041d50 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.192+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ffa44104370 con 0x7ffa2c041d50 2024-05-10T10:28:35.108 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.192+0000 7ffa337fe700 1 -- 172.21.15.112:0/1336298171 <== mgr.14150 v2:172.21.15.112:6800/1898750131 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ffa44104370 con 0x7ffa2c041d50 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.192+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa2c041d50 msgr2=0x7ffa2c044140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.192+0000 7ffa488d2700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa2c041d50 0x7ffa2c044140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffa38003a10 tx=0x7ffa380092b0 comp rx=0 tx=0).stop 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 msgr2=0x7ffa4419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa4419b9e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa34011670 tx=0x7ffa34011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 shutdown_connections 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa2c041d50 0x7ffa2c044140 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44107b80 0x7ffa4419b9e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 --2- 172.21.15.112:0/1336298171 >> [v2:172.21.15.112:6800/2836147288,v1:172.21.15.112:6801/2836147288] conn(0x7ffa2c03e4c0 0x7ffa2c040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 >> 172.21.15.112:0/1336298171 conn(0x7ffa44103120 msgr2=0x7ffa44103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 shutdown_connections 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:34.193+0000 7ffa488d2700 1 -- 172.21.15.112:0/1336298171 wait complete. 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-05-10T10:28:35.109 INFO:teuthology.orchestra.run.smithi112.stdout:Generating a dashboard self-signed certificate... 2024-05-10T10:28:35.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:35 smithi112 ceph-mon[30100]: [10/May/2024:10:28:34] ENGINE Serving on https://172.21.15.112:7150 2024-05-10T10:28:35.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:35 smithi112 ceph-mon[30100]: [10/May/2024:10:28:34] ENGINE Bus STARTED 2024-05-10T10:28:35.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:35 smithi112 ceph-mon[30100]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T10:28:35.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:35 smithi112 ceph-mon[30100]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T10:28:35.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:36.403 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-10T10:28:36.403 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.555+0000 7f14e1aaf700 1 Processor -- start 2024-05-10T10:28:36.403 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.555+0000 7f14e1aaf700 1 -- start start 2024-05-10T10:28:36.403 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.555+0000 7f14e1aaf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc109100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:36.403 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.555+0000 7f14e1aaf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1096d0 con 0x7f14dc108ce0 2024-05-10T10:28:36.403 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.555+0000 7f14db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:36.403 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.555+0000 7f14db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55408/0 (socket says 172.21.15.112:55408) 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.555+0000 7f14db7fe700 1 -- 172.21.15.112:0/864449154 learned_addr learned my addr 172.21.15.112:0/864449154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.556+0000 7f14db7fe700 1 -- 172.21.15.112:0/864449154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc109e40 con 0x7f14dc108ce0 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.556+0000 7f14db7fe700 1 --2- 172.21.15.112:0/864449154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc109100 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0098a0 tx=0x7f14cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c2cc1158bfd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.556+0000 7f14da7fc700 1 -- 172.21.15.112:0/864449154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14cc004160 con 0x7f14dc108ce0 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.556+0000 7f14da7fc700 1 -- 172.21.15.112:0/864449154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14cc004300 con 0x7f14dc108ce0 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.556+0000 7f14da7fc700 1 -- 172.21.15.112:0/864449154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14cc011960 con 0x7f14dc108ce0 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.557+0000 7f14e1aaf700 1 -- 172.21.15.112:0/864449154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 msgr2=0x7f14dc109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.557+0000 7f14e1aaf700 1 --2- 172.21.15.112:0/864449154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc109100 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0098a0 tx=0x7f14cc009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.557+0000 7f14e1aaf700 1 -- 172.21.15.112:0/864449154 shutdown_connections 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.557+0000 7f14e1aaf700 1 --2- 172.21.15.112:0/864449154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc109100 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.557+0000 7f14e1aaf700 1 -- 172.21.15.112:0/864449154 >> 172.21.15.112:0/864449154 conn(0x7f14dc1042a0 msgr2=0x7f14dc1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.557+0000 7f14e1aaf700 1 -- 172.21.15.112:0/864449154 shutdown_connections 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.557+0000 7f14e1aaf700 1 -- 172.21.15.112:0/864449154 wait complete. 2024-05-10T10:28:36.404 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14e1aaf700 1 Processor -- start 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14e1aaf700 1 -- start start 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14e1aaf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14e1aaf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc10d070 con 0x7f14dc108ce0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55412/0 (socket says 172.21.15.112:55412) 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14db7fe700 1 -- 172.21.15.112:0/4004974751 learned_addr learned my addr 172.21.15.112:0/4004974751 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14db7fe700 1 -- 172.21.15.112:0/4004974751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc19cf80 con 0x7f14dc108ce0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.558+0000 7f14db7fe700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc19ca40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14cc011670 tx=0x7f14cc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.559+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14cc003a60 con 0x7f14dc108ce0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.559+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14cc01b8d0 con 0x7f14dc108ce0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.559+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc19d210 con 0x7f14dc108ce0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.559+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14cc023750 con 0x7f14dc108ce0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.559+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc19ff00 con 0x7f14dc108ce0 2024-05-10T10:28:36.405 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.560+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc064b00 con 0x7f14dc108ce0 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.560+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f14cc023a50 con 0x7f14dc108ce0 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.560+0000 7f14d8ff9700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14c403e360 0x7f14c4040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.560+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14cc054040 con 0x7f14dc108ce0 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.560+0000 7f14daffd700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14c403e360 0x7f14c4040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.561+0000 7f14daffd700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14c403e360 0x7f14c4040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f14d0006fd0 tx=0x7f14d0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.565+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f14cc01f030 con 0x7f14dc108ce0 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.588+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f14cc02c3f0 con 0x7f14dc108ce0 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.780+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f14dc1053e0 con 0x7f14c403e360 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.829+0000 7f14d8ff9700 1 -- 172.21.15.112:0/4004974751 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f14dc1053e0 con 0x7f14c403e360 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14c403e360 msgr2=0x7f14c4040820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14c403e360 0x7f14c4040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f14d0006fd0 tx=0x7f14d0006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 msgr2=0x7f14dc19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:36.406 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc19ca40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14cc011670 tx=0x7f14cc011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:36.407 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 shutdown_connections 2024-05-10T10:28:36.407 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14c403e360 0x7f14c4040820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:36.407 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 --2- 172.21.15.112:0/4004974751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14dc108ce0 0x7f14dc19ca40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:36.407 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 >> 172.21.15.112:0/4004974751 conn(0x7f14dc1042a0 msgr2=0x7f14dc104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:36.407 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 shutdown_connections 2024-05-10T10:28:36.407 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:35.832+0000 7f14e1aaf700 1 -- 172.21.15.112:0/4004974751 wait complete. 2024-05-10T10:28:36.407 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-05-10T10:28:36.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:36 smithi112 ceph-mon[30100]: mgrmap e12: x(active, since 2s) 2024-05-10T10:28:36.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:36.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:36 smithi112 ceph-mon[30100]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:36.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:36.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:38.019 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:37 smithi112 ceph-mon[30100]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:38.019 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:38.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$6AobheNQxQQZdGYaYX1S5.VsjUMRk0uXoyTCW3QcnfKLKata56IZu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715336917, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-10T10:28:38.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122f03f700 1 Processor -- start 2024-05-10T10:28:38.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122f03f700 1 -- start start 2024-05-10T10:28:38.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122f03f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f122810bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122f03f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f122807ce10 con 0x7f1228109760 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122cddb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f122810bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122cddb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f122810bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55426/0 (socket says 172.21.15.112:55426) 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122cddb700 1 -- 172.21.15.112:0/1467582446 learned_addr learned my addr 172.21.15.112:0/1467582446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.796+0000 7f122cddb700 1 -- 172.21.15.112:0/1467582446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122810c090 con 0x7f1228109760 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122cddb700 1 --2- 172.21.15.112:0/1467582446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f122810bb50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f121c0098a0 tx=0x7f121c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee27d8f33b9cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f12277fe700 1 -- 172.21.15.112:0/1467582446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c004160 con 0x7f1228109760 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f12277fe700 1 -- 172.21.15.112:0/1467582446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f121c004300 con 0x7f1228109760 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f12277fe700 1 -- 172.21.15.112:0/1467582446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c011960 con 0x7f1228109760 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122f03f700 1 -- 172.21.15.112:0/1467582446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 msgr2=0x7f122810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122f03f700 1 --2- 172.21.15.112:0/1467582446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f122810bb50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f121c0098a0 tx=0x7f121c009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122f03f700 1 -- 172.21.15.112:0/1467582446 shutdown_connections 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122f03f700 1 --2- 172.21.15.112:0/1467582446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f122810bb50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122f03f700 1 -- 172.21.15.112:0/1467582446 >> 172.21.15.112:0/1467582446 conn(0x7f12281031b0 msgr2=0x7f12281055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122f03f700 1 -- 172.21.15.112:0/1467582446 shutdown_connections 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.797+0000 7f122f03f700 1 -- 172.21.15.112:0/1467582446 wait complete. 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.798+0000 7f122f03f700 1 Processor -- start 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.798+0000 7f122f03f700 1 -- start start 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.798+0000 7f122f03f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f1228108fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.798+0000 7f122f03f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f122810cfe0 con 0x7f1228109760 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.798+0000 7f122cddb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f1228108fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.798+0000 7f122cddb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f1228108fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55430/0 (socket says 172.21.15.112:55430) 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.798+0000 7f122cddb700 1 -- 172.21.15.112:0/1483458061 learned_addr learned my addr 172.21.15.112:0/1483458061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.799+0000 7f122cddb700 1 -- 172.21.15.112:0/1483458061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1228109520 con 0x7f1228109760 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.799+0000 7f122cddb700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f1228108fe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f121c003e50 tx=0x7f121c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.799+0000 7f1225ffb700 1 -- 172.21.15.112:0/1483458061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c003a60 con 0x7f1228109760 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.799+0000 7f1225ffb700 1 -- 172.21.15.112:0/1483458061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f121c01b8d0 con 0x7f1228109760 2024-05-10T10:28:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.799+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1228107710 con 0x7f1228109760 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.799+0000 7f1225ffb700 1 -- 172.21.15.112:0/1483458061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c0237d0 con 0x7f1228109760 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.799+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1228107c50 con 0x7f1228109760 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.800+0000 7f1225ffb700 1 -- 172.21.15.112:0/1483458061 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f121c02c3f0 con 0x7f1228109760 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.800+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1228064b00 con 0x7f1228109760 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.800+0000 7f1225ffb700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f121803e4d0 0x7f1218040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.800+0000 7f1225ffb700 1 -- 172.21.15.112:0/1483458061 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f121c054280 con 0x7f1228109760 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.800+0000 7f1227fff700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f121803e4d0 0x7f1218040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.801+0000 7f1227fff700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f121803e4d0 0x7f1218040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1214006fd0 tx=0x7f1214006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:36.805+0000 7f1225ffb700 1 -- 172.21.15.112:0/1483458061 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f121c01f030 con 0x7f1228109760 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.041+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f12281083c0 con 0x7f121803e4d0 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.265+0000 7f1225ffb700 1 -- 172.21.15.112:0/1483458061 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f12281083c0 con 0x7f121803e4d0 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f121803e4d0 msgr2=0x7f1218040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:38.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f121803e4d0 0x7f1218040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1214006fd0 tx=0x7f1214006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 msgr2=0x7f1228108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f1228108fe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f121c003e50 tx=0x7f121c011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 shutdown_connections 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f121803e4d0 0x7f1218040990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 --2- 172.21.15.112:0/1483458061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1228109760 0x7f1228108fe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 >> 172.21.15.112:0/1483458061 conn(0x7f12281031b0 msgr2=0x7f1228103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 shutdown_connections 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:37.268+0000 7f122f03f700 1 -- 172.21.15.112:0/1483458061 wait complete. 2024-05-10T10:28:38.031 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-05-10T10:28:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:38 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2779980860' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-10T10:28:39.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-05-10T10:28:39.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.431+0000 7fb7ccbef700 1 Processor -- start 2024-05-10T10:28:39.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.431+0000 7fb7ccbef700 1 -- start start 2024-05-10T10:28:39.378 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.431+0000 7fb7ccbef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c810baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.431+0000 7fb7ccbef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c807ce30 con 0x7fb7c8109700 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.431+0000 7fb7c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c810baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.431+0000 7fb7c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c810baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55436/0 (socket says 172.21.15.112:55436) 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.431+0000 7fb7c659c700 1 -- 172.21.15.112:0/3726496463 learned_addr learned my addr 172.21.15.112:0/3726496463 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.432+0000 7fb7c659c700 1 -- 172.21.15.112:0/3726496463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c810c030 con 0x7fb7c8109700 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.432+0000 7fb7c659c700 1 --2- 172.21.15.112:0/3726496463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c810baf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80098a0 tx=0x7fb7b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8623f444ac65fe70 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.432+0000 7fb7c559a700 1 -- 172.21.15.112:0/3726496463 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b8004160 con 0x7fb7c8109700 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.432+0000 7fb7c559a700 1 -- 172.21.15.112:0/3726496463 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7b8004300 con 0x7fb7c8109700 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.432+0000 7fb7c559a700 1 -- 172.21.15.112:0/3726496463 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b8011960 con 0x7fb7c8109700 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.432+0000 7fb7ccbef700 1 -- 172.21.15.112:0/3726496463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 msgr2=0x7fb7c810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.432+0000 7fb7ccbef700 1 --2- 172.21.15.112:0/3726496463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c810baf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80098a0 tx=0x7fb7b8009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.433+0000 7fb7ccbef700 1 -- 172.21.15.112:0/3726496463 shutdown_connections 2024-05-10T10:28:39.379 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.433+0000 7fb7ccbef700 1 --2- 172.21.15.112:0/3726496463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c810baf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.433+0000 7fb7ccbef700 1 -- 172.21.15.112:0/3726496463 >> 172.21.15.112:0/3726496463 conn(0x7fb7c81030f0 msgr2=0x7fb7c8105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.433+0000 7fb7ccbef700 1 -- 172.21.15.112:0/3726496463 shutdown_connections 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.433+0000 7fb7ccbef700 1 -- 172.21.15.112:0/3726496463 wait complete. 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.433+0000 7fb7ccbef700 1 Processor -- start 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.433+0000 7fb7ccbef700 1 -- start start 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7ccbef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7ccbef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c810cfd0 con 0x7fb7c8109700 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7c659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55446/0 (socket says 172.21.15.112:55446) 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7c659c700 1 -- 172.21.15.112:0/2779980860 learned_addr learned my addr 172.21.15.112:0/2779980860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:39.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7c659c700 1 -- 172.21.15.112:0/2779980860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c819f280 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7c659c700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c819ed40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8004660 tx=0x7fb7b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7b77fe700 1 -- 172.21.15.112:0/2779980860 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b8003aa0 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c819f510 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.434+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c81a2200 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.435+0000 7fb7b77fe700 1 -- 172.21.15.112:0/2779980860 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7b8004300 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.435+0000 7fb7b77fe700 1 -- 172.21.15.112:0/2779980860 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b8023400 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.435+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a8005330 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.436+0000 7fb7b77fe700 1 -- 172.21.15.112:0/2779980860 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fb7b80235a0 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.436+0000 7fb7b77fe700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb7b003e470 0x7fb7b0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.436+0000 7fb7b77fe700 1 -- 172.21.15.112:0/2779980860 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7b8053fa0 con 0x7fb7c8109700 2024-05-10T10:28:39.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.436+0000 7fb7c5d9b700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb7b003e470 0x7fb7b0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.436+0000 7fb7c5d9b700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb7b003e470 0x7fb7b0040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006fd0 tx=0x7fb7bc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.441+0000 7fb7b77fe700 1 -- 172.21.15.112:0/2779980860 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb7b801f030 con 0x7fb7c8109700 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.661+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb7a8005620 con 0x7fb7c8109700 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.661+0000 7fb7b77fe700 1 -- 172.21.15.112:0/2779980860 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb7b8019ce0 con 0x7fb7c8109700 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb7b003e470 msgr2=0x7fb7b0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb7b003e470 0x7fb7b0040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006fd0 tx=0x7fb7bc006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 msgr2=0x7fb7c819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c819ed40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8004660 tx=0x7fb7b8011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 shutdown_connections 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb7b003e470 0x7fb7b0040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 --2- 172.21.15.112:0/2779980860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7c8109700 0x7fb7c819ed40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:39.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 >> 172.21.15.112:0/2779980860 conn(0x7fb7c81030f0 msgr2=0x7fb7c8103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:39.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 shutdown_connections 2024-05-10T10:28:39.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:38.664+0000 7fb7ccbef700 1 -- 172.21.15.112:0/2779980860 wait complete. 2024-05-10T10:28:39.387 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T10:28:39.387 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-10T10:28:39.396 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T10:28:39.396 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-10T10:28:39.396 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-10T10:28:39.396 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-10T10:28:39.398 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-05-10T10:28:39.398 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:39.398 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112.front.sepia.ceph.com:8443/ 2024-05-10T10:28:39.398 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-05-10T10:28:39.398 INFO:teuthology.orchestra.run.smithi112.stdout: Password: mymmdfbn1k 2024-05-10T10:28:39.398 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:39.399 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config directory 2024-05-10T10:28:39.399 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling autotune for osd_memory_target 2024-05-10T10:28:40.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.839+0000 7f574fe18700 1 Processor -- start 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.839+0000 7f574fe18700 1 -- start start 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.839+0000 7f574fe18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f5748107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.839+0000 7f574fe18700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f574807cdd0 con 0x7f5748107570 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.839+0000 7f574dbb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f5748107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.839+0000 7f574dbb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f5748107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55450/0 (socket says 172.21.15.112:55450) 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.839+0000 7f574dbb4700 1 -- 172.21.15.112:0/2439890914 learned_addr learned my addr 172.21.15.112:0/2439890914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.840+0000 7f574dbb4700 1 -- 172.21.15.112:0/2439890914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5748107ed0 con 0x7f5748107570 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.840+0000 7f574dbb4700 1 --2- 172.21.15.112:0/2439890914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f5748107990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57440098a0 tx=0x7f5744009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baea61367781ca2c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.840+0000 7f574cbb2700 1 -- 172.21.15.112:0/2439890914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5744004160 con 0x7f5748107570 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.840+0000 7f574cbb2700 1 -- 172.21.15.112:0/2439890914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5744004300 con 0x7f5748107570 2024-05-10T10:28:40.618 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.840+0000 7f574cbb2700 1 -- 172.21.15.112:0/2439890914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5744011960 con 0x7f5748107570 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.840+0000 7f574fe18700 1 -- 172.21.15.112:0/2439890914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 msgr2=0x7f5748107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.840+0000 7f574fe18700 1 --2- 172.21.15.112:0/2439890914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f5748107990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57440098a0 tx=0x7f5744009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 -- 172.21.15.112:0/2439890914 shutdown_connections 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 --2- 172.21.15.112:0/2439890914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f5748107990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 -- 172.21.15.112:0/2439890914 >> 172.21.15.112:0/2439890914 conn(0x7f57481030f0 msgr2=0x7f5748105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 -- 172.21.15.112:0/2439890914 shutdown_connections 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 -- 172.21.15.112:0/2439890914 wait complete. 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 Processor -- start 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 -- start start 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f574819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.841+0000 7f574fe18700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5748108ee0 con 0x7f5748107570 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f574dbb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f574819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f574dbb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f574819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55462/0 (socket says 172.21.15.112:55462) 2024-05-10T10:28:40.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f574dbb4700 1 -- 172.21.15.112:0/3564977856 learned_addr learned my addr 172.21.15.112:0/3564977856 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f574dbb4700 1 -- 172.21.15.112:0/3564977856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574819f220 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f574dbb4700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f574819ece0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5744011670 tx=0x7f5744011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f573effd700 1 -- 172.21.15.112:0/3564977856 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f574401f070 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f573effd700 1 -- 172.21.15.112:0/3564977856 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5744003b20 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574819f4b0 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f573effd700 1 -- 172.21.15.112:0/3564977856 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57440195f0 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.842+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57481a2190 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.843+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5748064b00 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.843+0000 7f573effd700 1 -- 172.21.15.112:0/3564977856 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f5744022070 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.843+0000 7f573effd700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f573403e5c0 0x7f5734040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.843+0000 7f573effd700 1 -- 172.21.15.112:0/3564977856 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5744053cf0 con 0x7f5748107570 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.843+0000 7f574d3b3700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f573403e5c0 0x7f5734040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:40.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.844+0000 7f574d3b3700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f573403e5c0 0x7f5734040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5738006fd0 tx=0x7f5738006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:39.848+0000 7f573effd700 1 -- 172.21.15.112:0/3564977856 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f57440172b0 con 0x7f5748107570 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.048+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f574802e810 con 0x7f5748107570 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.048+0000 7f573effd700 1 -- 172.21.15.112:0/3564977856 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f574401baf0 con 0x7f5748107570 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f573403e5c0 msgr2=0x7f5734040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f573403e5c0 0x7f5734040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5738006fd0 tx=0x7f5738006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 msgr2=0x7f574819ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f574819ece0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5744011670 tx=0x7f5744011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 shutdown_connections 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f573403e5c0 0x7f5734040a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 --2- 172.21.15.112:0/3564977856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5748107570 0x7f574819ece0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.050+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 >> 172.21.15.112:0/3564977856 conn(0x7f57481030f0 msgr2=0x7f57481959e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.051+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 shutdown_connections 2024-05-10T10:28:40.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:40.051+0000 7f574fe18700 1 -- 172.21.15.112:0/3564977856 wait complete. 2024-05-10T10:28:40.937 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:40 smithi112 ceph-mon[30100]: mgrmap e13: x(active, since 6s) 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.038+0000 7f3ab000e700 1 Processor -- start 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.038+0000 7f3ab000e700 1 -- start start 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.038+0000 7f3ab000e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa810bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.038+0000 7f3ab000e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa807ce10 con 0x7f3aa8109740 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aaddaa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa810bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aaddaa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa810bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55464/0 (socket says 172.21.15.112:55464) 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aaddaa700 1 -- 172.21.15.112:0/1927861773 learned_addr learned my addr 172.21.15.112:0/1927861773 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aaddaa700 1 -- 172.21.15.112:0/1927861773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa810c070 con 0x7f3aa8109740 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aaddaa700 1 --2- 172.21.15.112:0/1927861773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa810bb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40098a0 tx=0x7f3aa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9af3dd19969f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aacda8700 1 -- 172.21.15.112:0/1927861773 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa4004160 con 0x7f3aa8109740 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aacda8700 1 -- 172.21.15.112:0/1927861773 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3aa4004300 con 0x7f3aa8109740 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.039+0000 7f3aacda8700 1 -- 172.21.15.112:0/1927861773 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa4011960 con 0x7f3aa8109740 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 -- 172.21.15.112:0/1927861773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 msgr2=0x7f3aa810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 --2- 172.21.15.112:0/1927861773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa810bb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40098a0 tx=0x7f3aa4009bb0 comp rx=0 tx=0).stop 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 -- 172.21.15.112:0/1927861773 shutdown_connections 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 --2- 172.21.15.112:0/1927861773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa810bb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 -- 172.21.15.112:0/1927861773 >> 172.21.15.112:0/1927861773 conn(0x7f3aa81030f0 msgr2=0x7f3aa8105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:41.938 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 -- 172.21.15.112:0/1927861773 shutdown_connections 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 -- 172.21.15.112:0/1927861773 wait complete. 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 Processor -- start 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.040+0000 7f3ab000e700 1 -- start start 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3ab000e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa8108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3ab000e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810cfc0 con 0x7f3aa8109740 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3aaddaa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa8108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3aaddaa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa8108fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55470/0 (socket says 172.21.15.112:55470) 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3aaddaa700 1 -- 172.21.15.112:0/1864739609 learned_addr learned my addr 172.21.15.112:0/1864739609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3aaddaa700 1 -- 172.21.15.112:0/1864739609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa8109500 con 0x7f3aa8109740 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3aaddaa700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa8108fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4004660 tx=0x7f3aa4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3a9effd700 1 -- 172.21.15.112:0/1864739609 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa4003aa0 con 0x7f3aa8109740 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3a9effd700 1 -- 172.21.15.112:0/1864739609 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3aa401b8d0 con 0x7f3aa8109740 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa81076f0 con 0x7f3aa8109740 2024-05-10T10:28:41.939 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3a9effd700 1 -- 172.21.15.112:0/1864739609 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa4023820 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.041+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa8107c30 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.042+0000 7f3a9effd700 1 -- 172.21.15.112:0/1864739609 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3aa401b440 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.042+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa8064b00 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.042+0000 7f3a9effd700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3a9403e570 0x7f3a94040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.043+0000 7f3a9effd700 1 -- 172.21.15.112:0/1864739609 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3aa4053fa0 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.043+0000 7f3aad5a9700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3a9403e570 0x7f3a94040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.043+0000 7f3aad5a9700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3a9403e570 0x7f3a94040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.047+0000 7f3a9effd700 1 -- 172.21.15.112:0/1864739609 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3aa401f030 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.362+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3aa8107eb0 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.365+0000 7f3a9effd700 1 -- 172.21.15.112:0/1864739609 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f3aa40239c0 con 0x7f3aa8109740 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3a9403e570 msgr2=0x7f3a94040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3a9403e570 0x7f3a94040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:41.940 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 msgr2=0x7f3aa8108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa8108fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4004660 tx=0x7f3aa4011a30 comp rx=0 tx=0).stop 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 shutdown_connections 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3a9403e570 0x7f3a94040a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 --2- 172.21.15.112:0/1864739609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa8109740 0x7f3aa8108fc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.367+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 >> 172.21.15.112:0/1864739609 conn(0x7f3aa81030f0 msgr2=0x7f3aa8103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.368+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 shutdown_connections 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-10T10:28:41.368+0000 7f3ab000e700 1 -- 172.21.15.112:0/1864739609 wait complete. 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-05-10T10:28:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:41.942 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-05-10T10:28:41.969 INFO:tasks.cephadm:Fetching config... 2024-05-10T10:28:41.969 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:28:41.970 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-10T10:28:41.985 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-10T10:28:41.985 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:28:41.985 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-10T10:28:42.040 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-10T10:28:42.041 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:28:42.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-10T10:28:42.105 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-10T10:28:42.105 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:28:42.105 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-10T10:28:42.161 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-10T10:28:42.162 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2hdUSu47XMzAMSNGOxfjVNEZMTm9TCpY4AeqDUzinCSXRTX4CTKj81XooOOwCdTepbszvHg+c5xeebnFClN7Oiwp2tdNGEoV9kT5HdVam3Nv5kVU2V8QtlDcsRXoFmgtGc6egp5RmasNY8PVC3nUFb2J6t+SfW784nyNMtspReiTbJutcYIhJbp2UMWRIGLf15pC5Ie5b5aThlEnEvjZhZnx0cB13XTS3M1EiJQW6KTFmEcJvy5ywshpxxVetigNohJhiqZGlqD1QTvdhAbW8/4T5oTf116kIBHsL2fOq3hYQr7OKL58WqfOHsERY6ugg37W99L4eCbnWoWdk721cbcgVNboERr/TLW09mRfnT8cS/mjHn2GAccafoKTMwaFjOTHa0r8ibn/b3DnF4MEOwlHnsDwnGx4jXSQXYZi/TeHoziNVWujvDQx1Dk67ccJ6cwk/9aesLIeylzY9UAzOVMrz/0D5Q98JUxiDgeWFqQV8gmIXdLFVbRxtAM5InWE= ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T10:28:42.250 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2hdUSu47XMzAMSNGOxfjVNEZMTm9TCpY4AeqDUzinCSXRTX4CTKj81XooOOwCdTepbszvHg+c5xeebnFClN7Oiwp2tdNGEoV9kT5HdVam3Nv5kVU2V8QtlDcsRXoFmgtGc6egp5RmasNY8PVC3nUFb2J6t+SfW784nyNMtspReiTbJutcYIhJbp2UMWRIGLf15pC5Ie5b5aThlEnEvjZhZnx0cB13XTS3M1EiJQW6KTFmEcJvy5ywshpxxVetigNohJhiqZGlqD1QTvdhAbW8/4T5oTf116kIBHsL2fOq3hYQr7OKL58WqfOHsERY6ugg37W99L4eCbnWoWdk721cbcgVNboERr/TLW09mRfnT8cS/mjHn2GAccafoKTMwaFjOTHa0r8ibn/b3DnF4MEOwlHnsDwnGx4jXSQXYZi/TeHoziNVWujvDQx1Dk67ccJ6cwk/9aesLIeylzY9UAzOVMrz/0D5Q98JUxiDgeWFqQV8gmIXdLFVbRxtAM5InWE= ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:28:42.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2hdUSu47XMzAMSNGOxfjVNEZMTm9TCpY4AeqDUzinCSXRTX4CTKj81XooOOwCdTepbszvHg+c5xeebnFClN7Oiwp2tdNGEoV9kT5HdVam3Nv5kVU2V8QtlDcsRXoFmgtGc6egp5RmasNY8PVC3nUFb2J6t+SfW784nyNMtspReiTbJutcYIhJbp2UMWRIGLf15pC5Ie5b5aThlEnEvjZhZnx0cB13XTS3M1EiJQW6KTFmEcJvy5ywshpxxVetigNohJhiqZGlqD1QTvdhAbW8/4T5oTf116kIBHsL2fOq3hYQr7OKL58WqfOHsERY6ugg37W99L4eCbnWoWdk721cbcgVNboERr/TLW09mRfnT8cS/mjHn2GAccafoKTMwaFjOTHa0r8ibn/b3DnF4MEOwlHnsDwnGx4jXSQXYZi/TeHoziNVWujvDQx1Dk67ccJ6cwk/9aesLIeylzY9UAzOVMrz/0D5Q98JUxiDgeWFqQV8gmIXdLFVbRxtAM5InWE= ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T10:28:42.305 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2hdUSu47XMzAMSNGOxfjVNEZMTm9TCpY4AeqDUzinCSXRTX4CTKj81XooOOwCdTepbszvHg+c5xeebnFClN7Oiwp2tdNGEoV9kT5HdVam3Nv5kVU2V8QtlDcsRXoFmgtGc6egp5RmasNY8PVC3nUFb2J6t+SfW784nyNMtspReiTbJutcYIhJbp2UMWRIGLf15pC5Ie5b5aThlEnEvjZhZnx0cB13XTS3M1EiJQW6KTFmEcJvy5ywshpxxVetigNohJhiqZGlqD1QTvdhAbW8/4T5oTf116kIBHsL2fOq3hYQr7OKL58WqfOHsERY6ugg37W99L4eCbnWoWdk721cbcgVNboERr/TLW09mRfnT8cS/mjHn2GAccafoKTMwaFjOTHa0r8ibn/b3DnF4MEOwlHnsDwnGx4jXSQXYZi/TeHoziNVWujvDQx1Dk67ccJ6cwk/9aesLIeylzY9UAzOVMrz/0D5Q98JUxiDgeWFqQV8gmIXdLFVbRxtAM5InWE= ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:28:42.317 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2hdUSu47XMzAMSNGOxfjVNEZMTm9TCpY4AeqDUzinCSXRTX4CTKj81XooOOwCdTepbszvHg+c5xeebnFClN7Oiwp2tdNGEoV9kT5HdVam3Nv5kVU2V8QtlDcsRXoFmgtGc6egp5RmasNY8PVC3nUFb2J6t+SfW784nyNMtspReiTbJutcYIhJbp2UMWRIGLf15pC5Ie5b5aThlEnEvjZhZnx0cB13XTS3M1EiJQW6KTFmEcJvy5ywshpxxVetigNohJhiqZGlqD1QTvdhAbW8/4T5oTf116kIBHsL2fOq3hYQr7OKL58WqfOHsERY6ugg37W99L4eCbnWoWdk721cbcgVNboERr/TLW09mRfnT8cS/mjHn2GAccafoKTMwaFjOTHa0r8ibn/b3DnF4MEOwlHnsDwnGx4jXSQXYZi/TeHoziNVWujvDQx1Dk67ccJ6cwk/9aesLIeylzY9UAzOVMrz/0D5Q98JUxiDgeWFqQV8gmIXdLFVbRxtAM5InWE= ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T10:28:42.359 INFO:teuthology.orchestra.run.smithi195.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2hdUSu47XMzAMSNGOxfjVNEZMTm9TCpY4AeqDUzinCSXRTX4CTKj81XooOOwCdTepbszvHg+c5xeebnFClN7Oiwp2tdNGEoV9kT5HdVam3Nv5kVU2V8QtlDcsRXoFmgtGc6egp5RmasNY8PVC3nUFb2J6t+SfW784nyNMtspReiTbJutcYIhJbp2UMWRIGLf15pC5Ie5b5aThlEnEvjZhZnx0cB13XTS3M1EiJQW6KTFmEcJvy5ywshpxxVetigNohJhiqZGlqD1QTvdhAbW8/4T5oTf116kIBHsL2fOq3hYQr7OKL58WqfOHsERY6ugg37W99L4eCbnWoWdk721cbcgVNboERr/TLW09mRfnT8cS/mjHn2GAccafoKTMwaFjOTHa0r8ibn/b3DnF4MEOwlHnsDwnGx4jXSQXYZi/TeHoziNVWujvDQx1Dk67ccJ6cwk/9aesLIeylzY9UAzOVMrz/0D5Q98JUxiDgeWFqQV8gmIXdLFVbRxtAM5InWE= ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:28:42.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-10T10:28:42.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:42 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1864739609' entity='client.admin' 2024-05-10T10:28:42.613 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:28:44.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.690+0000 7f8062e39700 1 -- 172.21.15.112:0/1258167975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 msgr2=0x7f805c1037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:44.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.690+0000 7f8062e39700 1 --2- 172.21.15.112:0/1258167975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c1037c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8050009910 tx=0x7f8050009c20 comp rx=0 tx=0).stop 2024-05-10T10:28:44.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.690+0000 7f8062e39700 1 -- 172.21.15.112:0/1258167975 shutdown_connections 2024-05-10T10:28:44.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.690+0000 7f8062e39700 1 --2- 172.21.15.112:0/1258167975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c1037c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:44.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.690+0000 7f8062e39700 1 -- 172.21.15.112:0/1258167975 >> 172.21.15.112:0/1258167975 conn(0x7f805c0fcb50 msgr2=0x7f805c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:44.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.690+0000 7f8062e39700 1 -- 172.21.15.112:0/1258167975 shutdown_connections 2024-05-10T10:28:44.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.690+0000 7f8062e39700 1 -- 172.21.15.112:0/1258167975 wait complete. 2024-05-10T10:28:44.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.691+0000 7f8062e39700 1 Processor -- start 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.691+0000 7f8062e39700 1 -- start start 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.691+0000 7f8062e39700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.691+0000 7f8062e39700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f805c107c00 con 0x7f805c1033a0 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.691+0000 7f8060bd5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.691+0000 7f8060bd5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55486/0 (socket says 172.21.15.112:55486) 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.691+0000 7f8060bd5700 1 -- 172.21.15.112:0/1174400830 learned_addr learned my addr 172.21.15.112:0/1174400830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.692+0000 7f8060bd5700 1 -- 172.21.15.112:0/1174400830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c198fb0 con 0x7f805c1033a0 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.692+0000 7f8060bd5700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c101a50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8050005790 tx=0x7f80500057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.692+0000 7f8059ffb700 1 -- 172.21.15.112:0/1174400830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8050027070 con 0x7f805c1033a0 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.692+0000 7f8059ffb700 1 -- 172.21.15.112:0/1174400830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f805000bdf0 con 0x7f805c1033a0 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.692+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c19d5c0 con 0x7f805c1033a0 2024-05-10T10:28:44.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.692+0000 7f8059ffb700 1 -- 172.21.15.112:0/1174400830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f805000fe50 con 0x7f805c1033a0 2024-05-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.692+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c19d960 con 0x7f805c1033a0 2024-05-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.693+0000 7f8059ffb700 1 -- 172.21.15.112:0/1174400830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f805002a070 con 0x7f805c1033a0 2024-05-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.693+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c108380 con 0x7f805c1033a0 2024-05-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.693+0000 7f8059ffb700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f804c03e710 0x7f804c040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.693+0000 7f8059ffb700 1 -- 172.21.15.112:0/1174400830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f80500538e0 con 0x7f805c1033a0 2024-05-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.693+0000 7f805bfff700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f804c03e710 0x7f804c040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:44.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.694+0000 7f805bfff700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f804c03e710 0x7f804c040bd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8048006fd0 tx=0x7f8048006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:44.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.698+0000 7f8059ffb700 1 -- 172.21.15.112:0/1174400830 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8050013070 con 0x7f805c1033a0 2024-05-10T10:28:44.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.898+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f805c02e710 con 0x7f805c1033a0 2024-05-10T10:28:44.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.904+0000 7f8059ffb700 1 -- 172.21.15.112:0/1174400830 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f805002a360 con 0x7f805c1033a0 2024-05-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f804c03e710 msgr2=0x7f804c040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f804c03e710 0x7f804c040bd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8048006fd0 tx=0x7f8048006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 msgr2=0x7f805c101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c101a50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8050005790 tx=0x7f80500057c0 comp rx=0 tx=0).stop 2024-05-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 shutdown_connections 2024-05-10T10:28:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f804c03e710 0x7f804c040bd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 --2- 172.21.15.112:0/1174400830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1033a0 0x7f805c101a50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 >> 172.21.15.112:0/1174400830 conn(0x7f805c0fcb50 msgr2=0x7f805c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 shutdown_connections 2024-05-10T10:28:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:44.907+0000 7f8062e39700 1 -- 172.21.15.112:0/1174400830 wait complete. 2024-05-10T10:28:45.457 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-10T10:28:45.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-10T10:28:45.693 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:28:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:28:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1174400830' entity='client.admin' 2024-05-10T10:28:45.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:45.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.207+0000 7ff34794b700 1 -- 172.21.15.112:0/136784868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 msgr2=0x7ff340103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.207+0000 7ff34794b700 1 --2- 172.21.15.112:0/136784868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340103890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00b0a0 tx=0x7ff33c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.207+0000 7ff34794b700 1 -- 172.21.15.112:0/136784868 shutdown_connections 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.207+0000 7ff34794b700 1 --2- 172.21.15.112:0/136784868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340103890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.207+0000 7ff34794b700 1 -- 172.21.15.112:0/136784868 >> 172.21.15.112:0/136784868 conn(0x7ff3400fcb80 msgr2=0x7ff3400fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.207+0000 7ff34794b700 1 -- 172.21.15.112:0/136784868 shutdown_connections 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.207+0000 7ff34794b700 1 -- 172.21.15.112:0/136784868 wait complete. 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff34794b700 1 Processor -- start 2024-05-10T10:28:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff34794b700 1 -- start start 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff34794b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff34794b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff340107bf0 con 0x7ff340103470 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff3456e7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff3456e7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340101a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34336/0 (socket says 172.21.15.112:34336) 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff3456e7700 1 -- 172.21.15.112:0/1504294838 learned_addr learned my addr 172.21.15.112:0/1504294838 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff3456e7700 1 -- 172.21.15.112:0/1504294838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34019b1f0 con 0x7ff340103470 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff3456e7700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340101a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009cd0 tx=0x7ff33c007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.208+0000 7ff3367fc700 1 -- 172.21.15.112:0/1504294838 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff33c017040 con 0x7ff340103470 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.209+0000 7ff3367fc700 1 -- 172.21.15.112:0/1504294838 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff33c022070 con 0x7ff340103470 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.209+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34019b3c0 con 0x7ff340103470 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.209+0000 7ff3367fc700 1 -- 172.21.15.112:0/1504294838 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff33c01dd50 con 0x7ff340103470 2024-05-10T10:28:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.209+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34019b760 con 0x7ff340103470 2024-05-10T10:28:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.209+0000 7ff3367fc700 1 -- 172.21.15.112:0/1504294838 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7ff33c027460 con 0x7ff340103470 2024-05-10T10:28:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.209+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff340108370 con 0x7ff340103470 2024-05-10T10:28:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.210+0000 7ff3367fc700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff32c03e560 0x7ff32c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:47.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.210+0000 7ff3367fc700 1 -- 172.21.15.112:0/1504294838 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff33c00f050 con 0x7ff340103470 2024-05-10T10:28:47.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.210+0000 7ff344ee6700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff32c03e560 0x7ff32c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:47.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.210+0000 7ff344ee6700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff32c03e560 0x7ff32c040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:47.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.215+0000 7ff3367fc700 1 -- 172.21.15.112:0/1504294838 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff33c025bd0 con 0x7ff340103470 2024-05-10T10:28:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.419+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff34002e6c0 con 0x7ff32c03e560 2024-05-10T10:28:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.422+0000 7ff3367fc700 1 -- 172.21.15.112:0/1504294838 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff34002e6c0 con 0x7ff32c03e560 2024-05-10T10:28:47.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff32c03e560 msgr2=0x7ff32c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:47.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff32c03e560 0x7ff32c040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:47.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 msgr2=0x7ff340101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:47.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340101a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009cd0 tx=0x7ff33c007b40 comp rx=0 tx=0).stop 2024-05-10T10:28:47.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 shutdown_connections 2024-05-10T10:28:47.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff32c03e560 0x7ff32c040a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:47.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 --2- 172.21.15.112:0/1504294838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff340103470 0x7ff340101a40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.424+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 >> 172.21.15.112:0/1504294838 conn(0x7ff3400fcb80 msgr2=0x7ff3400fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.425+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 shutdown_connections 2024-05-10T10:28:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:47.425+0000 7ff34794b700 1 -- 172.21.15.112:0/1504294838 wait complete. 2024-05-10T10:28:48.166 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-05-10T10:28:48.166 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:28:48.166 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-05-10T10:28:48.186 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:28:48.186 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:28:48.244 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-05-10T10:28:48.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch host add smithi148 2024-05-10T10:28:48.482 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: Updating smithi112:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:48.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:48.687 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:49.645 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:49 smithi112 ceph-mon[30100]: Updating smithi112:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T10:28:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 -- 172.21.15.112:0/490271684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 msgr2=0x7f3014104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 --2- 172.21.15.112:0/490271684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f3014104460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3004009910 tx=0x7f3004009c20 comp rx=0 tx=0).stop 2024-05-10T10:28:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 -- 172.21.15.112:0/490271684 shutdown_connections 2024-05-10T10:28:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 --2- 172.21.15.112:0/490271684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f3014104460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 -- 172.21.15.112:0/490271684 >> 172.21.15.112:0/490271684 conn(0x7f30140fd8d0 msgr2=0x7f30140ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:49.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 -- 172.21.15.112:0/490271684 shutdown_connections 2024-05-10T10:28:49.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 -- 172.21.15.112:0/490271684 wait complete. 2024-05-10T10:28:49.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 Processor -- start 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.849+0000 7f301a979700 1 -- start start 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f301a979700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f30141a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f301a979700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f301410d2b0 con 0x7f3014104080 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f3013fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f30141a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f3013fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f30141a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34356/0 (socket says 172.21.15.112:34356) 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f3013fff700 1 -- 172.21.15.112:0/1995124697 learned_addr learned my addr 172.21.15.112:0/1995124697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f3013fff700 1 -- 172.21.15.112:0/1995124697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30141a0c00 con 0x7f3014104080 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f3013fff700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f30141a06c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3004005060 tx=0x7f3004005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f30117fa700 1 -- 172.21.15.112:0/1995124697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3004026070 con 0x7f3014104080 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f30117fa700 1 -- 172.21.15.112:0/1995124697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f300400b750 con 0x7f3014104080 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30141a0e90 con 0x7f3014104080 2024-05-10T10:28:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f30117fa700 1 -- 172.21.15.112:0/1995124697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3004003850 con 0x7f3014104080 2024-05-10T10:28:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.850+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301419a880 con 0x7f3014104080 2024-05-10T10:28:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.851+0000 7f30117fa700 1 -- 172.21.15.112:0/1995124697 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3004029070 con 0x7f3014104080 2024-05-10T10:28:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.851+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301406bb80 con 0x7f3014104080 2024-05-10T10:28:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.851+0000 7f30117fa700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ffc03e570 0x7f2ffc040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.852+0000 7f30117fa700 1 -- 172.21.15.112:0/1995124697 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f300401e070 con 0x7f3014104080 2024-05-10T10:28:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.852+0000 7f30137fe700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ffc03e570 0x7f2ffc040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.852+0000 7f30137fe700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ffc03e570 0x7f2ffc040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3008006fd0 tx=0x7f3008006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:49.856+0000 7f30117fa700 1 -- 172.21.15.112:0/1995124697 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3004013070 con 0x7f3014104080 2024-05-10T10:28:50.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:50.062+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) -- 0x7f301419bdb0 con 0x7f2ffc03e570 2024-05-10T10:28:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:51 smithi112 ceph-mon[30100]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:28:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:51 smithi112 ceph-mon[30100]: Deploying cephadm binary to smithi148 2024-05-10T10:28:54.155 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-05-10T10:28:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.154+0000 7f30117fa700 1 -- 172.21.15.112:0/1995124697 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f301419bdb0 con 0x7f2ffc03e570 2024-05-10T10:28:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ffc03e570 msgr2=0x7f2ffc040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ffc03e570 0x7f2ffc040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3008006fd0 tx=0x7f3008006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 msgr2=0x7f30141a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f30141a06c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3004005060 tx=0x7f3004005710 comp rx=0 tx=0).stop 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 shutdown_connections 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ffc03e570 0x7f2ffc040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 --2- 172.21.15.112:0/1995124697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3014104080 0x7f30141a06c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 >> 172.21.15.112:0/1995124697 conn(0x7f30140fd8d0 msgr2=0x7f3014109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 shutdown_connections 2024-05-10T10:28:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:54.156+0000 7f301a979700 1 -- 172.21.15.112:0/1995124697 wait complete. 2024-05-10T10:28:54.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:54 smithi112 ceph-mon[30100]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:28:54.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:54 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:54.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T10:28:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:28:55.428 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:55 smithi112 ceph-mon[30100]: Added host smithi148 2024-05-10T10:28:55.428 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:28:55.428 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:56.307 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:56 smithi112 ceph-mon[30100]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:28:56.307 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:56 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:56.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 -- 172.21.15.112:0/2530296855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 msgr2=0x7f4528075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 --2- 172.21.15.112:0/2530296855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f4528075c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4524009910 tx=0x7f4524009c20 comp rx=0 tx=0).stop 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 -- 172.21.15.112:0/2530296855 shutdown_connections 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 --2- 172.21.15.112:0/2530296855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f4528075c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 -- 172.21.15.112:0/2530296855 >> 172.21.15.112:0/2530296855 conn(0x7f45280fe9c0 msgr2=0x7f4528100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 -- 172.21.15.112:0/2530296855 shutdown_connections 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 -- 172.21.15.112:0/2530296855 wait complete. 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 Processor -- start 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.517+0000 7f45303b2700 1 -- start start 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f45303b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f452819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f45303b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4528113890 con 0x7f4528075840 2024-05-10T10:28:56.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f452e14e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f452819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f452e14e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f452819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58424/0 (socket says 172.21.15.112:58424) 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f452e14e700 1 -- 172.21.15.112:0/80046257 learned_addr learned my addr 172.21.15.112:0/80046257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f452e14e700 1 -- 172.21.15.112:0/80046257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452819f3f0 con 0x7f4528075840 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f452e14e700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f452819eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4524005060 tx=0x7f4524005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f451f7fe700 1 -- 172.21.15.112:0/80046257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4524026070 con 0x7f4528075840 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452819f680 con 0x7f4528075840 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f451f7fe700 1 -- 172.21.15.112:0/80046257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f452400b750 con 0x7f4528075840 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f451f7fe700 1 -- 172.21.15.112:0/80046257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4524003850 con 0x7f4528075840 2024-05-10T10:28:56.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.518+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452819fae0 con 0x7f4528075840 2024-05-10T10:28:56.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.519+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4528076f80 con 0x7f4528075840 2024-05-10T10:28:56.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.519+0000 7f451f7fe700 1 -- 172.21.15.112:0/80046257 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f4524029070 con 0x7f4528075840 2024-05-10T10:28:56.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.519+0000 7f451f7fe700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f451403e510 0x7f45140409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:56.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.520+0000 7f451f7fe700 1 -- 172.21.15.112:0/80046257 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f452401e070 con 0x7f4528075840 2024-05-10T10:28:56.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.520+0000 7f452d94d700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f451403e510 0x7f45140409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:56.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.520+0000 7f452d94d700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f451403e510 0x7f45140409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:56.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.525+0000 7f451f7fe700 1 -- 172.21.15.112:0/80046257 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4524013070 con 0x7f4528075840 2024-05-10T10:28:56.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.735+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45281a0230 con 0x7f451403e510 2024-05-10T10:28:56.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.736+0000 7f451f7fe700 1 -- 172.21.15.112:0/80046257 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f45281a0230 con 0x7f451403e510 2024-05-10T10:28:56.737 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:28:56.737 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-05-10T10:28:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f451403e510 msgr2=0x7f45140409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f451403e510 0x7f45140409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518006e40 comp rx=0 tx=0).stop 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 msgr2=0x7f452819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f452819eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4524005060 tx=0x7f4524005710 comp rx=0 tx=0).stop 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 shutdown_connections 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f451403e510 0x7f45140409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 --2- 172.21.15.112:0/80046257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4528075840 0x7f452819eeb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 >> 172.21.15.112:0/80046257 conn(0x7f45280fe9c0 msgr2=0x7f45280feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 shutdown_connections 2024-05-10T10:28:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:56.738+0000 7f45303b2700 1 -- 172.21.15.112:0/80046257 wait complete. 2024-05-10T10:28:57.340 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi195 2024-05-10T10:28:57.340 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:28:57.341 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.conf 2024-05-10T10:28:57.361 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:28:57.361 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:28:57.419 INFO:tasks.cephadm:Adding host smithi195 to orchestrator... 2024-05-10T10:28:57.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch host add smithi195 2024-05-10T10:28:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:28:58.057 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:57 smithi112 ceph-mon[30100]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:28:58.057 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:28:59.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.959+0000 7f759f250700 1 -- 172.21.15.112:0/762451100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 msgr2=0x7f75981034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:28:59.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.959+0000 7f759f250700 1 --2- 172.21.15.112:0/762451100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f75981034d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f758c009910 tx=0x7f758c009c20 comp rx=0 tx=0).stop 2024-05-10T10:28:59.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.960+0000 7f759f250700 1 -- 172.21.15.112:0/762451100 shutdown_connections 2024-05-10T10:28:59.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.960+0000 7f759f250700 1 --2- 172.21.15.112:0/762451100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f75981034d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.960+0000 7f759f250700 1 -- 172.21.15.112:0/762451100 >> 172.21.15.112:0/762451100 conn(0x7f75980fe9c0 msgr2=0x7f7598100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.960+0000 7f759f250700 1 -- 172.21.15.112:0/762451100 shutdown_connections 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.960+0000 7f759f250700 1 -- 172.21.15.112:0/762451100 wait complete. 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.960+0000 7f759f250700 1 Processor -- start 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.960+0000 7f759f250700 1 -- start start 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759f250700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f759819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759f250700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f759810d220 con 0x7f75981030f0 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759cfec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f759819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759cfec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f759819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58436/0 (socket says 172.21.15.112:58436) 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759cfec700 1 -- 172.21.15.112:0/1111117229 learned_addr learned my addr 172.21.15.112:0/1111117229 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759cfec700 1 -- 172.21.15.112:0/1111117229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759819eb20 con 0x7f75981030f0 2024-05-10T10:28:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759cfec700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f759819e5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f758c005660 tx=0x7f758c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f7595ffb700 1 -- 172.21.15.112:0/1111117229 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c026070 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f7595ffb700 1 -- 172.21.15.112:0/1111117229 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f758c00b6f0 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759819ee10 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f7595ffb700 1 -- 172.21.15.112:0/1111117229 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c0037b0 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.961+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7598198880 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.962+0000 7f7595ffb700 1 -- 172.21.15.112:0/1111117229 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f758c029070 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.962+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75981045d0 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.962+0000 7f7595ffb700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f758803e570 0x7f7588040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.963+0000 7f7595ffb700 1 -- 172.21.15.112:0/1111117229 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f758c01e070 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.963+0000 7f7597fff700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f758803e570 0x7f7588040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.963+0000 7f7597fff700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f758803e570 0x7f7588040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7584006fd0 tx=0x7f7584006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:28:59.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:58.967+0000 7f7595ffb700 1 -- 172.21.15.112:0/1111117229 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f758c013070 con 0x7f75981030f0 2024-05-10T10:28:59.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:28:58 smithi112 ceph-mon[30100]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:28:59.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:28:59.174+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}) -- 0x7f7598199be0 con 0x7f758803e570 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: Deploying cephadm binary to smithi195 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:02.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:01 smithi112 ceph-mon[30100]: Updating smithi148:/etc/ceph/ceph.conf 2024-05-10T10:29:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:02 smithi112 ceph-mon[30100]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.326+0000 7f7595ffb700 1 -- 172.21.15.112:0/1111117229 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7598199be0 con 0x7f758803e570 2024-05-10T10:29:03.329 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi195' with addr '172.21.15.195' 2024-05-10T10:29:03.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f758803e570 msgr2=0x7f7588040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:03.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f758803e570 0x7f7588040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7584006fd0 tx=0x7f7584006e40 comp rx=0 tx=0).stop 2024-05-10T10:29:03.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 msgr2=0x7f759819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:03.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f759819e5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f758c005660 tx=0x7f758c005740 comp rx=0 tx=0).stop 2024-05-10T10:29:03.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 shutdown_connections 2024-05-10T10:29:03.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f758803e570 0x7f7588040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:03.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 --2- 172.21.15.112:0/1111117229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75981030f0 0x7f759819e5e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:03.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 >> 172.21.15.112:0/1111117229 conn(0x7f75980fe9c0 msgr2=0x7f7598104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:03.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 shutdown_connections 2024-05-10T10:29:03.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:03.329+0000 7f759f250700 1 -- 172.21.15.112:0/1111117229 wait complete. 2024-05-10T10:29:03.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:03 smithi112 ceph-mon[30100]: Updating smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:03.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:03 smithi112 ceph-mon[30100]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:29:03.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:03.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:03.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:03.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:03.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:03.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T10:29:04.207 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:29:04.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:04 smithi112 ceph-mon[30100]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:04.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:04 smithi112 ceph-mon[30100]: Updating smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T10:29:04.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:04 smithi112 ceph-mon[30100]: Added host smithi195 2024-05-10T10:29:04.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:04.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.718+0000 7f6efd39d700 1 -- 172.21.15.112:0/4288959470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 msgr2=0x7f6ef8103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.718+0000 7f6efd39d700 1 --2- 172.21.15.112:0/4288959470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8103300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009910 tx=0x7f6ee8009c20 comp rx=0 tx=0).stop 2024-05-10T10:29:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.718+0000 7f6efd39d700 1 -- 172.21.15.112:0/4288959470 shutdown_connections 2024-05-10T10:29:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.718+0000 7f6efd39d700 1 --2- 172.21.15.112:0/4288959470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8103300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.718+0000 7f6efd39d700 1 -- 172.21.15.112:0/4288959470 >> 172.21.15.112:0/4288959470 conn(0x7f6ef80fe7f0 msgr2=0x7f6ef8100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.718+0000 7f6efd39d700 1 -- 172.21.15.112:0/4288959470 shutdown_connections 2024-05-10T10:29:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.718+0000 7f6efd39d700 1 -- 172.21.15.112:0/4288959470 wait complete. 2024-05-10T10:29:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.719+0000 7f6efd39d700 1 Processor -- start 2024-05-10T10:29:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.719+0000 7f6efd39d700 1 -- start start 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6efd39d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6efd39d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef810d050 con 0x7f6ef8102f20 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54380/0 (socket says 172.21.15.112:54380) 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef7fff700 1 -- 172.21.15.112:0/4126500842 learned_addr learned my addr 172.21.15.112:0/4126500842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef7fff700 1 -- 172.21.15.112:0/4126500842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef8198c50 con 0x7f6ef8102f20 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef7fff700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8198710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006010 tx=0x7f6ee8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef57fa700 1 -- 172.21.15.112:0/4126500842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8026070 con 0x7f6ef8102f20 2024-05-10T10:29:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef8198f40 con 0x7f6ef8102f20 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef57fa700 1 -- 172.21.15.112:0/4126500842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ee800b6f0 con 0x7f6ef8102f20 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6ef57fa700 1 -- 172.21.15.112:0/4126500842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8003810 con 0x7f6ef8102f20 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.720+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef819cfe0 con 0x7f6ef8102f20 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.721+0000 7f6ef57fa700 1 -- 172.21.15.112:0/4126500842 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f6ee8029070 con 0x7f6ef8102f20 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.721+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8104400 con 0x7f6ef8102f20 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.722+0000 7f6ef57fa700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ee003e510 0x7f6ee00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.722+0000 7f6ef57fa700 1 -- 172.21.15.112:0/4126500842 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ee801e070 con 0x7f6ef8102f20 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.722+0000 7f6ef77fe700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ee003e510 0x7f6ee00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.722+0000 7f6ef77fe700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ee003e510 0x7f6ee00409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6eec006fd0 tx=0x7f6eec006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.727+0000 7f6ef57fa700 1 -- 172.21.15.112:0/4126500842 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6ee8013070 con 0x7f6ef8102f20 2024-05-10T10:29:05.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.938+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ef8064b00 con 0x7f6ee003e510 2024-05-10T10:29:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.939+0000 7f6ef57fa700 1 -- 172.21.15.112:0/4126500842 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f6ef8064b00 con 0x7f6ee003e510 2024-05-10T10:29:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:29:05.941 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}, {"addr": "172.21.15.195", "hostname": "smithi195", "labels": [], "status": ""}] 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.942+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ee003e510 msgr2=0x7f6ee00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.942+0000 7f6efd39d700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ee003e510 0x7f6ee00409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6eec006fd0 tx=0x7f6eec006e40 comp rx=0 tx=0).stop 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.942+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 msgr2=0x7f6ef8198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.942+0000 7f6efd39d700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8198710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006010 tx=0x7f6ee8005710 comp rx=0 tx=0).stop 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.942+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 shutdown_connections 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.942+0000 7f6efd39d700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ee003e510 0x7f6ee00409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.942+0000 7f6efd39d700 1 --2- 172.21.15.112:0/4126500842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8102f20 0x7f6ef8198710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:05.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.943+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 >> 172.21.15.112:0/4126500842 conn(0x7f6ef80fe7f0 msgr2=0x7f6ef81048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:05.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.943+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 shutdown_connections 2024-05-10T10:29:05.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:05.943+0000 7f6efd39d700 1 -- 172.21.15.112:0/4126500842 wait complete. 2024-05-10T10:29:06.550 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-10T10:29:06.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-10T10:29:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:29:07.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:06 smithi112 ceph-mon[30100]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:07.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:06 smithi112 ceph-mon[30100]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:29:07.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.198+0000 7f7a31115700 1 -- 172.21.15.112:0/3656550983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 msgr2=0x7f7a2c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.198+0000 7f7a31115700 1 --2- 172.21.15.112:0/3656550983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c1023f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009910 tx=0x7f7a1c009c20 comp rx=0 tx=0).stop 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.198+0000 7f7a31115700 1 -- 172.21.15.112:0/3656550983 shutdown_connections 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.198+0000 7f7a31115700 1 --2- 172.21.15.112:0/3656550983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c1023f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.198+0000 7f7a31115700 1 -- 172.21.15.112:0/3656550983 >> 172.21.15.112:0/3656550983 conn(0x7f7a2c0fd8e0 msgr2=0x7f7a2c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.198+0000 7f7a31115700 1 -- 172.21.15.112:0/3656550983 shutdown_connections 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.198+0000 7f7a31115700 1 -- 172.21.15.112:0/3656550983 wait complete. 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.199+0000 7f7a31115700 1 Processor -- start 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.199+0000 7f7a31115700 1 -- start start 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.199+0000 7f7a31115700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:08.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.199+0000 7f7a31115700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c10c140 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.199+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.199+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54394/0 (socket says 172.21.15.112:54394) 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.199+0000 7f7a2ad9d700 1 -- 172.21.15.112:0/784829039 learned_addr learned my addr 172.21.15.112:0/784829039 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.200+0000 7f7a2ad9d700 1 -- 172.21.15.112:0/784829039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c19eac0 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.200+0000 7f7a2ad9d700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c19e580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005660 tx=0x7f7a1c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.200+0000 7f7a28d99700 1 -- 172.21.15.112:0/784829039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c026070 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.200+0000 7f7a28d99700 1 -- 172.21.15.112:0/784829039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a1c00b6f0 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.200+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c19edb0 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.200+0000 7f7a28d99700 1 -- 172.21.15.112:0/784829039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c0037b0 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.200+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c198820 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.201+0000 7f7a28d99700 1 -- 172.21.15.112:0/784829039 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f7a1c029070 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.201+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c1034f0 con 0x7f7a2c102010 2024-05-10T10:29:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.201+0000 7f7a28d99700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7a1803e510 0x7f7a180409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:08.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.201+0000 7f7a28d99700 1 -- 172.21.15.112:0/784829039 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a1c01e070 con 0x7f7a2c102010 2024-05-10T10:29:08.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.201+0000 7f7a2259c700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7a1803e510 0x7f7a180409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:08.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.201+0000 7f7a2259c700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7a1803e510 0x7f7a180409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a14006fd0 tx=0x7f7a14006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.206+0000 7f7a28d99700 1 -- 172.21.15.112:0/784829039 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7a1c013070 con 0x7f7a2c102010 2024-05-10T10:29:08.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.414+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7a2c199b80 con 0x7f7a2c102010 2024-05-10T10:29:08.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.829+0000 7f7a28d99700 1 -- 172.21.15.112:0/784829039 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7a1c029360 con 0x7f7a2c102010 2024-05-10T10:29:08.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7a1803e510 msgr2=0x7f7a180409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7a1803e510 0x7f7a180409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a14006fd0 tx=0x7f7a14006e40 comp rx=0 tx=0).stop 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 msgr2=0x7f7a2c19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c19e580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005660 tx=0x7f7a1c005740 comp rx=0 tx=0).stop 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 shutdown_connections 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7a1803e510 0x7f7a180409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 --2- 172.21.15.112:0/784829039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a2c102010 0x7f7a2c19e580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.831+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 >> 172.21.15.112:0/784829039 conn(0x7f7a2c0fd8e0 msgr2=0x7f7a2c06b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.832+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 shutdown_connections 2024-05-10T10:29:08.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:08.832+0000 7f7a31115700 1 -- 172.21.15.112:0/784829039 wait complete. 2024-05-10T10:29:08.834 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-05-10T10:29:09.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:08 smithi112 ceph-mon[30100]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:09.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:08 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/784829039' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T10:29:09.418 INFO:tasks.cephadm:Adding mon.a on smithi112 2024-05-10T10:29:09.418 INFO:tasks.cephadm:Adding mon.b on smithi148 2024-05-10T10:29:09.419 INFO:tasks.cephadm:Adding mon.c on smithi195 2024-05-10T10:29:09.419 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi112:172.21.15.112=a;smithi148:172.21.15.148=b;smithi195:172.21.15.195=c' 2024-05-10T10:29:09.655 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-10T10:29:09.706 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-10T10:29:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/784829039' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T10:29:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:29:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:11.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.200+0000 7f4d6f267700 1 -- 172.21.15.195:0/3496255667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 msgr2=0x7f4d681082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:11.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.200+0000 7f4d6f267700 1 --2- 172.21.15.195:0/3496255667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d681082a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009910 tx=0x7f4d64009c20 comp rx=0 tx=0).stop 2024-05-10T10:29:11.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.200+0000 7f4d6f267700 1 -- 172.21.15.195:0/3496255667 shutdown_connections 2024-05-10T10:29:11.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.200+0000 7f4d6f267700 1 --2- 172.21.15.195:0/3496255667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d681082a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.200+0000 7f4d6f267700 1 -- 172.21.15.195:0/3496255667 >> 172.21.15.195:0/3496255667 conn(0x7f4d680fd020 msgr2=0x7f4d680ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.200+0000 7f4d6f267700 1 -- 172.21.15.195:0/3496255667 shutdown_connections 2024-05-10T10:29:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.200+0000 7f4d6f267700 1 -- 172.21.15.195:0/3496255667 wait complete. 2024-05-10T10:29:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.201+0000 7f4d6f267700 1 Processor -- start 2024-05-10T10:29:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.201+0000 7f4d6f267700 1 -- start start 2024-05-10T10:29:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.201+0000 7f4d6f267700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d6819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.202+0000 7f4d6f267700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6810d380 con 0x7f4d68107ec0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.202+0000 7f4d6d003700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d6819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.202+0000 7f4d6d003700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d6819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.195:44360/0 (socket says 172.21.15.195:44360) 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.202+0000 7f4d6d003700 1 -- 172.21.15.195:0/3543478266 learned_addr learned my addr 172.21.15.195:0/3543478266 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.202+0000 7f4d6d003700 1 -- 172.21.15.195:0/3543478266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6819eb30 con 0x7f4d68107ec0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.202+0000 7f4d6d003700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d6819e5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005660 tx=0x7f4d64005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.203+0000 7f4d5e7fc700 1 -- 172.21.15.195:0/3543478266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d64026070 con 0x7f4d68107ec0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.203+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6819edc0 con 0x7f4d68107ec0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.203+0000 7f4d5e7fc700 1 -- 172.21.15.195:0/3543478266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d6400b750 con 0x7f4d68107ec0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.203+0000 7f4d5e7fc700 1 -- 172.21.15.195:0/3543478266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d640037b0 con 0x7f4d68107ec0 2024-05-10T10:29:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.203+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681986c0 con 0x7f4d68107ec0 2024-05-10T10:29:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.204+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d68104600 con 0x7f4d68107ec0 2024-05-10T10:29:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.204+0000 7f4d5e7fc700 1 -- 172.21.15.195:0/3543478266 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f4d64029070 con 0x7f4d68107ec0 2024-05-10T10:29:11.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.204+0000 7f4d5e7fc700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d5403e510 0x7f4d540409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:11.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.204+0000 7f4d5e7fc700 1 -- 172.21.15.195:0/3543478266 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4d6401e070 con 0x7f4d68107ec0 2024-05-10T10:29:11.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.204+0000 7f4d6c802700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d5403e510 0x7f4d540409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:11.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.205+0000 7f4d6c802700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d5403e510 0x7f4d540409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006fd0 tx=0x7f4d58006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:11.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.209+0000 7f4d5e7fc700 1 -- 172.21.15.195:0/3543478266 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4d64013070 con 0x7f4d68107ec0 2024-05-10T10:29:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:10 smithi112 ceph-mon[30100]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:10 smithi112 ceph-mon[30100]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-10T10:29:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:10 smithi112 ceph-mon[30100]: Updating smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:11.426 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.424+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi148:172.21.15.148=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}) -- 0x7f4d68199bf0 con 0x7f4d5403e510 2024-05-10T10:29:11.432 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.431+0000 7f4d5e7fc700 1 -- 172.21.15.195:0/3543478266 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4d68199bf0 con 0x7f4d5403e510 2024-05-10T10:29:11.432 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mon update... 2024-05-10T10:29:11.434 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d5403e510 msgr2=0x7f4d540409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:11.434 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d5403e510 0x7f4d540409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006fd0 tx=0x7f4d58006e40 comp rx=0 tx=0).stop 2024-05-10T10:29:11.434 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 msgr2=0x7f4d6819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:11.434 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d6819e5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005660 tx=0x7f4d64005740 comp rx=0 tx=0).stop 2024-05-10T10:29:11.434 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 shutdown_connections 2024-05-10T10:29:11.435 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d5403e510 0x7f4d540409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:11.435 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 --2- 172.21.15.195:0/3543478266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68107ec0 0x7f4d6819e5f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:11.435 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 >> 172.21.15.195:0/3543478266 conn(0x7f4d680fd020 msgr2=0x7f4d6810ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:11.435 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 shutdown_connections 2024-05-10T10:29:11.435 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:11.433+0000 7f4d6f267700 1 -- 172.21.15.195:0/3543478266 wait complete. 2024-05-10T10:29:12.071 DEBUG:teuthology.orchestra.run.smithi148:mon.b> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.b.service 2024-05-10T10:29:12.074 DEBUG:teuthology.orchestra.run.smithi195:mon.c> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.c.service 2024-05-10T10:29:12.076 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T10:29:12.077 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: Updating smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:12.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:12 smithi112 ceph-mon[30100]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:12 smithi112 ceph-mon[30100]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi148:172.21.15.148=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:12 smithi112 ceph-mon[30100]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi148:172.21.15.148=b;smithi195:172.21.15.195=c;count:3 2024-05-10T10:29:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:12 smithi112 ceph-mon[30100]: Deploying daemon mon.c on smithi195 2024-05-10T10:29:13.884 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:15.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:14 smithi112 ceph-mon[30100]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.162+0000 7f8939548700 1 -- 172.21.15.195:0/2932055898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 msgr2=0x7f89341023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.162+0000 7f8939548700 1 --2- 172.21.15.195:0/2932055898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f89341023f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8924009910 tx=0x7f8924009c20 comp rx=0 tx=0).stop 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.162+0000 7f8939548700 1 -- 172.21.15.195:0/2932055898 shutdown_connections 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.162+0000 7f8939548700 1 --2- 172.21.15.195:0/2932055898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f89341023f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.162+0000 7f8939548700 1 -- 172.21.15.195:0/2932055898 >> 172.21.15.195:0/2932055898 conn(0x7f89340fd8e0 msgr2=0x7f89340ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.163+0000 7f8939548700 1 -- 172.21.15.195:0/2932055898 shutdown_connections 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.163+0000 7f8939548700 1 -- 172.21.15.195:0/2932055898 wait complete. 2024-05-10T10:29:16.166 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.163+0000 7f8939548700 1 Processor -- start 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.163+0000 7f8939548700 1 -- start start 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.163+0000 7f8939548700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f8934103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.163+0000 7f8939548700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f893410c140 con 0x7f8934102010 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8932ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f8934103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8932ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f8934103ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.195:44382/0 (socket says 172.21.15.195:44382) 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8932ffd700 1 -- 172.21.15.195:0/457199761 learned_addr learned my addr 172.21.15.195:0/457199761 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8932ffd700 1 -- 172.21.15.195:0/457199761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89341059a0 con 0x7f8934102010 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8932ffd700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f8934103ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8924005660 tx=0x7f8924005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8930ff9700 1 -- 172.21.15.195:0/457199761 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8924026070 con 0x7f8934102010 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8934104050 con 0x7f8934102010 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8930ff9700 1 -- 172.21.15.195:0/457199761 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f892400b750 con 0x7f8934102010 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8930ff9700 1 -- 172.21.15.195:0/457199761 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89240037b0 con 0x7f8934102010 2024-05-10T10:29:16.167 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.164+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8934104510 con 0x7f8934102010 2024-05-10T10:29:16.168 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.165+0000 7f8930ff9700 1 -- 172.21.15.195:0/457199761 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f8924029070 con 0x7f8934102010 2024-05-10T10:29:16.169 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.165+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89341034f0 con 0x7f8934102010 2024-05-10T10:29:16.169 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.166+0000 7f8930ff9700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f892003e510 0x7f89200409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:16.169 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.166+0000 7f8930ff9700 1 -- 172.21.15.195:0/457199761 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f892401e070 con 0x7f8934102010 2024-05-10T10:29:16.169 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.166+0000 7f892a7fc700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f892003e510 0x7f89200409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:16.169 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.166+0000 7f892a7fc700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f892003e510 0x7f89200409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f891c006fd0 tx=0x7f891c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:16.172 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.171+0000 7f8930ff9700 1 -- 172.21.15.195:0/457199761 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8924013070 con 0x7f8934102010 2024-05-10T10:29:16.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 podman[33732]: 2024-05-10 10:29:16.227028378 +0000 UTC m=+0.306684942 container create 249aec4d15a61c0e6b7634e635f6a709d749c30c38fb85493dd0607122380cfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:29:16.467 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.465+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8934104fe0 con 0x7f8934102010 2024-05-10T10:29:16.468 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.466+0000 7f8930ff9700 1 -- 172.21.15.195:0/457199761 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f8924030020 con 0x7f8934102010 2024-05-10T10:29:16.468 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:29:16.468 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":1,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","modified":"2024-05-10T10:27:31.745764Z","created":"2024-05-10T10:27:31.745764Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-10T10:29:16.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f892003e510 msgr2=0x7f89200409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:16.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f892003e510 0x7f89200409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f891c006fd0 tx=0x7f891c006e40 comp rx=0 tx=0).stop 2024-05-10T10:29:16.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 msgr2=0x7f8934103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:16.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f8934103ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8924005660 tx=0x7f8924005740 comp rx=0 tx=0).stop 2024-05-10T10:29:16.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 shutdown_connections 2024-05-10T10:29:16.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f892003e510 0x7f89200409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:16.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 --2- 172.21.15.195:0/457199761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8934102010 0x7f8934103ab0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:16.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 >> 172.21.15.195:0/457199761 conn(0x7f89340fd8e0 msgr2=0x7f893406b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:16.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 shutdown_connections 2024-05-10T10:29:16.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:16.469+0000 7f8939548700 1 -- 172.21.15.195:0/457199761 wait complete. 2024-05-10T10:29:16.471 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 1 2024-05-10T10:29:16.798 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 podman[33732]: 2024-05-10 10:29:16.43887349 +0000 UTC m=+0.518530060 container init 249aec4d15a61c0e6b7634e635f6a709d749c30c38fb85493dd0607122380cfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS) 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 podman[33732]: 2024-05-10 10:29:16.442338175 +0000 UTC m=+0.521994736 container start 249aec4d15a61c0e6b7634e635f6a709d749c30c38fb85493dd0607122380cfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev), process ceph-mon, pid 2 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pidfile_write: ignore empty --pid-file 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: load: jerasure load: lrc 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: RocksDB version: 7.9.2 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Git sha 0 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Compile date 2024-05-10 01:38:55 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: DB SUMMARY 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: DB Session ID: R7UMF0E4QDUOL7J6KYKI 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: CURRENT file: CURRENT 2024-05-10T10:29:16.799 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.error_if_exists: 0 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.create_if_missing: 0 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.paranoid_checks: 1 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.env: 0x56211ceb2cc0 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T10:29:16.800 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.info_log: 0x56211dc7dda0 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.statistics: (nil) 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.use_fsync: 0 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_log_file_size: 0 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.allow_fallocate: 1 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T10:29:16.801 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.use_direct_reads: 0 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.db_log_dir: 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.wal_dir: 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T10:29:16.802 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.write_buffer_manager: 0x56211dc4f0e0 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T10:29:16.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.unordered_write: 0 2024-05-10T10:29:16.804 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T10:29:16.804 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T10:29:16.804 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T10:29:16.804 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T10:29:16.804 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.row_cache: None 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.wal_filter: None 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.two_write_queues: 0 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.wal_compression: 0 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.atomic_flush: 0 2024-05-10T10:29:16.806 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.log_readahead_size: 0 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_background_jobs: 2 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_background_compactions: -1 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_subcompactions: 1 2024-05-10T10:29:16.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_open_files: -1 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_background_flushes: -1 2024-05-10T10:29:16.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Compression algorithms supported: 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kZSTD supported: 0 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kXpressCompression supported: 0 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kZlibCompression supported: 1 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kSnappyCompression supported: 1 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kLZ4Compression supported: 1 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: kBZip2Compression supported: 0 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T10:29:16.809 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.merge_operator: 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_filter: None 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56211dc7df60) 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: cache_index_and_filter_blocks: 1 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: pin_top_level_index_and_filter: 1 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_type: 0 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: data_block_index_type: 0 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_shortening: 1 2024-05-10T10:29:16.810 INFO:journalctl@ceph.mon.c.smithi195.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: checksum: 4 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: no_block_cache: 0 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache: 0x56211dc811f0 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_name: BinnedLRUCache 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_options: 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: capacity : 536870912 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: num_shard_bits : 4 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: strict_capacity_limit : 0 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: high_pri_pool_ratio: 0.000 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_compressed: (nil) 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: persistent_cache: (nil) 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_size: 4096 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_size_deviation: 10 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_restart_interval: 16 2024-05-10T10:29:16.811 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_block_restart_interval: 1 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: metadata_block_size: 4096 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: partition_filters: 0 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: use_delta_encoding: 1 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: filter_policy: bloomfilter 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: whole_key_filtering: 1 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: verify_compression: 0 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: read_amp_bytes_per_bit: 0 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: format_version: 5 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: enable_index_compression: 1 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_align: 0 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: max_auto_readahead_size: 262144 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: prepopulate_block_cache: 0 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: initial_auto_readahead_size: 8192 2024-05-10T10:29:16.812 INFO:journalctl@ceph.mon.c.smithi195.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression: NoCompression 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.num_levels: 7 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T10:29:16.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T10:29:16.814 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T10:29:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T10:29:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.inplace_update_support: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.bloom_locality: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.max_successive_merges: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T10:29:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.ttl: 2592000 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.enable_blob_files: false 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.min_blob_size: 0 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T10:29:16.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3f935e7a-9e10-4f2d-a745-8fe7dcd8d69f 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715336956473793, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715336956474225, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715336956, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3f935e7a-9e10-4f2d-a745-8fe7dcd8d69f", "db_session_id": "R7UMF0E4QDUOL7J6KYKI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715336956474307, "job": 1, "event": "recovery_finished"} 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56211dd9e000 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: DB pointer 0x56211dd8a000 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T10:29:16.819 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** DB Stats ** 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** Compaction Stats [default] ** 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T10:29:16.820 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** Compaction Stats [default] ** 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T10:29:16.821 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout: Block cache BinnedLRUCache@0x56211dc811f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: using public_addr v2:172.21.15.195:0/0 -> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: starting mon.c rank -1 at public addrs [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] at bind addrs [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(???) e0 preinit fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).mds e1 new map 2024-05-10T10:29:16.822 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout: e1 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout: legacy client fscid: -1 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout: No filesystems configured 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T10:29:16.823 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Deploying cephadm binary to smithi195 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi148:/etc/ceph/ceph.conf 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.824 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Added host smithi195 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.825 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/784829039' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/784829039' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.826 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Updating smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T10:29:16.827 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi148:172.21.15.148=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi148:172.21.15.148=b;smithi195:172.21.15.195=c;count:3 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: Deploying daemon mon.c on smithi195 2024-05-10T10:29:16.828 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:16.829 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 ceph-mon[33773]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T10:29:16.829 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 bash[33732]: 249aec4d15a61c0e6b7634e635f6a709d749c30c38fb85493dd0607122380cfe 2024-05-10T10:29:16.829 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:16 smithi195 systemd[1]: Started Ceph mon.c for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:29:18.171 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T10:29:18.171 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T10:29:18.418 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:29:21.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:29:21.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: mon.a calling monitor election 2024-05-10T10:29:21.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: mon.c calling monitor election 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: monmap epoch 2 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: last_changed 2024-05-10T10:29:16.499479+0000 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: min_mon_release 19 (squid) 2024-05-10T10:29:21.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: election_strategy: 1 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: fsmap 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: mgrmap e13: x(active, since 48s) 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: overall HEALTH_OK 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:21.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:21 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: mon.a calling monitor election 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: mon.c calling monitor election 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: monmap epoch 2 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: last_changed 2024-05-10T10:29:16.499479+0000 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: min_mon_release 19 (squid) 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: election_strategy: 1 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: fsmap 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: mgrmap e13: x(active, since 48s) 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: overall HEALTH_OK 2024-05-10T10:29:21.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:21.832 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:21.832 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:21.832 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:21.832 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:21 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:22.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:22 smithi112 ceph-mon[30100]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:22.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:22 smithi112 ceph-mon[30100]: Deploying daemon mon.b on smithi148 2024-05-10T10:29:22.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:22.852 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:22 smithi195 ceph-mon[33773]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:22.852 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:22 smithi195 ceph-mon[33773]: Deploying daemon mon.b on smithi148 2024-05-10T10:29:22.852 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:22.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.850+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2273703537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 msgr2=0x7ffbd4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:22.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.850+0000 7ffbeafc1700 1 --2- 172.21.15.195:0/2273703537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 0x7ffbd4005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4101dc0 tx=0x7ffbd800feb0 comp rx=0 tx=0).stop 2024-05-10T10:29:22.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.850+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2273703537 shutdown_connections 2024-05-10T10:29:22.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.850+0000 7ffbeafc1700 1 --2- 172.21.15.195:0/2273703537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 0x7ffbd4005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:22.854 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.850+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2273703537 >> 172.21.15.195:0/2273703537 conn(0x7ffbe40fd980 msgr2=0x7ffbe40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.850+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2273703537 shutdown_connections 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.850+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2273703537 wait complete. 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.851+0000 7ffbeafc1700 1 Processor -- start 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.851+0000 7ffbeafc1700 1 -- start start 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbeafc1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 0x7ffbe419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbeafc1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 0x7ffbe4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbeafc1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4109c90 con 0x7ffbd40035c0 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbeafc1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4109b10 con 0x7ffbe419ec10 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 0x7ffbe4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe8d5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 0x7ffbe419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 0x7ffbe4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:55966/0 (socket says 172.21.15.195:55966) 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe3fff700 1 -- 172.21.15.195:0/2302955335 learned_addr learned my addr 172.21.15.195:0/2302955335 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe3fff700 1 -- 172.21.15.195:0/2302955335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 msgr2=0x7ffbe4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe3fff700 1 -- 172.21.15.195:0/2302955335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 msgr2=0x7ffbe4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T10:29:22.855 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe3fff700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 0x7ffbe4198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe3fff700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 0x7ffbe4198850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe8d5d700 1 -- 172.21.15.195:0/2302955335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 msgr2=0x7ffbe4198850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe8d5d700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 0x7ffbe4198850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.852+0000 7ffbe8d5d700 1 -- 172.21.15.195:0/2302955335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe419f250 con 0x7ffbd40035c0 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.853+0000 7ffbe8d5d700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 0x7ffbe419e6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4101dc0 tx=0x7ffbd8005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.853+0000 7ffbe1ffb700 1 -- 172.21.15.195:0/2302955335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffbd8013960 con 0x7ffbd40035c0 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.853+0000 7ffbe1ffb700 1 -- 172.21.15.195:0/2302955335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbd8004030 con 0x7ffbd40035c0 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.853+0000 7ffbe1ffb700 1 -- 172.21.15.195:0/2302955335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffbd8003b70 con 0x7ffbd40035c0 2024-05-10T10:29:22.856 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.853+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe4198df0 con 0x7ffbd40035c0 2024-05-10T10:29:22.857 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.853+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe4199350 con 0x7ffbd40035c0 2024-05-10T10:29:22.857 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.854+0000 7ffbe1ffb700 1 -- 172.21.15.195:0/2302955335 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7ffbd8023070 con 0x7ffbd40035c0 2024-05-10T10:29:22.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.854+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe41035d0 con 0x7ffbd40035c0 2024-05-10T10:29:22.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.855+0000 7ffbe1ffb700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffbc803e260 0x7ffbc8040720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:22.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.855+0000 7ffbe1ffb700 1 -- 172.21.15.195:0/2302955335 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffbd8020050 con 0x7ffbd40035c0 2024-05-10T10:29:22.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.855+0000 7ffbe3fff700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffbc803e260 0x7ffbc8040720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:22.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.855+0000 7ffbe3fff700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffbc803e260 0x7ffbc8040720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009180 tx=0x7ffbd0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:22.865 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:22.864+0000 7ffbe1ffb700 1 -- 172.21.15.195:0/2302955335 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffbd8028020 con 0x7ffbd40035c0 2024-05-10T10:29:23.156 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.155+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ffbe4199d80 con 0x7ffbd40035c0 2024-05-10T10:29:23.157 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.155+0000 7ffbe1ffb700 1 -- 172.21.15.195:0/2302955335 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7ffbd8023360 con 0x7ffbd40035c0 2024-05-10T10:29:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:29:23.157 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":2,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","modified":"2024-05-10T10:29:16.499479Z","created":"2024-05-10T10:27:31.745764Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T10:29:23.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.157+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffbc803e260 msgr2=0x7ffbc8040720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:23.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.157+0000 7ffbeafc1700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffbc803e260 0x7ffbc8040720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009180 tx=0x7ffbd0008040 comp rx=0 tx=0).stop 2024-05-10T10:29:23.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.157+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 msgr2=0x7ffbe419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:23.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.157+0000 7ffbeafc1700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 0x7ffbe419e6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4101dc0 tx=0x7ffbd8005c70 comp rx=0 tx=0).stop 2024-05-10T10:29:23.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.158+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 shutdown_connections 2024-05-10T10:29:23.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.158+0000 7ffbeafc1700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd40035c0 0x7ffbe419e6d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:23.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.158+0000 7ffbeafc1700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffbc803e260 0x7ffbc8040720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:23.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.158+0000 7ffbeafc1700 1 --2- 172.21.15.195:0/2302955335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbe419ec10 0x7ffbe4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:23.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.158+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 >> 172.21.15.195:0/2302955335 conn(0x7ffbe40fd980 msgr2=0x7ffbe406b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:23.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.158+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 shutdown_connections 2024-05-10T10:29:23.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:23.158+0000 7ffbeafc1700 1 -- 172.21.15.195:0/2302955335 wait complete. 2024-05-10T10:29:23.160 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 2 2024-05-10T10:29:23.705 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:23 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/2302955335' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T10:29:23.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:23 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/2302955335' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T10:29:24.734 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T10:29:24.734 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T10:29:24.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:24 smithi112 ceph-mon[30100]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:24.971 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:29:25.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:24 smithi195 ceph-mon[33773]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:25.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:25 smithi148 systemd[1]: Started Ceph mon.b for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:29:30.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.711+0000 7f62e4b67700 1 -- 172.21.15.195:0/214097818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80035c0 msgr2=0x7f62c8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:30.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.711+0000 7f62e4b67700 1 --2- 172.21.15.195:0/214097818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80035c0 0x7f62c8005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f62e0101dc0 tx=0x7f62d000fe60 comp rx=0 tx=0).stop 2024-05-10T10:29:30.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.711+0000 7f62e4b67700 1 -- 172.21.15.195:0/214097818 shutdown_connections 2024-05-10T10:29:30.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.711+0000 7f62e4b67700 1 --2- 172.21.15.195:0/214097818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80035c0 0x7f62c8005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:30.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.711+0000 7f62e4b67700 1 -- 172.21.15.195:0/214097818 >> 172.21.15.195:0/214097818 conn(0x7f62e00fd980 msgr2=0x7f62e00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:30.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.711+0000 7f62e4b67700 1 -- 172.21.15.195:0/214097818 shutdown_connections 2024-05-10T10:29:30.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.711+0000 7f62e4b67700 1 -- 172.21.15.195:0/214097818 wait complete. 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.712+0000 7f62e4b67700 1 Processor -- start 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.712+0000 7f62e4b67700 1 -- start start 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62e4b67700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 0x7f62e0199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62e4b67700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 0x7f62e019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62e4b67700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e01a29b0 0x7f62e01a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62e4b67700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0109c30 con 0x7f62e01a29b0 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62e4b67700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0109ab0 con 0x7f62e0199e80 2024-05-10T10:29:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62e4b67700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0109db0 con 0x7f62c80035c0 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62de59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 0x7f62e0199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62de59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 0x7f62e0199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:44186/0 (socket says 172.21.15.195:44186) 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62de59c700 1 -- 172.21.15.195:0/3091270598 learned_addr learned my addr 172.21.15.195:0/3091270598 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ddd9b700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 0x7f62e019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ded9d700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e01a29b0 0x7f62e01a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62de59c700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 msgr2=0x7f62e0199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62de59c700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 msgr2=0x7f62e0199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62de59c700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 0x7f62e0199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ddd9b700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 msgr2=0x7f62e019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ddd9b700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 msgr2=0x7f62e019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T10:29:30.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ddd9b700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 0x7f62e019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ddd9b700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 0x7f62e019a300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62de59c700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 0x7f62e0199940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ded9d700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 msgr2=0x7f62e0199940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ded9d700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 0x7f62e0199940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ded9d700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 msgr2=0x7f62e019a300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ded9d700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 0x7f62e019a300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.713+0000 7f62ded9d700 1 -- 172.21.15.195:0/3091270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e01a5570 con 0x7f62e01a29b0 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.714+0000 7f62ded9d700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e01a29b0 0x7f62e01a4da0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f62d800bd50 tx=0x7f62d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.714+0000 7f62cf7fe700 1 -- 172.21.15.195:0/3091270598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8019070 con 0x7f62e01a29b0 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.714+0000 7f62cf7fe700 1 -- 172.21.15.195:0/3091270598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62d800cd10 con 0x7f62e01a29b0 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.714+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e01a5860 con 0x7f62e01a29b0 2024-05-10T10:29:30.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.714+0000 7f62cf7fe700 1 -- 172.21.15.195:0/3091270598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d80125e0 con 0x7f62e01a29b0 2024-05-10T10:29:30.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.714+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e01a5da0 con 0x7f62e01a29b0 2024-05-10T10:29:30.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.715+0000 7f62cf7fe700 1 -- 172.21.15.195:0/3091270598 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f62d801c070 con 0x7f62e01a29b0 2024-05-10T10:29:30.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.715+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e0064b00 con 0x7f62e01a29b0 2024-05-10T10:29:30.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.715+0000 7f62cf7fe700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f62c003e9a0 0x7f62c0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:30.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.715+0000 7f62cf7fe700 1 -- 172.21.15.195:0/3091270598 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f62d8052160 con 0x7f62e01a29b0 2024-05-10T10:29:30.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.716+0000 7f62de59c700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f62c003e9a0 0x7f62c0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:30.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.716+0000 7f62de59c700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f62c003e9a0 0x7f62c0040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62d0020040 tx=0x7f62d0001e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:30.721 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:30.721+0000 7f62cf7fe700 1 -- 172.21.15.195:0/3091270598 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f62d8022170 con 0x7f62e01a29b0 2024-05-10T10:29:31.020 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.019+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f62e01a6140 con 0x7f62e01a29b0 2024-05-10T10:29:31.020 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: mon.c calling monitor election 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: mon.a calling monitor election 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T10:29:31.021 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: monmap epoch 3 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: last_changed 2024-05-10T10:29:25.597255+0000 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: min_mon_release 19 (squid) 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: election_strategy: 1 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: fsmap 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: mgrmap e13: x(active, since 57s) 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: overall HEALTH_OK 2024-05-10T10:29:31.022 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.023 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.023 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.023 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.023 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.020+0000 7f62cf7fe700 1 -- 172.21.15.195:0/3091270598 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f62d80149f0 con 0x7f62e01a29b0 2024-05-10T10:29:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-10T10:29:31.023 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":3,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","modified":"2024-05-10T10:29:25.597255Z","created":"2024-05-10T10:27:31.745764Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T10:29:31.024 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f62c003e9a0 msgr2=0x7f62c0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:31.024 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f62c003e9a0 0x7f62c0040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62d0020040 tx=0x7f62d0001e70 comp rx=0 tx=0).stop 2024-05-10T10:29:31.024 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e01a29b0 msgr2=0x7f62e01a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:31.024 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e01a29b0 0x7f62e01a4da0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f62d800bd50 tx=0x7f62d800be30 comp rx=0 tx=0).stop 2024-05-10T10:29:31.024 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 shutdown_connections 2024-05-10T10:29:31.024 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e01a29b0 0x7f62e01a4da0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:31.025 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f62c003e9a0 0x7f62c0040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:31.025 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62c80035c0 0x7f62e0199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:31.025 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 --2- 172.21.15.195:0/3091270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0199e80 0x7f62e019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:31.025 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 >> 172.21.15.195:0/3091270598 conn(0x7f62e00fd980 msgr2=0x7f62e0105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:31.025 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 shutdown_connections 2024-05-10T10:29:31.025 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:31.023+0000 7f62e4b67700 1 -- 172.21.15.195:0/3091270598 wait complete. 2024-05-10T10:29:31.025 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 3 2024-05-10T10:29:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: mon.c calling monitor election 2024-05-10T10:29:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:29:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: mon.a calling monitor election 2024-05-10T10:29:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: monmap epoch 3 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: last_changed 2024-05-10T10:29:25.597255+0000 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:31.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: min_mon_release 19 (squid) 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: election_strategy: 1 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: fsmap 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: mgrmap e13: x(active, since 57s) 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: overall HEALTH_OK 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:31.512 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-10T10:29:31.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-10T10:29:31.762 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:29:31.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:31.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:31 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/3091270598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T10:29:31.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:31.932 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:31.933 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:31 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/3091270598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T10:29:31.933 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:33.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: mon.c calling monitor election 2024-05-10T10:29:33.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:29:33.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: mon.a calling monitor election 2024-05-10T10:29:33.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: monmap epoch 3 2024-05-10T10:29:33.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: last_changed 2024-05-10T10:29:25.597255+0000 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: min_mon_release 19 (squid) 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: election_strategy: 1 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-05-10T10:29:33.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: fsmap 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: mgrmap e13: x(active, since 57s) 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: overall HEALTH_OK 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:33.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/3091270598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T10:29:33.049 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.410+0000 7f3731bd3700 1 -- 172.21.15.112:0/1198439543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c101fe0 msgr2=0x7f372c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.410+0000 7f3731bd3700 1 --2- 172.21.15.112:0/1198439543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c101fe0 0x7f372c1023c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f371c00b0a0 tx=0x7f371c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.410+0000 7f3731bd3700 1 -- 172.21.15.112:0/1198439543 shutdown_connections 2024-05-10T10:29:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.410+0000 7f3731bd3700 1 --2- 172.21.15.112:0/1198439543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c101fe0 0x7f372c1023c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.410+0000 7f3731bd3700 1 -- 172.21.15.112:0/1198439543 >> 172.21.15.112:0/1198439543 conn(0x7f372c078a70 msgr2=0x7f372c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.411+0000 7f3731bd3700 1 -- 172.21.15.112:0/1198439543 shutdown_connections 2024-05-10T10:29:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.411+0000 7f3731bd3700 1 -- 172.21.15.112:0/1198439543 wait complete. 2024-05-10T10:29:33.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 Processor -- start 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 -- start start 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f372c101fe0 0x7f372c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f372c19db80 0x7f372c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c197c50 0x7f372c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f372c10c0d0 con 0x7f372c197c50 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f372c10bf50 con 0x7f372c101fe0 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f3731bd3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f372c10c250 con 0x7f372c19db80 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f372bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c197c50 0x7f372c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f372bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c197c50 0x7f372c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42632/0 (socket says 172.21.15.112:42632) 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f372bfff700 1 -- 172.21.15.112:0/3620020803 learned_addr learned my addr 172.21.15.112:0/3620020803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f372affd700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f372c19db80 0x7f372c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.412+0000 7f372b7fe700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f372c101fe0 0x7f372c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f372bfff700 1 -- 172.21.15.112:0/3620020803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f372c19db80 msgr2=0x7f372c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f372bfff700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f372c19db80 0x7f372c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f372bfff700 1 -- 172.21.15.112:0/3620020803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f372c101fe0 msgr2=0x7f372c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f372bfff700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f372c101fe0 0x7f372c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f372bfff700 1 -- 172.21.15.112:0/3620020803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1a43f0 con 0x7f372c197c50 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f372bfff700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c197c50 0x7f372c198100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f372400b550 tx=0x7f372400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f3728ff9700 1 -- 172.21.15.112:0/3620020803 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724015020 con 0x7f372c197c50 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f3728ff9700 1 -- 172.21.15.112:0/3620020803 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3724011cc0 con 0x7f372c197c50 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1a4680 con 0x7f372c197c50 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f3728ff9700 1 -- 172.21.15.112:0/3620020803 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724010c90 con 0x7f372c197c50 2024-05-10T10:29:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.413+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1a4bc0 con 0x7f372c197c50 2024-05-10T10:29:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.414+0000 7f3728ff9700 1 -- 172.21.15.112:0/3620020803 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3724018020 con 0x7f372c197c50 2024-05-10T10:29:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.414+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c1034c0 con 0x7f372c197c50 2024-05-10T10:29:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.415+0000 7f3728ff9700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f371803e9a0 0x7f3718040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.415+0000 7f3728ff9700 1 -- 172.21.15.112:0/3620020803 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3724051ff0 con 0x7f372c197c50 2024-05-10T10:29:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.415+0000 7f372b7fe700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f371803e9a0 0x7f3718040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:33.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.415+0000 7f372b7fe700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f371803e9a0 0x7f3718040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f371c015040 tx=0x7f371c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.420+0000 7f3728ff9700 1 -- 172.21.15.112:0/3620020803 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f372401f020 con 0x7f372c197c50 2024-05-10T10:29:33.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: mon.b calling monitor election 2024-05-10T10:29:33.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Updating smithi148:/etc/ceph/ceph.conf 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Updating smithi112:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Updating smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Updating smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: mon.b calling monitor election 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: mon.a calling monitor election 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: mon.c calling monitor election 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: monmap epoch 3 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: last_changed 2024-05-10T10:29:25.597255+0000 2024-05-10T10:29:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: min_mon_release 19 (squid) 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: election_strategy: 1 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: fsmap 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: mgrmap e13: x(active, since 59s) 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: overall HEALTH_OK 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:33 smithi112 ceph-mon[30100]: Reconfiguring daemon mon.a on smithi112 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: mon.b calling monitor election 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Updating smithi148:/etc/ceph/ceph.conf 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Updating smithi112:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Updating smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Updating smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: mon.b calling monitor election 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: mon.a calling monitor election 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: mon.c calling monitor election 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T10:29:33.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: monmap epoch 3 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: last_changed 2024-05-10T10:29:25.597255+0000 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: min_mon_release 19 (squid) 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: election_strategy: 1 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-05-10T10:29:33.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: fsmap 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: mgrmap e13: x(active, since 59s) 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: overall HEALTH_OK 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T10:29:33.548 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:33.548 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:33.548 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:33.548 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:33 smithi148 ceph-mon[33568]: Reconfiguring daemon mon.a on smithi112 2024-05-10T10:29:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: mon.b calling monitor election 2024-05-10T10:29:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Updating smithi148:/etc/ceph/ceph.conf 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Updating smithi112:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Updating smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Updating smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: mon.b calling monitor election 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: mon.a calling monitor election 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: mon.c calling monitor election 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: monmap epoch 3 2024-05-10T10:29:33.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: last_changed 2024-05-10T10:29:25.597255+0000 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: created 2024-05-10T10:27:31.745764+0000 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: min_mon_release 19 (squid) 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: election_strategy: 1 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: fsmap 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: mgrmap e13: x(active, since 59s) 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: overall HEALTH_OK 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:33.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:33 smithi195 ceph-mon[33773]: Reconfiguring daemon mon.a on smithi112 2024-05-10T10:29:33.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.627+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f372c1a4e80 con 0x7f372c197c50 2024-05-10T10:29:33.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.627+0000 7f3728ff9700 1 -- 172.21.15.112:0/3620020803 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f372400cd80 con 0x7f372c197c50 2024-05-10T10:29:33.629 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:33.629 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-05-10T10:29:33.629 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:29:33.629 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f371803e9a0 msgr2=0x7f3718040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f371803e9a0 0x7f3718040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f371c015040 tx=0x7f371c005040 comp rx=0 tx=0).stop 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c197c50 msgr2=0x7f372c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c197c50 0x7f372c198100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f372400b550 tx=0x7f372400b860 comp rx=0 tx=0).stop 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 shutdown_connections 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c197c50 0x7f372c198100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f372c101fe0 0x7f372c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f371803e9a0 0x7f3718040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 --2- 172.21.15.112:0/3620020803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f372c19db80 0x7f372c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.629+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 >> 172.21.15.112:0/3620020803 conn(0x7f372c078a70 msgr2=0x7f372c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.630+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 shutdown_connections 2024-05-10T10:29:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:33.630+0000 7f3731bd3700 1 -- 172.21.15.112:0/3620020803 wait complete. 2024-05-10T10:29:34.251 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-10T10:29:34.252 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:29:34.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T10:29:34.278 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:29:34.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:29:34.344 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:29:34.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T10:29:34.378 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:29:34.378 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:29:34.448 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:29:34.448 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T10:29:34.481 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:29:34.481 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T10:29:34.518 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:34 smithi112 ceph-mon[30100]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:34.518 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:34.518 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:34.518 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:34 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3620020803' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:34.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:34 smithi148 ceph-mon[33568]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:34.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:34.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:34.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:34 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3620020803' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:34.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:34 smithi195 ceph-mon[33773]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:34.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:34.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:29:34.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:34 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3620020803' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:34.550 INFO:tasks.cephadm:Adding mgr.x on smithi112 2024-05-10T10:29:34.550 INFO:tasks.cephadm:Adding mgr.y on smithi148 2024-05-10T10:29:34.550 INFO:tasks.cephadm:Adding mgr.z on smithi195 2024-05-10T10:29:34.550 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi112=x;smithi148=y;smithi195=z' 2024-05-10T10:29:34.827 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:29:36.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.267+0000 7f65c3780700 1 -- 172.21.15.195:0/223280582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc103010 msgr2=0x7f65bc103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.267+0000 7f65c3780700 1 --2- 172.21.15.195:0/223280582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc103010 0x7f65bc103430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009910 tx=0x7f65b8009c20 comp rx=0 tx=0).stop 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.267+0000 7f65c3780700 1 -- 172.21.15.195:0/223280582 shutdown_connections 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.267+0000 7f65c3780700 1 --2- 172.21.15.195:0/223280582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc103010 0x7f65bc103430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.267+0000 7f65c3780700 1 -- 172.21.15.195:0/223280582 >> 172.21.15.195:0/223280582 conn(0x7f65bc0fe5d0 msgr2=0x7f65bc1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.267+0000 7f65c3780700 1 -- 172.21.15.195:0/223280582 shutdown_connections 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.267+0000 7f65c3780700 1 -- 172.21.15.195:0/223280582 wait complete. 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 Processor -- start 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 -- start start 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc103010 0x7f65bc077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc075890 0x7f65bc075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65bc076250 0x7f65bc1a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc108db0 con 0x7f65bc075890 2024-05-10T10:29:36.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc108c30 con 0x7f65bc103010 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c3780700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc108f30 con 0x7f65bc076250 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c1d1d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65bc076250 0x7f65bc1a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c1d1d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65bc076250 0x7f65bc1a2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:44198/0 (socket says 172.21.15.195:44198) 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c1d1d700 1 -- 172.21.15.195:0/1697255104 learned_addr learned my addr 172.21.15.195:0/1697255104 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c0d1b700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc075890 0x7f65bc075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c151c700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc103010 0x7f65bc077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c1d1d700 1 -- 172.21.15.195:0/1697255104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc103010 msgr2=0x7f65bc077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c1d1d700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc103010 0x7f65bc077240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.268+0000 7f65c1d1d700 1 -- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc075890 msgr2=0x7f65bc075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65c1d1d700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc075890 0x7f65bc075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65c1d1d700 1 -- 172.21.15.195:0/1697255104 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc1a2cc0 con 0x7f65bc076250 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65c1d1d700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65bc076250 0x7f65bc1a2700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f65b400ba50 tx=0x7f65b400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65b27fc700 1 -- 172.21.15.195:0/1697255104 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4019070 con 0x7f65bc076250 2024-05-10T10:29:36.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65b27fc700 1 -- 172.21.15.195:0/1697255104 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65b400cdc0 con 0x7f65bc076250 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65b27fc700 1 -- 172.21.15.195:0/1697255104 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4014e60 con 0x7f65bc076250 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc1a2fb0 con 0x7f65bc076250 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.269+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1a3490 con 0x7f65bc076250 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.270+0000 7f65b27fc700 1 -- 172.21.15.195:0/1697255104 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f65b401c070 con 0x7f65bc076250 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.270+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc064b00 con 0x7f65bc076250 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.270+0000 7f65b27fc700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65a803e9a0 0x7f65a8040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.270+0000 7f65b27fc700 1 -- 172.21.15.195:0/1697255104 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f65b4051dd0 con 0x7f65bc076250 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.270+0000 7f65c151c700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65a803e9a0 0x7f65a8040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:36.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.271+0000 7f65c151c700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65a803e9a0 0x7f65a8040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65b800b5c0 tx=0x7f65b8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:36.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.275+0000 7f65b27fc700 1 -- 172.21.15.195:0/1697255104 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f65b40145f0 con 0x7f65bc076250 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: Reconfiguring mon.b (monmap changed)... 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:36 smithi195 ceph-mon[33773]: Reconfiguring daemon mon.b on smithi148 2024-05-10T10:29:36.491 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.491+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi148=y;smithi195=z", "target": ["mon-mgr", ""]}) -- 0x7f65bc1005e0 con 0x7f65a803e9a0 2024-05-10T10:29:36.500 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.500+0000 7f65b27fc700 1 -- 172.21.15.195:0/1697255104 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f65bc1005e0 con 0x7f65a803e9a0 2024-05-10T10:29:36.500 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mgr update... 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65a803e9a0 msgr2=0x7f65a8040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65a803e9a0 0x7f65a8040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65b800b5c0 tx=0x7f65b8005fb0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65bc076250 msgr2=0x7f65bc1a2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65bc076250 0x7f65bc1a2700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f65b400ba50 tx=0x7f65b400bd60 comp rx=0 tx=0).stop 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 shutdown_connections 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65a803e9a0 0x7f65a8040e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65bc076250 0x7f65bc1a2700 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc103010 0x7f65bc077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 --2- 172.21.15.195:0/1697255104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65bc075890 0x7f65bc075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:36.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 >> 172.21.15.195:0/1697255104 conn(0x7f65bc0fe5d0 msgr2=0x7f65bc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:36.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 shutdown_connections 2024-05-10T10:29:36.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:29:36.502+0000 7f65c3780700 1 -- 172.21.15.195:0/1697255104 wait complete. 2024-05-10T10:29:36.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:36.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:36.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: Reconfiguring mon.b (monmap changed)... 2024-05-10T10:29:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:36 smithi148 ceph-mon[33568]: Reconfiguring daemon mon.b on smithi148 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: Reconfiguring mon.b (monmap changed)... 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:36.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:36 smithi112 ceph-mon[30100]: Reconfiguring daemon mon.b on smithi148 2024-05-10T10:29:37.087 DEBUG:teuthology.orchestra.run.smithi148:mgr.y> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.y.service 2024-05-10T10:29:37.090 DEBUG:teuthology.orchestra.run.smithi195:mgr.z> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.z.service 2024-05-10T10:29:37.092 INFO:tasks.cephadm:Deploying OSDs... 2024-05-10T10:29:37.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:29:37.092 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T10:29:37.112 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T10:29:37.112 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-05-10T10:29:37.172 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T10:29:37.172 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:37.172 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-10T10:29:37.172 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:37.172 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:37.173 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-10 10:28:42.969347758 +0000 2024-05-10T10:29:37.173 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-10 10:24:34.539738709 +0000 2024-05-10T10:29:37.173 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-10 10:24:34.539738709 +0000 2024-05-10T10:29:37.173 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-10 10:24:34.539738709 +0000 2024-05-10T10:29:37.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T10:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-10T10:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-10T10:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000232533 s, 2.2 MB/s 2024-05-10T10:29:37.244 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T10:29:37.302 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-05-10T10:29:37.360 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T10:29:37.360 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:37.361 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-10T10:29:37.361 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:37.361 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:37.361 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-10 10:28:43.000346935 +0000 2024-05-10T10:29:37.361 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-10 10:24:34.900728911 +0000 2024-05-10T10:29:37.361 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-10 10:24:34.900728911 +0000 2024-05-10T10:29:37.361 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-10 10:24:34.900728911 +0000 2024-05-10T10:29:37.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T10:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-10T10:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-10T10:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000239341 s, 2.1 MB/s 2024-05-10T10:29:37.429 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T10:29:37.487 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-05-10T10:29:37.544 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T10:29:37.544 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:37.544 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-10T10:29:37.545 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:37.545 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:37.545 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-10 10:28:43.032346086 +0000 2024-05-10T10:29:37.545 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-10 10:24:35.234719846 +0000 2024-05-10T10:29:37.545 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-10 10:24:35.234719846 +0000 2024-05-10T10:29:37.545 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-10 10:24:35.234719846 +0000 2024-05-10T10:29:37.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T10:29:37.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi148=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:37.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: Saving service mgr spec with placement smithi112=x;smithi148=y;smithi195=z;count:3 2024-05-10T10:29:37.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: Reconfiguring mon.c (monmap changed)... 2024-05-10T10:29:37.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:37.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:37.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:37.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:37 smithi195 ceph-mon[33773]: Reconfiguring daemon mon.c on smithi195 2024-05-10T10:29:37.608 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-10T10:29:37.609 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-10T10:29:37.609 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000188936 s, 2.7 MB/s 2024-05-10T10:29:37.610 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T10:29:37.667 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-05-10T10:29:37.724 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T10:29:37.724 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:37.724 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-10T10:29:37.724 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:37.724 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:37.724 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-10 10:28:43.065345211 +0000 2024-05-10T10:29:37.725 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-10 10:24:35.582710401 +0000 2024-05-10T10:29:37.725 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-10 10:24:35.582710401 +0000 2024-05-10T10:29:37.725 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-10 10:24:35.582710401 +0000 2024-05-10T10:29:37.725 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T10:29:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi148=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: Saving service mgr spec with placement smithi112=x;smithi148=y;smithi195=z;count:3 2024-05-10T10:29:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: Reconfiguring mon.c (monmap changed)... 2024-05-10T10:29:37.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:37.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:37.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:37.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:37 smithi112 ceph-mon[30100]: Reconfiguring daemon mon.c on smithi195 2024-05-10T10:29:37.789 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-10T10:29:37.790 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-10T10:29:37.790 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000205051 s, 2.5 MB/s 2024-05-10T10:29:37.791 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T10:29:37.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi148=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:37.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: Saving service mgr spec with placement smithi112=x;smithi148=y;smithi195=z;count:3 2024-05-10T10:29:37.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:37.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: Reconfiguring mon.c (monmap changed)... 2024-05-10T10:29:37.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T10:29:37.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T10:29:37.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:37.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:37 smithi148 ceph-mon[33568]: Reconfiguring daemon mon.c on smithi195 2024-05-10T10:29:37.848 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-05-10T10:29:37.848 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T10:29:37.869 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T10:29:37.870 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-05-10T10:29:37.928 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T10:29:37.928 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:37.928 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-10T10:29:37.929 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:37.929 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:37.929 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-05-10 10:29:24.958262984 +0000 2024-05-10T10:29:37.929 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-05-10 10:24:35.947514984 +0000 2024-05-10T10:29:37.929 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-05-10 10:24:35.947514984 +0000 2024-05-10T10:29:37.929 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-05-10 10:24:35.947514984 +0000 2024-05-10T10:29:37.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T10:29:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-05-10T10:29:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-05-10T10:29:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000269696 s, 1.9 MB/s 2024-05-10T10:29:37.999 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T10:29:38.059 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-05-10T10:29:38.116 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T10:29:38.116 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:38.116 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-10T10:29:38.116 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:38.116 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:38.116 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-05-10 10:29:24.958262984 +0000 2024-05-10T10:29:38.117 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-05-10 10:24:36.308506574 +0000 2024-05-10T10:29:38.117 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-05-10 10:24:36.308506574 +0000 2024-05-10T10:29:38.117 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-05-10 10:24:36.308506574 +0000 2024-05-10T10:29:38.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T10:29:38.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-05-10T10:29:38.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-05-10T10:29:38.183 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000270058 s, 1.9 MB/s 2024-05-10T10:29:38.185 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T10:29:38.242 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-05-10T10:29:38.303 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T10:29:38.303 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:38.303 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-10T10:29:38.303 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:38.304 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:38.304 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-05-10 10:29:24.958262984 +0000 2024-05-10T10:29:38.304 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-05-10 10:24:36.667498211 +0000 2024-05-10T10:29:38.304 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-05-10 10:24:36.667498211 +0000 2024-05-10T10:29:38.304 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-05-10 10:24:36.667498211 +0000 2024-05-10T10:29:38.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T10:29:38.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-05-10T10:29:38.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-05-10T10:29:38.368 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000203251 s, 2.5 MB/s 2024-05-10T10:29:38.369 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T10:29:38.427 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-05-10T10:29:38.484 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T10:29:38.484 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:38.484 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-10T10:29:38.484 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:38.484 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:38.484 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-05-10 10:29:24.959262962 +0000 2024-05-10T10:29:38.484 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-05-10 10:24:37.033489685 +0000 2024-05-10T10:29:38.485 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-05-10 10:24:37.033489685 +0000 2024-05-10T10:29:38.485 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-05-10 10:24:37.033489685 +0000 2024-05-10T10:29:38.485 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T10:29:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-05-10T10:29:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-05-10T10:29:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.00024707 s, 2.1 MB/s 2024-05-10T10:29:38.550 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T10:29:38.607 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-10T10:29:38.608 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T10:29:38.626 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T10:29:38.626 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-05-10T10:29:38.684 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T10:29:38.684 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:38.684 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-10T10:29:38.684 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:38.684 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:38.685 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-10 10:29:15.375470004 +0000 2024-05-10T10:29:38.685 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-10 10:24:35.510786987 +0000 2024-05-10T10:29:38.685 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-10 10:24:35.510786987 +0000 2024-05-10T10:29:38.685 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-10 10:24:35.510786987 +0000 2024-05-10T10:29:38.686 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T10:29:38.750 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-10T10:29:38.751 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-10T10:29:38.751 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.00012095 s, 4.2 MB/s 2024-05-10T10:29:38.752 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T10:29:38.809 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-05-10T10:29:38.864 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T10:29:38.864 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:38.865 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-10T10:29:38.865 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:38.865 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:38.865 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-10 10:29:15.375470004 +0000 2024-05-10T10:29:38.865 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-10 10:24:35.861780065 +0000 2024-05-10T10:29:38.865 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-10 10:24:35.861780065 +0000 2024-05-10T10:29:38.865 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-10 10:24:35.861780065 +0000 2024-05-10T10:29:38.865 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T10:29:38.927 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:38 smithi195 ceph-mon[33773]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:38.927 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:38.927 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:38.927 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:38.929 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-10T10:29:38.929 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-10T10:29:38.930 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000112976 s, 4.5 MB/s 2024-05-10T10:29:38.931 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T10:29:38.987 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-05-10T10:29:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T10:29:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:39.045 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-10T10:29:39.045 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:39.045 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:39.045 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-10 10:29:15.376469986 +0000 2024-05-10T10:29:39.045 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-10 10:24:36.210773183 +0000 2024-05-10T10:29:39.045 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-10 10:24:36.210773183 +0000 2024-05-10T10:29:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-10 10:24:36.210773183 +0000 2024-05-10T10:29:39.046 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T10:29:39.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:38 smithi148 ceph-mon[33568]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:39.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:39.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:39.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:39.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:38 smithi112 ceph-mon[30100]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:39.108 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-10T10:29:39.108 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-10T10:29:39.109 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000115352 s, 4.4 MB/s 2024-05-10T10:29:39.110 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T10:29:39.166 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-05-10T10:29:39.223 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-10 10:29:15.376469986 +0000 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-10 10:24:36.549766498 +0000 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-10 10:24:36.549766498 +0000 2024-05-10T10:29:39.224 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-10 10:24:36.549766498 +0000 2024-05-10T10:29:39.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T10:29:39.289 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-10T10:29:39.289 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-10T10:29:39.289 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000111995 s, 4.6 MB/s 2024-05-10T10:29:39.291 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T10:29:39.348 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-05-10T10:29:39.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T10:29:39.581 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:40.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:39 smithi112 ceph-mon[30100]: Deploying daemon mgr.z on smithi195 2024-05-10T10:29:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T10:29:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:40.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:40.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:39 smithi148 ceph-mon[33568]: Deploying daemon mgr.z on smithi195 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:39 smithi195 ceph-mon[33773]: Deploying daemon mgr.z on smithi195 2024-05-10T10:29:40.982 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:40 smithi112 ceph-mon[30100]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:40.982 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:40 smithi195 ceph-mon[33773]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:41.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:40 smithi148 ceph-mon[33568]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:41.276 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:41 smithi195 ceph-mgr[35641]: -- 172.21.15.195:0/2338294315 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a8e9965a0 con 0x557a8e96ac00 2024-05-10T10:29:41.545 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:41 smithi195 bash[35622]: c839f810009dd06697fd482171633689826292b8717c695db5161bc399c7496b 2024-05-10T10:29:41.545 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:41 smithi195 systemd[1]: Started Ceph mgr.z for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:29:41.545 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:41.356+0000 7fe65d676200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T10:29:41.545 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:41.543+0000 7fe65d676200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:29:41.555 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T10:29:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T10:29:41.556 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-10T10:29:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-10T10:29:41.556 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134897 s, 777 MB/s 2024-05-10T10:29:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-10T10:29:41.816 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:41.645+0000 7fe65d676200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T10:29:42.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-05-10T10:29:42.316 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:41.953+0000 7fe65d676200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T10:29:42.316 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:42.038+0000 7fe65d676200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T10:29:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:29:42.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:42.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:42.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T10:29:42.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:42.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:42.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:42 smithi112 ceph-mon[30100]: Deploying daemon mgr.y on smithi148 2024-05-10T10:29:42.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:42 smithi148 ceph-mon[33568]: Deploying daemon mgr.y on smithi148 2024-05-10T10:29:42.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:42.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:42.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:42.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T10:29:42.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:42.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:42.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:42 smithi195 ceph-mon[33773]: Deploying daemon mgr.y on smithi148 2024-05-10T10:29:43.545 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 systemd[1]: Starting Ceph mgr.y for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:29:43.566 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:43.238+0000 7fe65d676200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T10:29:43.566 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:43.414+0000 7fe65d676200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T10:29:43.870 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 podman[34931]: 2024-05-10 10:29:43.502275219 +0000 UTC m=+0.018307794 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:29:43.871 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 podman[34931]: 2024-05-10 10:29:43.631719502 +0000 UTC m=+0.147752072 container create 6f205209571bb871ca6eee07c67c7d57026db2d9c88dde622410cbb1ce6622f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-10T10:29:43.871 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 podman[34931]: 2024-05-10 10:29:43.842367527 +0000 UTC m=+0.358400108 container init 6f205209571bb871ca6eee07c67c7d57026db2d9c88dde622410cbb1ce6622f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:29:43.871 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 podman[34931]: 2024-05-10 10:29:43.845245247 +0000 UTC m=+0.361277818 container start 6f205209571bb871ca6eee07c67c7d57026db2d9c88dde622410cbb1ce6622f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-10T10:29:43.871 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 ceph-mgr[34951]: -- 172.21.15.148:0/2723328293 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555aa1da25a0 con 0x555aa1d77000 2024-05-10T10:29:43.897 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:43.593+0000 7fe65d676200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T10:29:43.897 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:43.895+0000 7fe65d676200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T10:29:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 -- 172.21.15.112:0/585471615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc464102e40 msgr2=0x7fc4641032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/585471615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc464102e40 0x7fc4641032d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00b0a0 tx=0x7fc45c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:29:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 -- 172.21.15.112:0/585471615 shutdown_connections 2024-05-10T10:29:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/585471615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc464102e40 0x7fc4641032d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/585471615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc46406b3a0 0x7fc464101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/585471615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406b850 0x7fc4641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 -- 172.21.15.112:0/585471615 >> 172.21.15.112:0/585471615 conn(0x7fc4640fd1e0 msgr2=0x7fc4640ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 -- 172.21.15.112:0/585471615 shutdown_connections 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 -- 172.21.15.112:0/585471615 wait complete. 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 Processor -- start 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.940+0000 7fc46bcc2700 1 -- start start 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406b3a0 0x7fc4641995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc46406b850 0x7fc464199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc464102e40 0x7fc4641a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc46410de80 con 0x7fc46406b850 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc46410dd00 con 0x7fc46406b3a0 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc46410e000 con 0x7fc464102e40 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc46406b850 0x7fc464199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:43.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc46406b850 0x7fc464199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37782/0 (socket says 172.21.15.112:37782) 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 -- 172.21.15.112:0/2057296818 learned_addr learned my addr 172.21.15.112:0/2057296818 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc469a5e700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406b3a0 0x7fc4641995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46a25f700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc464102e40 0x7fc4641a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 -- 172.21.15.112:0/2057296818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc464102e40 msgr2=0x7fc4641a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc464102e40 0x7fc4641a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 -- 172.21.15.112:0/2057296818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406b3a0 msgr2=0x7fc4641995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406b3a0 0x7fc4641995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 -- 172.21.15.112:0/2057296818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4641a1200 con 0x7fc46406b850 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46925d700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc46406b850 0x7fc464199b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d9b0 tx=0x7fc45400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454017070 con 0x7fc46406b850 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc454009dc0 con 0x7fc46406b850 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4641a14f0 con 0x7fc46406b850 2024-05-10T10:29:43.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454010e10 con 0x7fc46406b850 2024-05-10T10:29:43.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.941+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4641a1a80 con 0x7fc46406b850 2024-05-10T10:29:43.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.942+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46406a190 con 0x7fc46406b850 2024-05-10T10:29:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.942+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fc45400f3c0 con 0x7fc46406b850 2024-05-10T10:29:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.943+0000 7fc45affd700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc45003e750 0x7fc450040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:29:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.943+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc4540523e0 con 0x7fc46406b850 2024-05-10T10:29:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.943+0000 7fc469a5e700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc45003e750 0x7fc450040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:29:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.943+0000 7fc469a5e700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc45003e750 0x7fc450040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc464102ca0 tx=0x7fc46000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:29:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:43.947+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc454010580 con 0x7fc46406b850 2024-05-10T10:29:44.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:44.169+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc4640638e0 con 0x7fc45003e750 2024-05-10T10:29:44.189 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 bash[34931]: 6f205209571bb871ca6eee07c67c7d57026db2d9c88dde622410cbb1ce6622f4 2024-05-10T10:29:44.190 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 systemd[1]: Started Ceph mgr.y for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:29:44.190 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:43.998+0000 7fc8da6c9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T10:29:44.190 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:44.188+0000 7fc8da6c9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:29:44.207 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:43.992+0000 7fe65d676200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T10:29:44.208 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:44.206+0000 7fe65d676200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T10:29:44.544 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:44.297+0000 7fc8da6c9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T10:29:44.566 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:44.411+0000 7fe65d676200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:29:44.996 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:44.637+0000 7fc8da6c9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T10:29:44.996 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:44.732+0000 7fc8da6c9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T10:29:45.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:45.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:29:45.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:29:45.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:45.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:45.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:45.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:45.263 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:45.064+0000 7fe65d676200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T10:29:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:29:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:29:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:45.566 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:45.296+0000 7fe65d676200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T10:29:46.006 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:45.733+0000 7fe65d676200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T10:29:46.006 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:45.833+0000 7fe65d676200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:46.070 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.071 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:46.071 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:46.071 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:46.294 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:46.034+0000 7fc8da6c9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T10:29:46.294 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:46.206+0000 7fc8da6c9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T10:29:46.297 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:46.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:46.316 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:46.208+0000 7fe65d676200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T10:29:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:29:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:46.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:46.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:46.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T10:29:46.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T10:29:46.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:46.670 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:46.379+0000 7fc8da6c9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T10:29:46.670 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:46.669+0000 7fc8da6c9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T10:29:46.906 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:46.566+0000 7fe65d676200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T10:29:46.906 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:46.660+0000 7fe65d676200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T10:29:46.972 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:46.764+0000 7fc8da6c9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T10:29:46.972 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:46.971+0000 7fc8da6c9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T10:29:47.294 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:47.168+0000 7fc8da6c9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:29:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:47 smithi148 ceph-mon[33568]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:47.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:47 smithi148 ceph-mon[33568]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T10:29:47.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:47 smithi148 ceph-mon[33568]: Reconfiguring daemon mgr.x on smithi112 2024-05-10T10:29:47.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:47 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2648552454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a67e2c4c-f481-4fd7-b502-3d561eb6ffc6"}]: dispatch 2024-05-10T10:29:47.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:47 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2648552454' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a67e2c4c-f481-4fd7-b502-3d561eb6ffc6"}]': finished 2024-05-10T10:29:47.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:47 smithi148 ceph-mon[33568]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T10:29:47.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:29:47.316 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:46.904+0000 7fe65d676200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T10:29:47.316 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:47.039+0000 7fe65d676200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T10:29:47.316 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:47.134+0000 7fe65d676200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T10:29:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:47 smithi195 ceph-mon[33773]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:47 smithi195 ceph-mon[33773]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T10:29:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:47 smithi195 ceph-mon[33773]: Reconfiguring daemon mgr.x on smithi112 2024-05-10T10:29:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:47 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2648552454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a67e2c4c-f481-4fd7-b502-3d561eb6ffc6"}]: dispatch 2024-05-10T10:29:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:47 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2648552454' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a67e2c4c-f481-4fd7-b502-3d561eb6ffc6"}]': finished 2024-05-10T10:29:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:47 smithi195 ceph-mon[33773]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T10:29:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:29:47.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:47 smithi112 ceph-mon[30100]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:47.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:47 smithi112 ceph-mon[30100]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T10:29:47.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:47 smithi112 ceph-mon[30100]: Reconfiguring daemon mgr.x on smithi112 2024-05-10T10:29:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:47 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2648552454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a67e2c4c-f481-4fd7-b502-3d561eb6ffc6"}]: dispatch 2024-05-10T10:29:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:47 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2648552454' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a67e2c4c-f481-4fd7-b502-3d561eb6ffc6"}]': finished 2024-05-10T10:29:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:47 smithi112 ceph-mon[30100]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T10:29:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:29:47.816 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:47.465+0000 7fe65d676200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T10:29:47.816 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:47.559+0000 7fe65d676200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T10:29:48.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:48 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2596416121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:29:48.045 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:47.771+0000 7fc8da6c9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T10:29:48.045 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:47.992+0000 7fc8da6c9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T10:29:48.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:48 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2596416121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:29:48.316 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:48.167+0000 7fe65d676200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T10:29:48.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:48 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2596416121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:29:48.794 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:48.421+0000 7fc8da6c9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T10:29:48.794 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:48.517+0000 7fc8da6c9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T10:29:48.815 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:29:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z[35637]: 2024-05-10T10:29:48.383+0000 7fe65d676200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T10:29:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:49.245+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7fc4540224e0 con 0x7fc46406b850 2024-05-10T10:29:49.252 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:48.890+0000 7fc8da6c9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T10:29:49.252 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:49.251+0000 7fc8da6c9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T10:29:49.252 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:49 smithi148 ceph-mon[33568]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:49.252 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:49.252 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:49 smithi148 ceph-mon[33568]: Standby manager daemon z started 2024-05-10T10:29:49.252 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:49 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T10:29:49.252 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:49 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T10:29:49.253 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:49 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T10:29:49.253 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:49 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T10:29:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:49 smithi195 ceph-mon[33773]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:49 smithi195 ceph-mon[33773]: Standby manager daemon z started 2024-05-10T10:29:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:49 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T10:29:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:49 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T10:29:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:49 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T10:29:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:49 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T10:29:49.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:49 smithi112 ceph-mon[30100]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:49 smithi112 ceph-mon[30100]: Standby manager daemon z started 2024-05-10T10:29:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:49 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T10:29:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:49 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T10:29:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:49 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T10:29:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:49 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.195:0/1634619187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T10:29:49.544 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:49.343+0000 7fc8da6c9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T10:29:50.044 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:49.588+0000 7fc8da6c9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T10:29:50.044 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:49.721+0000 7fc8da6c9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T10:29:50.044 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:49.817+0000 7fc8da6c9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T10:29:50.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:50 smithi148 ceph-mon[33568]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:50.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:50 smithi148 ceph-mon[33568]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-10T10:29:50.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T10:29:50.545 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:50.154+0000 7fc8da6c9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T10:29:50.545 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:50.250+0000 7fc8da6c9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T10:29:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:50 smithi195 ceph-mon[33773]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:50 smithi195 ceph-mon[33773]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-10T10:29:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T10:29:50.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:50 smithi112 ceph-mon[30100]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:50.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:50 smithi112 ceph-mon[30100]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-10T10:29:50.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T10:29:51.246 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:50.853+0000 7fc8da6c9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T10:29:51.246 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:29:51 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y[34947]: 2024-05-10T10:29:51.071+0000 7fc8da6c9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T10:29:51.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:29:51.255+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fc454065cc0 con 0x7fc46406b850 2024-05-10T10:29:51.256 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:51 smithi112 ceph-mon[30100]: Standby manager daemon y started 2024-05-10T10:29:51.257 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:51 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T10:29:51.257 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:51 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T10:29:51.257 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:51 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T10:29:51.257 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:51 smithi112 ceph-mon[30100]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T10:29:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:51 smithi148 ceph-mon[33568]: Standby manager daemon y started 2024-05-10T10:29:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:51 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T10:29:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:51 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T10:29:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:51 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T10:29:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:51 smithi148 ceph-mon[33568]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T10:29:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:51 smithi195 ceph-mon[33773]: Standby manager daemon y started 2024-05-10T10:29:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:51 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T10:29:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:51 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T10:29:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:51 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T10:29:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:51 smithi195 ceph-mon[33773]: from='mgr.? 172.21.15.148:0/3499768780' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T10:29:52.413 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:52.413 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-10T10:29:52.413 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T10:29:52.413 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:52.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:52.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:52.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:52.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:29:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:29:54.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:54 smithi195 ceph-mon[33773]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:54.518 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:54 smithi112 ceph-mon[30100]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:54.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:54 smithi148 ceph-mon[33568]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:55.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T10:29:55.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:55.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:55 smithi112 ceph-mon[30100]: Deploying daemon osd.0 on smithi112 2024-05-10T10:29:55.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T10:29:55.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:55.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:55 smithi148 ceph-mon[33568]: Deploying daemon osd.0 on smithi112 2024-05-10T10:29:55.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T10:29:55.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:29:55.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:55 smithi195 ceph-mon[33773]: Deploying daemon osd.0 on smithi112 2024-05-10T10:29:56.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:56 smithi112 ceph-mon[30100]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:56.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:56 smithi148 ceph-mon[33568]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:56.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:56 smithi195 ceph-mon[33773]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:58.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:29:58 smithi148 ceph-mon[33568]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:58.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:29:58 smithi195 ceph-mon[33773]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:29:58.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:29:58 smithi112 ceph-mon[30100]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:00 smithi148 ceph-mon[33568]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:00 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:00 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:01.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:00 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:01.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:00 smithi148 ceph-mon[33568]: overall HEALTH_OK 2024-05-10T10:30:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:00 smithi195 ceph-mon[33773]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:00 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:00 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:00 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:00 smithi195 ceph-mon[33773]: overall HEALTH_OK 2024-05-10T10:30:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:00 smithi112 ceph-mon[30100]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:00 smithi112 ceph-mon[30100]: overall HEALTH_OK 2024-05-10T10:30:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:02 smithi148 ceph-mon[33568]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:03.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:03.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:03.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:02 smithi195 ceph-mon[33773]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:02 smithi112 ceph-mon[30100]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:03.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.476+0000 7fc45affd700 1 -- 172.21.15.112:0/2057296818 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc4640638e0 con 0x7fc45003e750 2024-05-10T10:30:03.478 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-05-10T10:30:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.479+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc45003e750 msgr2=0x7fc450040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.479+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc45003e750 0x7fc450040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc464102ca0 tx=0x7fc46000b410 comp rx=0 tx=0).stop 2024-05-10T10:30:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.479+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc46406b850 msgr2=0x7fc464199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.479+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc46406b850 0x7fc464199b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d9b0 tx=0x7fc45400dcc0 comp rx=0 tx=0).stop 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 shutdown_connections 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc45003e750 0x7fc450040c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc464102e40 0x7fc4641a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406b3a0 0x7fc4641995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 --2- 172.21.15.112:0/2057296818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc46406b850 0x7fc464199b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 >> 172.21.15.112:0/2057296818 conn(0x7fc4640fd1e0 msgr2=0x7fc4640fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 shutdown_connections 2024-05-10T10:30:03.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:03.480+0000 7fc46bcc2700 1 -- 172.21.15.112:0/2057296818 wait complete. 2024-05-10T10:30:04.176 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.0.service 2024-05-10T10:30:04.179 INFO:tasks.cephadm:Deploying osd.1 on smithi148 with /dev/vg_nvme/lv_4... 2024-05-10T10:30:04.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T10:30:04.448 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:30:04.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:04 smithi148 ceph-mon[33568]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:04.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:04 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:04.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:04 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:04.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:04 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:04.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:04 smithi148 ceph-mon[33568]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T10:30:04.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:04 smithi112 ceph-mon[30100]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:04.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:04.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:04.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:04.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:04 smithi112 ceph-mon[30100]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T10:30:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:04 smithi195 ceph-mon[33773]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:04 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:04 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:04 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:04 smithi195 ceph-mon[33773]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T10:30:05.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:05 smithi148 ceph-mon[33568]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T10:30:05.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:05 smithi148 ceph-mon[33568]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T10:30:05.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:05 smithi148 ceph-mon[33568]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:30:05.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:05 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:05.746 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:05 smithi112 ceph-mon[30100]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T10:30:05.746 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:05 smithi112 ceph-mon[30100]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T10:30:05.746 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:05 smithi112 ceph-mon[30100]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:30:05.746 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:05 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:05.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:05 smithi195 ceph-mon[33773]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T10:30:05.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:05 smithi195 ceph-mon[33773]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T10:30:05.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:05 smithi195 ceph-mon[33773]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:30:05.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:05 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0[42393]: 2024-05-10T10:30:06.139+0000 7f31ec5fe700 -1 osd.0 0 waiting for initial osdmap 2024-05-10T10:30:06.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0[42393]: 2024-05-10T10:30:06.143+0000 7f31e43e9700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:30:06.408 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T10:30:06.409 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T10:30:06.409 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-05-10T10:30:06.409 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-05-10T10:30:06.409 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112086 s, 936 MB/s 2024-05-10T10:30:06.409 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-05-10T10:30:06.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:06 smithi148 ceph-mon[33568]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:06.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:06 smithi148 ceph-mon[33568]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:30:06.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:06 smithi148 ceph-mon[33568]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T10:30:06.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:06 smithi148 ceph-mon[33568]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' 2024-05-10T10:30:06.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:06 smithi195 ceph-mon[33773]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:06.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:06 smithi195 ceph-mon[33773]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:30:06.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:06 smithi195 ceph-mon[33773]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T10:30:06.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:06 smithi195 ceph-mon[33773]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' 2024-05-10T10:30:06.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:06 smithi112 ceph-mon[30100]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:06.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:06 smithi112 ceph-mon[30100]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:30:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:06 smithi112 ceph-mon[30100]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T10:30:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:06 smithi112 ceph-mon[30100]: from='osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326]' entity='osd.0' 2024-05-10T10:30:06.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:06.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-05-10T10:30:07.188 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] boot 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.567 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:07.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:07.568 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:30:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:30:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] boot 2024-05-10T10:30:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T10:30:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:07.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:07.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:07.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:30:07.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:30:07.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: osd.0 [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] boot 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:07.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:08.590 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:08 smithi148 ceph-mon[33568]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:08.590 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:08 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi112 2024-05-10T10:30:08.590 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:08 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi112 to 17088M 2024-05-10T10:30:08.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 -- 172.21.15.148:0/4091162264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 msgr2=0x7fa8601032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:08.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 --2- 172.21.15.148:0/4091162264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa8601032f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa850009990 tx=0x7fa850009ca0 comp rx=0 tx=0).stop 2024-05-10T10:30:08.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 -- 172.21.15.148:0/4091162264 shutdown_connections 2024-05-10T10:30:08.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 --2- 172.21.15.148:0/4091162264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa860107b70 0x7fa860109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:08.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 --2- 172.21.15.148:0/4091162264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa86006a4a0 0x7fa860102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:08.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 --2- 172.21.15.148:0/4091162264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa8601032f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:08.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 -- 172.21.15.148:0/4091162264 >> 172.21.15.148:0/4091162264 conn(0x7fa8600fccf0 msgr2=0x7fa8600ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:30:08.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 -- 172.21.15.148:0/4091162264 shutdown_connections 2024-05-10T10:30:08.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 -- 172.21.15.148:0/4091162264 wait complete. 2024-05-10T10:30:08.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.804+0000 7fa867755700 1 Processor -- start 2024-05-10T10:30:08.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa867755700 1 -- start start 2024-05-10T10:30:08.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa867755700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa86006a4a0 0x7fa860197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:08.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa867755700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa860197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:08.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa867755700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa860107b70 0x7fa8601a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:08.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa867755700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa86010d4b0 con 0x7fa860107b70 2024-05-10T10:30:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa867755700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa86010d330 con 0x7fa860102e70 2024-05-10T10:30:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa867755700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa86010d630 con 0x7fa86006a4a0 2024-05-10T10:30:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa860197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa860197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34926/0 (socket says 172.21.15.148:34926) 2024-05-10T10:30:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 -- 172.21.15.148:0/709148600 learned_addr learned my addr 172.21.15.148:0/709148600 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-05-10T10:30:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa8654f1700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa86006a4a0 0x7fa860197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa865cf2700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa860107b70 0x7fa8601a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 -- 172.21.15.148:0/709148600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa86006a4a0 msgr2=0x7fa860197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa86006a4a0 0x7fa860197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 -- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa860107b70 msgr2=0x7fa8601a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa860107b70 0x7fa8601a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 -- 172.21.15.148:0/709148600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8601a1140 con 0x7fa860102e70 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa8654f1700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa86006a4a0 0x7fa860197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.805+0000 7fa864cf0700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa860197e60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa850005910 tx=0x7fa8500047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.806+0000 7fa8567fc700 1 -- 172.21.15.148:0/709148600 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850027070 con 0x7fa860102e70 2024-05-10T10:30:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.806+0000 7fa8567fc700 1 -- 172.21.15.148:0/709148600 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa850022470 con 0x7fa860102e70 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.806+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8601a13d0 con 0x7fa860102e70 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.806+0000 7fa8567fc700 1 -- 172.21.15.148:0/709148600 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85000f6c0 con 0x7fa860102e70 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.806+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8601a1880 con 0x7fa860102e70 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.806+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86006a190 con 0x7fa860102e70 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.808+0000 7fa8567fc700 1 -- 172.21.15.148:0/709148600 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fa85002a070 con 0x7fa860102e70 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.809+0000 7fa8567fc700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa84c0b2870 0x7fa84c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.809+0000 7fa8567fc700 1 -- 172.21.15.148:0/709148600 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa8500e1cc0 con 0x7fa860102e70 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.809+0000 7fa8654f1700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa84c0b2870 0x7fa84c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.809+0000 7fa8654f1700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa84c0b2870 0x7fa84c0b4d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa860102790 tx=0x7fa85c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:30:08.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:08 smithi195 ceph-mon[33773]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:08.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:08 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi112 2024-05-10T10:30:08.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:08 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi112 to 17088M 2024-05-10T10:30:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:08.812+0000 7fa8567fc700 1 -- 172.21.15.148:0/709148600 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa850031070 con 0x7fa860102e70 2024-05-10T10:30:08.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:08 smithi112 ceph-mon[30100]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T10:30:08.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:08 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi112 2024-05-10T10:30:08.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:08 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi112 to 17088M 2024-05-10T10:30:09.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:09.023+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa8600638e0 con 0x7fa84c0b2870 2024-05-10T10:30:09.760 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:09 smithi148 ceph-mon[33568]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T10:30:09.760 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:09 smithi148 ceph-mon[33568]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:09.761 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:09.761 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:09.761 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:09.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:09 smithi195 ceph-mon[33773]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T10:30:09.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:09 smithi195 ceph-mon[33773]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:09.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:09.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:09.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:09.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:09 smithi112 ceph-mon[30100]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T10:30:09.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:09 smithi112 ceph-mon[30100]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:09.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:09.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:09.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:10.572 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:10 smithi148 ceph-mon[33568]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:10.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:10 smithi195 ceph-mon[33773]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:10.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:10 smithi112 ceph-mon[30100]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:12.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:12 smithi148 ceph-mon[33568]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:12.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:12 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/579830736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]: dispatch 2024-05-10T10:30:12.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:12 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]: dispatch 2024-05-10T10:30:12.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:12 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]': finished 2024-05-10T10:30:12.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:12 smithi148 ceph-mon[33568]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T10:30:12.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:12 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:12.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:12 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/3940510867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:30:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:12 smithi195 ceph-mon[33773]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:12 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/579830736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]: dispatch 2024-05-10T10:30:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:12 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]: dispatch 2024-05-10T10:30:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:12 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]': finished 2024-05-10T10:30:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:12 smithi195 ceph-mon[33773]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T10:30:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:12 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:12 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/3940510867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:30:12.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:12 smithi112 ceph-mon[30100]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:12.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:12 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/579830736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]: dispatch 2024-05-10T10:30:12.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:12 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]: dispatch 2024-05-10T10:30:12.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:12 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de68637a-80f9-450c-9208-784f54dc3df2"}]': finished 2024-05-10T10:30:12.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:12 smithi112 ceph-mon[30100]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T10:30:12.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:12 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:12.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:12 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/3940510867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:30:14.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:14 smithi148 ceph-mon[33568]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:14.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:14 smithi195 ceph-mon[33773]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:14.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:14 smithi112 ceph-mon[30100]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:16.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:16 smithi148 ceph-mon[33568]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:16 smithi195 ceph-mon[33773]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:16 smithi112 ceph-mon[30100]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:18.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:18 smithi148 ceph-mon[33568]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:18.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:18 smithi195 ceph-mon[33773]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:18.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:18 smithi112 ceph-mon[30100]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:19.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:19 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T10:30:19.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:19 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:19.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:19 smithi148 ceph-mon[33568]: Deploying daemon osd.1 on smithi148 2024-05-10T10:30:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:19 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T10:30:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:19 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:19 smithi195 ceph-mon[33773]: Deploying daemon osd.1 on smithi148 2024-05-10T10:30:19.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:19 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T10:30:19.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:19 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:19.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:19 smithi112 ceph-mon[30100]: Deploying daemon osd.1 on smithi148 2024-05-10T10:30:20.735 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:20 smithi148 ceph-mon[33568]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:20 smithi195 ceph-mon[33773]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:20 smithi112 ceph-mon[30100]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:22.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:22 smithi148 ceph-mon[33568]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:22 smithi195 ceph-mon[33773]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:22.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:22 smithi112 ceph-mon[30100]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:24.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:24 smithi148 ceph-mon[33568]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:24.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:24.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:24.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:24 smithi195 ceph-mon[33773]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:24.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:24 smithi112 ceph-mon[30100]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:24.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:24.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:24.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:26.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:26 smithi148 ceph-mon[33568]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:26.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:26 smithi195 ceph-mon[33773]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:26.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:26 smithi112 ceph-mon[30100]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:27.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:27.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:27.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:27.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:27.915 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:27.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.986+0000 7fa8567fc700 1 -- 172.21.15.148:0/709148600 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa8600638e0 con 0x7fa84c0b2870 2024-05-10T10:30:27.988 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 1 on host 'smithi148' 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa84c0b2870 msgr2=0x7fa84c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa84c0b2870 0x7fa84c0b4d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa860102790 tx=0x7fa85c00a680 comp rx=0 tx=0).stop 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 msgr2=0x7fa860197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa860197e60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa850005910 tx=0x7fa8500047b0 comp rx=0 tx=0).stop 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 shutdown_connections 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa860107b70 0x7fa8601a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa84c0b2870 0x7fa84c0b4d30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa86006a4a0 0x7fa860197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:27.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 --2- 172.21.15.148:0/709148600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa860102e70 0x7fa860197e60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:27.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 >> 172.21.15.148:0/709148600 conn(0x7fa8600fccf0 msgr2=0x7fa860103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:30:27.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 shutdown_connections 2024-05-10T10:30:27.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:30:27.988+0000 7fa867755700 1 -- 172.21.15.148:0/709148600 wait complete. 2024-05-10T10:30:28.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:28.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:28.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:28.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:28.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:28.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:28.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:28.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:28.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:28.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:28.670 DEBUG:teuthology.orchestra.run.smithi148:osd.1> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.1.service 2024-05-10T10:30:28.672 INFO:tasks.cephadm:Deploying osd.2 on smithi195 with /dev/vg_nvme/lv_4... 2024-05-10T10:30:28.672 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T10:30:28.925 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:30:29.044 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:28 smithi195 ceph-mon[33773]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:29.045 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:29.045 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:29.045 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:29.045 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:28 smithi195 ceph-mon[33773]: from='osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T10:30:29.045 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:28 smithi195 ceph-mon[33773]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T10:30:29.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:28 smithi148 ceph-mon[33568]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:29.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:29.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:29.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:29.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:28 smithi148 ceph-mon[33568]: from='osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T10:30:29.230 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:28 smithi148 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T10:30:29.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:28 smithi112 ceph-mon[30100]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:29.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:29.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:29.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:29.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:28 smithi112 ceph-mon[30100]: from='osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T10:30:29.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:28 smithi112 ceph-mon[30100]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T10:30:30.151 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:29 smithi195 ceph-mon[33773]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T10:30:30.152 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:29 smithi195 ceph-mon[33773]: from='osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:30:30.152 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:29 smithi195 ceph-mon[33773]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T10:30:30.152 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:29 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:30.152 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:29 smithi195 ceph-mon[33773]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:30:30.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:29 smithi148 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T10:30:30.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:29 smithi148 ceph-mon[33568]: from='osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:30:30.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:29 smithi148 ceph-mon[33568]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T10:30:30.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:29 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:30.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:29 smithi148 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:30:30.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:29 smithi112 ceph-mon[30100]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T10:30:30.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:29 smithi112 ceph-mon[30100]: from='osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:30:30.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:29 smithi112 ceph-mon[30100]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T10:30:30.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:29 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:30.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:29 smithi112 ceph-mon[30100]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:30:30.714 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T10:30:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T10:30:30.715 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-10T10:30:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-10T10:30:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00882335 s, 1.2 GB/s 2024-05-10T10:30:30.715 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-10T10:30:30.961 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1[38152]: 2024-05-10T10:30:30.690+0000 7fc0945ec700 -1 osd.1 0 waiting for initial osdmap 2024-05-10T10:30:30.962 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1[38152]: 2024-05-10T10:30:30.695+0000 7fc08c3d7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:30:31.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:30 smithi195 ceph-mon[33773]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:31.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:30 smithi195 ceph-mon[33773]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:30:31.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:30 smithi195 ceph-mon[33773]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T10:30:31.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:31.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:31.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:30 smithi195 ceph-mon[33773]: from='osd.1 ' entity='osd.1' 2024-05-10T10:30:31.284 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_4 2024-05-10T10:30:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:30 smithi148 ceph-mon[33568]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:30 smithi148 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:30:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:30 smithi148 ceph-mon[33568]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T10:30:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:30 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:30 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:31.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:30 smithi148 ceph-mon[33568]: from='osd.1 ' entity='osd.1' 2024-05-10T10:30:31.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:30 smithi112 ceph-mon[30100]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:31.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:30 smithi112 ceph-mon[30100]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:30:31.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:30 smithi112 ceph-mon[30100]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T10:30:31.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:31.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:31.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:30 smithi112 ceph-mon[30100]: from='osd.1 ' entity='osd.1' 2024-05-10T10:30:31.532 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:30:32.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:30:32.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] boot 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:32.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:30:32.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:30:32.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:32.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] boot 2024-05-10T10:30:32.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T10:30:32.262 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:32.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:32.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:32.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:32.263 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:30:32.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:30:32.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:32.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: osd.1 [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] boot 2024-05-10T10:30:32.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:32.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 -- 172.21.15.195:0/1277123979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 msgr2=0x7fbc08104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 --2- 172.21.15.195:0/1277123979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08104b60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009990 tx=0x7fbbf0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 -- 172.21.15.195:0/1277123979 shutdown_connections 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 --2- 172.21.15.195:0/1277123979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081050a0 0x7fbc0810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 --2- 172.21.15.195:0/1277123979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc08103500 0x7fbc08103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 --2- 172.21.15.195:0/1277123979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08104b60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 -- 172.21.15.195:0/1277123979 >> 172.21.15.195:0/1277123979 conn(0x7fbc080feaa0 msgr2=0x7fbc08100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:30:33.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 -- 172.21.15.195:0/1277123979 shutdown_connections 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 -- 172.21.15.195:0/1277123979 wait complete. 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 Processor -- start 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.029+0000 7fbc0d904700 1 -- start start 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc0d904700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc08103500 0x7fbc08077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc0d904700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc0d904700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081050a0 0x7fbc08075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc0d904700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0810ef40 con 0x7fbc081050a0 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc0d904700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0810edc0 con 0x7fbc08103500 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc0d904700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0810f0c0 con 0x7fbc08104700 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08075890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:46906/0 (socket says 172.21.15.195:46906) 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 -- 172.21.15.195:0/646860778 learned_addr learned my addr 172.21.15.195:0/646860778 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:30:33.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc06ffd700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc08103500 0x7fbc08077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 -- 172.21.15.195:0/646860778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc08103500 msgr2=0x7fbc08077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc08103500 0x7fbc08077240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 -- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081050a0 msgr2=0x7fbc08075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc077fe700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081050a0 0x7fbc08075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081050a0 0x7fbc08075dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 -- 172.21.15.195:0/646860778 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc08076680 con 0x7fbc08104700 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc06ffd700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc08103500 0x7fbc08077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbbfe7fc700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08075890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005910 tx=0x7fbbf00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.030+0000 7fbc0c902700 1 -- 172.21.15.195:0/646860778 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0027070 con 0x7fbc08104700 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.031+0000 7fbc0c902700 1 -- 172.21.15.195:0/646860778 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbf0022470 con 0x7fbc08104700 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.031+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc081a3810 con 0x7fbc08104700 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.031+0000 7fbc0c902700 1 -- 172.21.15.195:0/646860778 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000f6c0 con 0x7fbc08104700 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.031+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc081a3ce0 con 0x7fbc08104700 2024-05-10T10:30:33.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.031+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0806a190 con 0x7fbc08104700 2024-05-10T10:30:33.035 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.033+0000 7fbc0c902700 1 -- 172.21.15.195:0/646860778 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fbbf002a070 con 0x7fbc08104700 2024-05-10T10:30:33.035 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.033+0000 7fbc0c902700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbbf40b27f0 0x7fbbf40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:33.036 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.034+0000 7fbc0c902700 1 -- 172.21.15.195:0/646860778 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fbbf00e1f80 con 0x7fbc08104700 2024-05-10T10:30:33.036 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.034+0000 7fbc06ffd700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbbf40b27f0 0x7fbbf40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:33.036 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.034+0000 7fbc06ffd700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbbf40b27f0 0x7fbbf40b4cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc08104560 tx=0x7fbbf8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:30:33.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.037+0000 7fbc0c902700 1 -- 172.21.15.195:0/646860778 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbbf0031070 con 0x7fbc08104700 2024-05-10T10:30:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:32 smithi195 ceph-mon[33773]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:32 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi148 2024-05-10T10:30:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:32 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi148 to 17088M 2024-05-10T10:30:33.252 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:33.252+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbc080638e0 con 0x7fbbf40b27f0 2024-05-10T10:30:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:32 smithi148 ceph-mon[33568]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:32 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi148 2024-05-10T10:30:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:32 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi148 to 17088M 2024-05-10T10:30:33.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:32 smithi112 ceph-mon[30100]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T10:30:33.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:32 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi148 2024-05-10T10:30:33.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:32 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi148 to 17088M 2024-05-10T10:30:34.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:33 smithi195 ceph-mon[33773]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T10:30:34.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:34.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:34.160 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:34.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:33 smithi148 ceph-mon[33568]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T10:30:34.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:34.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:34.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:34.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:33 smithi112 ceph-mon[30100]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T10:30:34.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:34.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:34.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:34 smithi195 ceph-mon[33773]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:34 smithi195 ceph-mon[33773]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:35.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:34 smithi148 ceph-mon[33568]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:35.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:34 smithi148 ceph-mon[33568]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:34 smithi112 ceph-mon[30100]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:34 smithi112 ceph-mon[30100]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:36.249 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:35 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/2129345081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]: dispatch 2024-05-10T10:30:36.249 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:35 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]: dispatch 2024-05-10T10:30:36.250 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:35 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]': finished 2024-05-10T10:30:36.250 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:35 smithi195 ceph-mon[33773]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T10:30:36.250 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:36.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:35 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/2129345081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]: dispatch 2024-05-10T10:30:36.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:35 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]: dispatch 2024-05-10T10:30:36.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:35 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]': finished 2024-05-10T10:30:36.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:35 smithi148 ceph-mon[33568]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T10:30:36.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:36.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:35 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/2129345081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]: dispatch 2024-05-10T10:30:36.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:35 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]: dispatch 2024-05-10T10:30:36.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:35 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951b8444-0377-457c-b0c4-916113883584"}]': finished 2024-05-10T10:30:36.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:35 smithi112 ceph-mon[30100]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T10:30:36.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:37.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:36 smithi148 ceph-mon[33568]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:37.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:36 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/1014498123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:30:37.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:36 smithi195 ceph-mon[33773]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:37.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:36 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/1014498123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:30:37.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:36 smithi112 ceph-mon[30100]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:37.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:36 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/1014498123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:30:39.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:38 smithi148 ceph-mon[33568]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:39.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:38 smithi195 ceph-mon[33773]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:39.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:38 smithi112 ceph-mon[30100]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:41.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:40 smithi195 ceph-mon[33773]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:41.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:40 smithi148 ceph-mon[33568]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:41.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:40 smithi112 ceph-mon[30100]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:43 smithi195 ceph-mon[33773]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T10:30:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:43 smithi148 ceph-mon[33568]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:43.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T10:30:43.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:43 smithi112 ceph-mon[30100]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T10:30:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:44.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:44 smithi148 ceph-mon[33568]: Deploying daemon osd.2 on smithi195 2024-05-10T10:30:44.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:44 smithi195 ceph-mon[33773]: Deploying daemon osd.2 on smithi195 2024-05-10T10:30:44.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:44 smithi112 ceph-mon[30100]: Deploying daemon osd.2 on smithi195 2024-05-10T10:30:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:45 smithi148 ceph-mon[33568]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:45 smithi195 ceph-mon[33773]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:45.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:45 smithi112 ceph-mon[30100]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:47.149 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:47 smithi195 ceph-mon[33773]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:47 smithi148 ceph-mon[33568]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:47.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:47 smithi112 ceph-mon[30100]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:48 smithi195 ceph-mon[33773]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:48 smithi148 ceph-mon[33568]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:48 smithi112 ceph-mon[30100]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:50 smithi195 ceph-mon[33773]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:51.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:50 smithi148 ceph-mon[33568]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:51.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:51.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:51.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:50 smithi112 ceph-mon[30100]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:51.416 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 2 on host 'smithi195' 2024-05-10T10:30:51.416 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.414+0000 7fbc0c902700 1 -- 172.21.15.195:0/646860778 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbc080638e0 con 0x7fbbf40b27f0 2024-05-10T10:30:51.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbbf40b27f0 msgr2=0x7fbbf40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:51.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbbf40b27f0 0x7fbbf40b4cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc08104560 tx=0x7fbbf8008040 comp rx=0 tx=0).stop 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 msgr2=0x7fbc08075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08075890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005910 tx=0x7fbbf00047b0 comp rx=0 tx=0).stop 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 shutdown_connections 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081050a0 0x7fbc08075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc08103500 0x7fbc08077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbbf40b27f0 0x7fbbf40b4cb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 --2- 172.21.15.195:0/646860778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc08104700 0x7fbc08075890 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 >> 172.21.15.195:0/646860778 conn(0x7fbc080feaa0 msgr2=0x7fbc08107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 shutdown_connections 2024-05-10T10:30:51.419 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:30:51.417+0000 7fbc0d904700 1 -- 172.21.15.195:0/646860778 wait complete. 2024-05-10T10:30:52.062 DEBUG:teuthology.orchestra.run.smithi195:osd.2> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.2.service 2024-05-10T10:30:52.065 INFO:tasks.cephadm:Deploying osd.3 on smithi112 with /dev/vg_nvme/lv_3... 2024-05-10T10:30:52.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T10:30:52.316 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:30:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:52 smithi195 ceph-mon[33773]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:52 smithi195 ceph-mon[33773]: from='osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T10:30:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:52 smithi195 ceph-mon[33773]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T10:30:52.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:52 smithi112 ceph-mon[30100]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:52.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:52.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:52.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:52.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:52 smithi112 ceph-mon[30100]: from='osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T10:30:52.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:52 smithi112 ceph-mon[30100]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T10:30:52.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:52 smithi148 ceph-mon[33568]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:52.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:52.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:52.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:30:52.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:52 smithi148 ceph-mon[33568]: from='osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T10:30:52.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:52 smithi148 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T10:30:53.489 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:53 smithi195 ceph-mon[33773]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T10:30:53.489 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:53 smithi195 ceph-mon[33773]: from='osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:30:53.490 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:53 smithi195 ceph-mon[33773]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T10:30:53.490 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:53.490 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:53 smithi195 ceph-mon[33773]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:30:53.490 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2[38935]: 2024-05-10T10:30:53.411+0000 7fbbe5d0d700 -1 osd.2 0 waiting for initial osdmap 2024-05-10T10:30:53.490 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2[38935]: 2024-05-10T10:30:53.416+0000 7fbbe1b00700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:30:53.740 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:53 smithi112 ceph-mon[30100]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T10:30:53.740 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:53 smithi112 ceph-mon[30100]: from='osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:30:53.740 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:53 smithi112 ceph-mon[30100]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T10:30:53.740 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:53.740 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:53 smithi112 ceph-mon[30100]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:30:53.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:53 smithi148 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T10:30:53.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:53 smithi148 ceph-mon[33568]: from='osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:30:53.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:53 smithi148 ceph-mon[33568]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T10:30:53.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:53.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:53 smithi148 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:30:54.202 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T10:30:54.202 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T10:30:54.202 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-10T10:30:54.203 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-10T10:30:54.203 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116089 s, 903 MB/s 2024-05-10T10:30:54.203 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 2024-05-10T10:30:54.203 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-10T10:30:54.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:54 smithi112 ceph-mon[30100]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:54.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:54 smithi112 ceph-mon[30100]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:30:54.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:54 smithi112 ceph-mon[30100]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T10:30:54.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:54 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:54.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:54 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:54.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:54 smithi195 ceph-mon[33773]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:54.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:54 smithi195 ceph-mon[33773]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:30:54.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:54 smithi195 ceph-mon[33773]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T10:30:54.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:54 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:54.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:54 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:54 smithi148 ceph-mon[33568]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:54 smithi148 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:30:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:54 smithi148 ceph-mon[33568]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T10:30:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:54 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:54.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:54 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:54.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-05-10T10:30:55.131 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:30:55.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:30:55.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] boot 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi195 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi195 to 17088M 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:55.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:30:55.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] boot 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi195 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi195 to 17088M 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:30:55.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:30:55.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: osd.2 [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] boot 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi195 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi195 to 17088M 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:55.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:30:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 -- 172.21.15.112:0/1841152238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d4103500 msgr2=0x7f81d4103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/1841152238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d4103500 0x7f81d4103920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f81c40099c0 tx=0x7f81c4009cd0 comp rx=0 tx=0).stop 2024-05-10T10:30:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 -- 172.21.15.112:0/1841152238 shutdown_connections 2024-05-10T10:30:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/1841152238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81d41050a0 0x7f81d410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/1841152238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d4103500 0x7f81d4103920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/1841152238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81d4104700 0x7f81d4104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 -- 172.21.15.112:0/1841152238 >> 172.21.15.112:0/1841152238 conn(0x7f81d40feaa0 msgr2=0x7f81d4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:30:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 -- 172.21.15.112:0/1841152238 shutdown_connections 2024-05-10T10:30:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.659+0000 7f81d9d3a700 1 -- 172.21.15.112:0/1841152238 wait complete. 2024-05-10T10:30:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 Processor -- start 2024-05-10T10:30:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 -- start start 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81d4103500 0x7f81d4077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81d4104700 0x7f81d4075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d41050a0 0x7f81d4075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81d410ee60 con 0x7f81d41050a0 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81d410ece0 con 0x7f81d4104700 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.660+0000 7f81d9d3a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81d410efe0 con 0x7f81d4103500 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d41050a0 0x7f81d4075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81d4104700 0x7f81d4075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d41050a0 0x7f81d4075dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40722/0 (socket says 172.21.15.112:40722) 2024-05-10T10:30:56.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 -- 172.21.15.112:0/2632965044 learned_addr learned my addr 172.21.15.112:0/2632965044 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d37fe700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81d4103500 0x7f81d4077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 -- 172.21.15.112:0/2632965044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81d4103500 msgr2=0x7f81d4077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81d4103500 0x7f81d4077240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 -- 172.21.15.112:0/2632965044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81d4104700 msgr2=0x7f81d4075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81d4104700 0x7f81d4075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 -- 172.21.15.112:0/2632965044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d4076680 con 0x7f81d41050a0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d3fff700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d41050a0 0x7f81d4075dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007ee0 tx=0x7f81cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc004bb0 con 0x7f81d41050a0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81cc004d50 con 0x7f81d41050a0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d41a3780 con 0x7f81d41050a0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0058b0 con 0x7f81d41050a0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.661+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d41a3c40 con 0x7f81d41050a0 2024-05-10T10:30:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.662+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d406a190 con 0x7f81d41050a0 2024-05-10T10:30:56.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.664+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f81cc01a020 con 0x7f81d41050a0 2024-05-10T10:30:56.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.665+0000 7f81d0ff9700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f81bc0b27f0 0x7f81bc0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:30:56.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.665+0000 7f81d37fe700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f81bc0b27f0 0x7f81bc0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:30:56.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.665+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f81cc0e0b00 con 0x7f81d41050a0 2024-05-10T10:30:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.665+0000 7f81d37fe700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f81bc0b27f0 0x7f81bc0b4cb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81c400b580 tx=0x7f81c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:30:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.669+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f81cc010070 con 0x7f81d41050a0 2024-05-10T10:30:56.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:56 smithi148 ceph-mon[33568]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:56.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:56 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T10:30:56.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:56 smithi148 ceph-mon[33568]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T10:30:56.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:56 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:30:56.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:56 smithi195 ceph-mon[33773]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:56.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:56 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T10:30:56.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:56 smithi195 ceph-mon[33773]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T10:30:56.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:56 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:30:56.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:56 smithi112 ceph-mon[30100]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T10:30:56.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:56 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T10:30:56.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:56 smithi112 ceph-mon[30100]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T10:30:56.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:56 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:30:56.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:56.888+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f81d40638e0 con 0x7f81bc0b27f0 2024-05-10T10:30:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T10:30:57.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42100]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42103]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T10:30:57.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42103]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42103]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 sudo[42103]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.296 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42094]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T10:30:57.296 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42094]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.296 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42094]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.296 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42094]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.297 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T10:30:57.297 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.297 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42097]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.297 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:30:56 smithi148 sudo[42097]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.316 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42934]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T10:30:57.316 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42934]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.316 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42934]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.316 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42934]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.316 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42937]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T10:30:57.317 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42937]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.317 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42937]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.317 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42937]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42940]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42940]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42940]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42940]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42943]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 sudo[42943]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47071]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47071]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47071]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47071]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47091]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.330 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:30:56 smithi112 sudo[47091]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47144]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T10:30:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47144]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47144]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47147]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T10:30:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47147]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T10:30:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47147]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T10:30:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 sudo[47147]: pam_unix(sudo:session): session closed for user root 2024-05-10T10:30:57.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T10:30:57.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T10:30:57.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:57.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.692 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:57 smithi112 ceph-mon[30100]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:57 smithi148 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T10:30:57.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T10:30:57.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:30:57.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:30:57.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:30:57.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:57.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T10:30:57.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T10:30:57.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T10:30:57.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T10:30:57.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:57 smithi195 ceph-mon[33773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T10:30:58.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:30:58.087+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f81cc0a57c0 con 0x7f81d41050a0 2024-05-10T10:30:58.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:58 smithi112 ceph-mon[30100]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:30:58.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:58 smithi112 ceph-mon[30100]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T10:30:58.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:30:58 smithi112 ceph-mon[30100]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T10:30:58.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:58 smithi148 ceph-mon[33568]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:30:58.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:58 smithi148 ceph-mon[33568]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T10:30:58.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:30:58 smithi148 ceph-mon[33568]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T10:30:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:58 smithi195 ceph-mon[33773]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:30:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:58 smithi195 ceph-mon[33773]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T10:30:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:30:58 smithi195 ceph-mon[33773]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T10:31:00.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:00 smithi148 ceph-mon[33568]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:00.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:00 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3251574104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c27d781-3ab0-4739-836d-c4016bb1e0a3"}]: dispatch 2024-05-10T10:31:00.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:00 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3251574104' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c27d781-3ab0-4739-836d-c4016bb1e0a3"}]': finished 2024-05-10T10:31:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:00 smithi148 ceph-mon[33568]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T10:31:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:00 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:00 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2070103251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:00 smithi195 ceph-mon[33773]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:00 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3251574104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c27d781-3ab0-4739-836d-c4016bb1e0a3"}]: dispatch 2024-05-10T10:31:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:00 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3251574104' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c27d781-3ab0-4739-836d-c4016bb1e0a3"}]': finished 2024-05-10T10:31:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:00 smithi195 ceph-mon[33773]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T10:31:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:00 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:00 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2070103251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:00.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:00 smithi112 ceph-mon[30100]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:00.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:00 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3251574104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c27d781-3ab0-4739-836d-c4016bb1e0a3"}]: dispatch 2024-05-10T10:31:00.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:00 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3251574104' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c27d781-3ab0-4739-836d-c4016bb1e0a3"}]': finished 2024-05-10T10:31:00.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:00 smithi112 ceph-mon[30100]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T10:31:00.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:00.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:00 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2070103251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:02.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:02 smithi148 ceph-mon[33568]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:02.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:02 smithi195 ceph-mon[33773]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:02.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:02 smithi112 ceph-mon[30100]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:04.768 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:04 smithi112 ceph-mon[30100]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-05-10T10:31:04.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:04 smithi148 ceph-mon[33568]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-05-10T10:31:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:04 smithi195 ceph-mon[33773]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-05-10T10:31:06.791 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:06 smithi112 ceph-mon[30100]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2024-05-10T10:31:06.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:06 smithi148 ceph-mon[33568]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2024-05-10T10:31:06.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:06 smithi195 ceph-mon[33773]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2024-05-10T10:31:07.713 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T10:31:07.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:07.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T10:31:07.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T10:31:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:08.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:08 smithi148 ceph-mon[33568]: Deploying daemon osd.3 on smithi112 2024-05-10T10:31:08.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:08 smithi148 ceph-mon[33568]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 134 KiB/s wr, 9 op/s 2024-05-10T10:31:08.809 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:08 smithi112 ceph-mon[30100]: Deploying daemon osd.3 on smithi112 2024-05-10T10:31:08.809 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:08 smithi112 ceph-mon[30100]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 134 KiB/s wr, 9 op/s 2024-05-10T10:31:08.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:08 smithi195 ceph-mon[33773]: Deploying daemon osd.3 on smithi112 2024-05-10T10:31:08.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:08 smithi195 ceph-mon[33773]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 134 KiB/s wr, 9 op/s 2024-05-10T10:31:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:10 smithi112 ceph-mon[30100]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-10T10:31:10.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:10 smithi148 ceph-mon[33568]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-10T10:31:10.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:10 smithi195 ceph-mon[33773]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-10T10:31:12.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:12 smithi148 ceph-mon[33568]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-05-10T10:31:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:12 smithi195 ceph-mon[33773]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-05-10T10:31:12.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:12 smithi112 ceph-mon[30100]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-05-10T10:31:13.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:13 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:13.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:13 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:13.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:13 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:13.900 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:13 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:13.901 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:13 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:13.901 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:13 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:13.962 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:13 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:13.963 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:13 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:13.963 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:13 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:14.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:14 smithi112 ceph-mon[30100]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-10T10:31:14.844 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:14 smithi195 ceph-mon[33773]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-10T10:31:14.896 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:14 smithi148 ceph-mon[33568]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-10T10:31:15.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.843+0000 7f81d0ff9700 1 -- 172.21.15.112:0/2632965044 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81d40638e0 con 0x7f81bc0b27f0 2024-05-10T10:31:15.846 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 3 on host 'smithi112' 2024-05-10T10:31:15.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f81bc0b27f0 msgr2=0x7f81bc0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:15.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f81bc0b27f0 0x7f81bc0b4cb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81c400b580 tx=0x7f81c4005fb0 comp rx=0 tx=0).stop 2024-05-10T10:31:15.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d41050a0 msgr2=0x7f81d4075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:15.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d41050a0 0x7f81d4075dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f81cc007ee0 tx=0x7f81cc00d3b0 comp rx=0 tx=0).stop 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 shutdown_connections 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d41050a0 0x7f81d4075dd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f81bc0b27f0 0x7f81bc0b4cb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81d4103500 0x7f81d4077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 --2- 172.21.15.112:0/2632965044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81d4104700 0x7f81d4075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 >> 172.21.15.112:0/2632965044 conn(0x7f81d40feaa0 msgr2=0x7f81d4107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 shutdown_connections 2024-05-10T10:31:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:31:15.846+0000 7f81d9d3a700 1 -- 172.21.15.112:0/2632965044 wait complete. 2024-05-10T10:31:16.520 DEBUG:teuthology.orchestra.run.smithi112:osd.3> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.3.service 2024-05-10T10:31:16.522 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2024-05-10T10:31:16.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T10:31:16.772 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:31:17.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:16 smithi112 ceph-mon[30100]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:17.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:16 smithi112 ceph-mon[30100]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T10:31:17.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:16 smithi148 ceph-mon[33568]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:17.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.132 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:16 smithi148 ceph-mon[33568]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T10:31:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:16 smithi195 ceph-mon[33773]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:16 smithi195 ceph-mon[33773]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T10:31:17.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:17 smithi112 ceph-mon[30100]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T10:31:17.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:17 smithi112 ceph-mon[30100]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T10:31:17.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:17 smithi112 ceph-mon[30100]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:31:17.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:17 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:18.158 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:17 smithi148 ceph-mon[33568]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T10:31:18.159 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:17 smithi148 ceph-mon[33568]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T10:31:18.159 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:17 smithi148 ceph-mon[33568]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:31:18.159 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:17 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:18.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:17 smithi195 ceph-mon[33773]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T10:31:18.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:17 smithi195 ceph-mon[33773]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T10:31:18.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:17 smithi195 ceph-mon[33773]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:31:18.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:17 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:18.816 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:31:18 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3[49336]: 2024-05-10T10:31:18.547+0000 7fddfdda2700 -1 osd.3 0 waiting for initial osdmap 2024-05-10T10:31:18.816 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:31:18 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3[49336]: 2024-05-10T10:31:18.551+0000 7fddf5b8d700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:31:18.866 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T10:31:18.867 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T10:31:18.867 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-05-10T10:31:18.867 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-05-10T10:31:18.867 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117829 s, 890 MB/s 2024-05-10T10:31:18.867 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-05-10T10:31:19.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] boot 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T10:31:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.209 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:19.209 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:31:19.209 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T10:31:19.209 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.209 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' 2024-05-10T10:31:19.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] boot 2024-05-10T10:31:19.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T10:31:19.210 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T10:31:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:31:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T10:31:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: from='osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850]' entity='osd.3' 2024-05-10T10:31:19.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: osd.3 [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] boot 2024-05-10T10:31:19.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T10:31:19.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T10:31:19.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-05-10T10:31:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi112 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi112 to 8544M 2024-05-10T10:31:20.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:20.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:20.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:20.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:20.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:20.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:31:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:31:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T10:31:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi112 2024-05-10T10:31:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi112 to 8544M 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:20.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:31:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:31:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T10:31:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi112 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi112 to 8544M 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:20.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:21.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 -- 172.21.15.148:0/114011927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 msgr2=0x7f13a4102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/114011927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a4102830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f139400b0d0 tx=0x7f139400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 -- 172.21.15.148:0/114011927 shutdown_connections 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/114011927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13a4103fb0 0x7f13a410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/114011927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a4102830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/114011927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13a4103610 0x7f13a4103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 -- 172.21.15.148:0/114011927 >> 172.21.15.148:0/114011927 conn(0x7f13a40fd9d0 msgr2=0x7f13a40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 -- 172.21.15.148:0/114011927 shutdown_connections 2024-05-10T10:31:21.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 -- 172.21.15.148:0/114011927 wait complete. 2024-05-10T10:31:21.495 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:21 smithi148 ceph-mon[33568]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T10:31:21.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.493+0000 7f13aa0b0700 1 Processor -- start 2024-05-10T10:31:21.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13aa0b0700 1 -- start start 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13aa0b0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a406c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13aa0b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13a4103610 0x7f13a419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13aa0b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13a4103fb0 0x7f13a41a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13aa0b0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13a410ddb0 con 0x7f13a4103610 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13aa0b0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13a410dc30 con 0x7f13a4102410 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13aa0b0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13a410df30 con 0x7f13a4103fb0 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55238/0 (socket says 172.21.15.148:55238) 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 -- 172.21.15.148:0/25860900 learned_addr learned my addr 172.21.15.148:0/25860900 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a2ffd700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13a4103610 0x7f13a419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a3fff700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13a4103fb0 0x7f13a41a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:21.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 -- 172.21.15.148:0/25860900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13a4103fb0 msgr2=0x7f13a41a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13a4103fb0 0x7f13a41a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 -- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13a4103610 msgr2=0x7f13a419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13a4103610 0x7f13a419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.494+0000 7f13a37fe700 1 -- 172.21.15.148:0/25860900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a41a12c0 con 0x7f13a4102410 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.495+0000 7f13a37fe700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a406c700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1394009010 tx=0x7f1394009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.495+0000 7f13a0ff9700 1 -- 172.21.15.148:0/25860900 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139401f070 con 0x7f13a4102410 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.495+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a41a1550 con 0x7f13a4102410 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.495+0000 7f13a0ff9700 1 -- 172.21.15.148:0/25860900 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1394007c40 con 0x7f13a4102410 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.495+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41a1a80 con 0x7f13a4102410 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.495+0000 7f13a0ff9700 1 -- 172.21.15.148:0/25860900 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139401b690 con 0x7f13a4102410 2024-05-10T10:31:21.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.496+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a406a190 con 0x7f13a4102410 2024-05-10T10:31:21.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.497+0000 7f13a0ff9700 1 -- 172.21.15.148:0/25860900 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1394022070 con 0x7f13a4102410 2024-05-10T10:31:21.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.498+0000 7f13a0ff9700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f138c0b2900 0x7f138c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:21.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.499+0000 7f13a0ff9700 1 -- 172.21.15.148:0/25860900 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f139400f020 con 0x7f13a4102410 2024-05-10T10:31:21.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.499+0000 7f13a2ffd700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f138c0b2900 0x7f138c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:21.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.499+0000 7f13a2ffd700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f138c0b2900 0x7f138c0b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1398009a90 tx=0x7f1398009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:31:21.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.502+0000 7f13a0ff9700 1 -- 172.21.15.148:0/25860900 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1394029020 con 0x7f13a4102410 2024-05-10T10:31:21.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:21.712+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f13a40638e0 con 0x7f138c0b2900 2024-05-10T10:31:21.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:21 smithi195 ceph-mon[33773]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T10:31:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:21 smithi112 ceph-mon[30100]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T10:31:22.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:22 smithi148 ceph-mon[33568]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:22.659 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:22 smithi148 ceph-mon[33568]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:31:22.659 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:31:22.659 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:31:22.659 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:22.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:22 smithi112 ceph-mon[30100]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:22.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:22 smithi112 ceph-mon[30100]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:31:22.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:31:22.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:31:22.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:22 smithi195 ceph-mon[33773]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:22 smithi195 ceph-mon[33773]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:31:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:31:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:31:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:24.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:24 smithi148 ceph-mon[33568]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-10T10:31:24.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:24 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/3252444537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]: dispatch 2024-05-10T10:31:24.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:24 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]: dispatch 2024-05-10T10:31:24.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:24 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]': finished 2024-05-10T10:31:24.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:24 smithi148 ceph-mon[33568]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T10:31:24.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:24 smithi195 ceph-mon[33773]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-10T10:31:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:24 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/3252444537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]: dispatch 2024-05-10T10:31:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:24 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]: dispatch 2024-05-10T10:31:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:24 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]': finished 2024-05-10T10:31:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:24 smithi195 ceph-mon[33773]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T10:31:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:24.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:24 smithi112 ceph-mon[30100]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-10T10:31:24.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:24 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/3252444537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]: dispatch 2024-05-10T10:31:24.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:24 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]: dispatch 2024-05-10T10:31:24.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:24 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8781c4-9d04-449c-a8df-45568364e555"}]': finished 2024-05-10T10:31:24.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:24 smithi112 ceph-mon[30100]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T10:31:24.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:25.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:25 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/872975394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:25.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:25 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/872975394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:25.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:25 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/872975394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:26.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:26 smithi148 ceph-mon[33568]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:31:26.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:26 smithi195 ceph-mon[33773]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:31:26.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:26 smithi112 ceph-mon[30100]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:31:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:28 smithi148 ceph-mon[33568]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:31:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:28 smithi195 ceph-mon[33773]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:31:28.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:28 smithi112 ceph-mon[30100]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:31:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:30 smithi148 ceph-mon[33568]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:31:30.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:30 smithi195 ceph-mon[33773]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:31:30.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:30 smithi112 ceph-mon[30100]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:31:32.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:32 smithi148 ceph-mon[33568]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:31:32.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T10:31:32.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:32.771 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:32 smithi148 ceph-mon[33568]: Deploying daemon osd.4 on smithi148 2024-05-10T10:31:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:32 smithi195 ceph-mon[33773]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:31:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T10:31:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:32 smithi195 ceph-mon[33773]: Deploying daemon osd.4 on smithi148 2024-05-10T10:31:32.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:32 smithi112 ceph-mon[30100]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:31:32.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T10:31:32.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:32.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:32 smithi112 ceph-mon[30100]: Deploying daemon osd.4 on smithi148 2024-05-10T10:31:34.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:34 smithi148 ceph-mon[33568]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:34 smithi195 ceph-mon[33773]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:34.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:34 smithi112 ceph-mon[30100]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:36.747 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:36 smithi148 ceph-mon[33568]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:36 smithi195 ceph-mon[33773]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:36 smithi112 ceph-mon[30100]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:38 smithi148 ceph-mon[33568]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:39.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:39.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:38 smithi195 ceph-mon[33773]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:39.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:38 smithi112 ceph-mon[30100]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:41.018 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:40 smithi148 ceph-mon[33568]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:40 smithi195 ceph-mon[33773]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:41.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:40 smithi112 ceph-mon[30100]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:41.083 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-05-10T10:31:41.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.082+0000 7f13a0ff9700 1 -- 172.21.15.148:0/25860900 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f13a40638e0 con 0x7f138c0b2900 2024-05-10T10:31:41.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f138c0b2900 msgr2=0x7f138c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:41.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f138c0b2900 0x7f138c0b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1398009a90 tx=0x7f1398009190 comp rx=0 tx=0).stop 2024-05-10T10:31:41.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 msgr2=0x7f13a406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:41.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a406c700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1394009010 tx=0x7f1394009b80 comp rx=0 tx=0).stop 2024-05-10T10:31:41.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 shutdown_connections 2024-05-10T10:31:41.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f138c0b2900 0x7f138c0b4dc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:41.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13a4103fb0 0x7f13a41a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:41.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13a4102410 0x7f13a406c700 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:41.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 --2- 172.21.15.148:0/25860900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13a4103610 0x7f13a419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:41.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 >> 172.21.15.148:0/25860900 conn(0x7f13a40fd9d0 msgr2=0x7f13a4106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:31:41.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 shutdown_connections 2024-05-10T10:31:41.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:31:41.085+0000 7f13aa0b0700 1 -- 172.21.15.148:0/25860900 wait complete. 2024-05-10T10:31:42.118 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.4.service 2024-05-10T10:31:42.120 INFO:tasks.cephadm:Deploying osd.5 on smithi195 with /dev/vg_nvme/lv_3... 2024-05-10T10:31:42.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T10:31:42.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:42.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:42.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:42 smithi195 ceph-mon[33773]: from='osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T10:31:42.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:42 smithi195 ceph-mon[33773]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T10:31:42.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:42.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:42.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:42 smithi112 ceph-mon[30100]: from='osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T10:31:42.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:42 smithi112 ceph-mon[30100]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T10:31:42.372 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:31:42.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:42.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:42.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:42 smithi148 ceph-mon[33568]: from='osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T10:31:42.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:42 smithi148 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T10:31:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T10:31:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T10:31:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: from='osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:31:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:31:43.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:43.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:43.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T10:31:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T10:31:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: from='osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:31:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:31:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:43.345 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:43.345 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T10:31:43.345 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T10:31:43.345 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:43.345 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: from='osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:31:43.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:31:43.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:43.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:43.346 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:31:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4[44919]: 2024-05-10T10:31:43.068+0000 7fe5acf1b700 -1 osd.4 0 waiting for initial osdmap 2024-05-10T10:31:43.346 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:31:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4[44919]: 2024-05-10T10:31:43.073+0000 7fe5a8d0e700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:31:44.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:44 smithi195 ceph-mon[33773]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:31:44.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:44 smithi195 ceph-mon[33773]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T10:31:44.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:44.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:44 smithi112 ceph-mon[30100]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:31:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:44 smithi112 ceph-mon[30100]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T10:31:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:44.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:44 smithi148 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:31:44.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:44 smithi148 ceph-mon[33568]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T10:31:44.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:44.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:44.364 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T10:31:44.365 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T10:31:44.365 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-10T10:31:44.365 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-10T10:31:44.365 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852067 s, 1.2 GB/s 2024-05-10T10:31:44.365 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-10T10:31:45.018 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_3 2024-05-10T10:31:45.237 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:45 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:31:45.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:45 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:31:45.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:45 smithi148 ceph-mon[33568]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:45.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:45 smithi148 ceph-mon[33568]: osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] boot 2024-05-10T10:31:45.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:45 smithi148 ceph-mon[33568]: osdmap e31: 5 total, 5 up, 5 in 2024-05-10T10:31:45.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:45 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:45.257 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:31:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:45 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:31:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:45 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:31:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:45 smithi195 ceph-mon[33773]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:45 smithi195 ceph-mon[33773]: osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] boot 2024-05-10T10:31:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:45 smithi195 ceph-mon[33773]: osdmap e31: 5 total, 5 up, 5 in 2024-05-10T10:31:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:45 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:45.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:45 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:31:45.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:45 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:31:45.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:45 smithi112 ceph-mon[30100]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-10T10:31:45.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:45 smithi112 ceph-mon[30100]: osd.4 [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] boot 2024-05-10T10:31:45.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:45 smithi112 ceph-mon[30100]: osdmap e31: 5 total, 5 up, 5 in 2024-05-10T10:31:45.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T10:31:46.296 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T10:31:46.296 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.296 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:46.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:31:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:46.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:31:46.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:31:46.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.819+0000 7f531c091700 1 -- 172.21.15.195:0/2584171456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314103600 msgr2=0x7f5314103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:46.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.819+0000 7f531c091700 1 --2- 172.21.15.195:0/2584171456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314103600 0x7f5314103a60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5304009990 tx=0x7f5304009ca0 comp rx=0 tx=0).stop 2024-05-10T10:31:46.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.819+0000 7f531c091700 1 -- 172.21.15.195:0/2584171456 shutdown_connections 2024-05-10T10:31:46.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.819+0000 7f531c091700 1 --2- 172.21.15.195:0/2584171456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5314103fa0 0x7f531410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:46.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.819+0000 7f531c091700 1 --2- 172.21.15.195:0/2584171456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5314102400 0x7f5314102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.819+0000 7f531c091700 1 --2- 172.21.15.195:0/2584171456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314103600 0x7f5314103a60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.819+0000 7f531c091700 1 -- 172.21.15.195:0/2584171456 >> 172.21.15.195:0/2584171456 conn(0x7f53140fd9c0 msgr2=0x7f53140ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 -- 172.21.15.195:0/2584171456 shutdown_connections 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 -- 172.21.15.195:0/2584171456 wait complete. 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 Processor -- start 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 -- start start 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314102400 0x7f5314198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5314103600 0x7f5314198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5314103fa0 0x7f531419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f531410dd30 con 0x7f5314103600 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f531410dbb0 con 0x7f5314103fa0 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f531c091700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f531410deb0 con 0x7f5314102400 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f5319e2d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314102400 0x7f5314198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:46.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.820+0000 7f5319e2d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314102400 0x7f5314198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:53626/0 (socket says 172.21.15.195:53626) 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f5319e2d700 1 -- 172.21.15.195:0/2853685540 learned_addr learned my addr 172.21.15.195:0/2853685540 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f531a62e700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5314103fa0 0x7f531419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f531962c700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5314103600 0x7f5314198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f5319e2d700 1 -- 172.21.15.195:0/2853685540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5314103fa0 msgr2=0x7f531419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f5319e2d700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5314103fa0 0x7f531419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f5319e2d700 1 -- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5314103600 msgr2=0x7f5314198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f5319e2d700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5314103600 0x7f5314198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f5319e2d700 1 -- 172.21.15.195:0/2853685540 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53141a0040 con 0x7f5314102400 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f5319e2d700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314102400 0x7f5314198430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5310007b80 tx=0x7f5310007e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f530affd700 1 -- 172.21.15.195:0/2853685540 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531001a070 con 0x7f5314102400 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f530affd700 1 -- 172.21.15.195:0/2853685540 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53100044c0 con 0x7f5314102400 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531406bc70 con 0x7f5314102400 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f530affd700 1 -- 172.21.15.195:0/2853685540 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531000e540 con 0x7f5314102400 2024-05-10T10:31:46.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.821+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531406c1b0 con 0x7f5314102400 2024-05-10T10:31:46.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.822+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531406a190 con 0x7f5314102400 2024-05-10T10:31:46.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.824+0000 7f530affd700 1 -- 172.21.15.195:0/2853685540 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f531001d070 con 0x7f5314102400 2024-05-10T10:31:46.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.825+0000 7f530affd700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53000b2880 0x7f53000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:31:46.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.825+0000 7f530affd700 1 -- 172.21.15.195:0/2853685540 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f53100e2220 con 0x7f5314102400 2024-05-10T10:31:46.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.825+0000 7f531962c700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53000b2880 0x7f53000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:31:46.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.826+0000 7f531962c700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53000b2880 0x7f53000b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f530400b5c0 tx=0x7f5304005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:31:46.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:46.829+0000 7f530affd700 1 -- 172.21.15.195:0/2853685540 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f53100aafb0 con 0x7f5314102400 2024-05-10T10:31:47.040 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:31:47.039+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f531406c410 con 0x7f53000b2880 2024-05-10T10:31:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:47 smithi195 ceph-mon[33773]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:47 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi148 2024-05-10T10:31:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:47 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi148 to 8544M 2024-05-10T10:31:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:31:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:31:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:47 smithi148 ceph-mon[33568]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:47 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi148 2024-05-10T10:31:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:47 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi148 to 8544M 2024-05-10T10:31:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:31:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:31:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:47.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:47 smithi112 ceph-mon[30100]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:47.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:47 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi148 2024-05-10T10:31:47.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:47 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi148 to 8544M 2024-05-10T10:31:47.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:31:47.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:31:47.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:48.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:48 smithi195 ceph-mon[33773]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:31:48.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:48 smithi148 ceph-mon[33568]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:31:48.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:48 smithi112 ceph-mon[30100]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:31:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:49 smithi195 ceph-mon[33773]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:49.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:49 smithi148 ceph-mon[33568]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:49.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:49 smithi112 ceph-mon[30100]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:50.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:50 smithi148 ceph-mon[33568]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:50.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:50 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/3228991842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]: dispatch 2024-05-10T10:31:50.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:50 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]: dispatch 2024-05-10T10:31:50.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:50 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]': finished 2024-05-10T10:31:50.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:50 smithi148 ceph-mon[33568]: osdmap e33: 6 total, 5 up, 6 in 2024-05-10T10:31:50.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:31:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:50 smithi195 ceph-mon[33773]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:50 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/3228991842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]: dispatch 2024-05-10T10:31:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:50 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]: dispatch 2024-05-10T10:31:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:50 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]': finished 2024-05-10T10:31:50.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:50 smithi195 ceph-mon[33773]: osdmap e33: 6 total, 5 up, 6 in 2024-05-10T10:31:50.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:31:50.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:50 smithi112 ceph-mon[30100]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:50.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:50 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/3228991842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]: dispatch 2024-05-10T10:31:50.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:50 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]: dispatch 2024-05-10T10:31:50.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:50 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c65aefb7-ce11-4194-b1f4-ab1e323102cb"}]': finished 2024-05-10T10:31:50.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:50 smithi112 ceph-mon[30100]: osdmap e33: 6 total, 5 up, 6 in 2024-05-10T10:31:50.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:31:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:51 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/694364645' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:51.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:51 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/694364645' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:51.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:51 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/694364645' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:31:52.481 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:52 smithi112 ceph-mon[30100]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:52.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:52 smithi148 ceph-mon[33568]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:52 smithi195 ceph-mon[33773]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:54.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:54 smithi148 ceph-mon[33568]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:54.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:54 smithi195 ceph-mon[33773]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:54.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:54 smithi112 ceph-mon[30100]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:56.412 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:56 smithi195 ceph-mon[33773]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:56.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:56 smithi148 ceph-mon[33568]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:56.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:56 smithi112 ceph-mon[30100]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T10:31:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:57 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:57.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T10:31:57.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:57 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:57.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T10:31:57.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:57 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:31:58.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:58 smithi195 ceph-mon[33773]: Deploying daemon osd.5 on smithi195 2024-05-10T10:31:58.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:31:58 smithi195 ceph-mon[33773]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:58.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:58 smithi148 ceph-mon[33568]: Deploying daemon osd.5 on smithi195 2024-05-10T10:31:58.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:31:58 smithi148 ceph-mon[33568]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:31:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:58 smithi112 ceph-mon[30100]: Deploying daemon osd.5 on smithi195 2024-05-10T10:31:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:31:58 smithi112 ceph-mon[30100]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:00.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:00 smithi148 ceph-mon[33568]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:00 smithi195 ceph-mon[33773]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:00.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:00 smithi112 ceph-mon[30100]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:02.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:02 smithi195 ceph-mon[33773]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:02.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:02 smithi148 ceph-mon[33568]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:02.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:02 smithi112 ceph-mon[30100]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:04.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:03 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:04.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:03 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:04.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:03 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:04.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:03 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:04.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:03 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:04.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:03 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:04.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:04.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:04.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:03 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:04.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:04 smithi195 ceph-mon[33773]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:04 smithi148 ceph-mon[33568]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:05.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:04 smithi112 ceph-mon[30100]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:05.958 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.956+0000 7f530affd700 1 -- 172.21.15.195:0/2853685540 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f531406c410 con 0x7f53000b2880 2024-05-10T10:32:05.958 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 5 on host 'smithi195' 2024-05-10T10:32:05.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53000b2880 msgr2=0x7f53000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:05.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53000b2880 0x7f53000b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f530400b5c0 tx=0x7f5304005fb0 comp rx=0 tx=0).stop 2024-05-10T10:32:05.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314102400 msgr2=0x7f5314198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:05.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314102400 0x7f5314198430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5310007b80 tx=0x7f5310007e90 comp rx=0 tx=0).stop 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 shutdown_connections 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5314103fa0 0x7f531419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53000b2880 0x7f53000b4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5314102400 0x7f5314198430 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 --2- 172.21.15.195:0/2853685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5314103600 0x7f5314198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 >> 172.21.15.195:0/2853685540 conn(0x7f53140fd9c0 msgr2=0x7f53141069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 shutdown_connections 2024-05-10T10:32:05.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:32:05.959+0000 7f531c091700 1 -- 172.21.15.195:0/2853685540 wait complete. 2024-05-10T10:32:06.663 DEBUG:teuthology.orchestra.run.smithi195:osd.5> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.5.service 2024-05-10T10:32:06.666 INFO:tasks.cephadm:Deploying osd.6 on smithi112 with /dev/vg_nvme/lv_2... 2024-05-10T10:32:06.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T10:32:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:32:07.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:06 smithi112 ceph-mon[30100]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:07.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:06 smithi112 ceph-mon[30100]: from='osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T10:32:07.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:06 smithi112 ceph-mon[30100]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T10:32:07.189 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:06 smithi195 ceph-mon[33773]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:07.190 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.190 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.190 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.190 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.190 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:06 smithi195 ceph-mon[33773]: from='osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T10:32:07.190 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:06 smithi195 ceph-mon[33773]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T10:32:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:06 smithi148 ceph-mon[33568]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:07.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:06 smithi148 ceph-mon[33568]: from='osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T10:32:07.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:06 smithi148 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T10:32:08.129 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:07 smithi195 ceph-mon[33773]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T10:32:08.130 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:07 smithi195 ceph-mon[33773]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T10:32:08.130 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:08.130 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:07 smithi195 ceph-mon[33773]: from='osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:32:08.130 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:07 smithi195 ceph-mon[33773]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:32:08.130 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:32:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:32:07.983+0000 7f86cf540700 -1 osd.5 0 waiting for initial osdmap 2024-05-10T10:32:08.130 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:32:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:32:07.989+0000 7f86cb333700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:32:08.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:07 smithi112 ceph-mon[30100]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T10:32:08.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:07 smithi112 ceph-mon[30100]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T10:32:08.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:08.291 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:07 smithi112 ceph-mon[30100]: from='osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:32:08.291 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:07 smithi112 ceph-mon[30100]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:32:08.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:07 smithi148 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T10:32:08.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:07 smithi148 ceph-mon[33568]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T10:32:08.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:08.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:07 smithi148 ceph-mon[33568]: from='osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:32:08.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:07 smithi148 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:32:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T10:32:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T10:32:08.865 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-10T10:32:08.865 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-10T10:32:08.865 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130251 s, 805 MB/s 2024-05-10T10:32:08.865 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-10T10:32:09.200 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:08 smithi112 ceph-mon[30100]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:09.200 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:08 smithi112 ceph-mon[30100]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:32:09.200 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:08 smithi112 ceph-mon[30100]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T10:32:09.201 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:08 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:09.201 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:08 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:09.216 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:08 smithi195 ceph-mon[33773]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:09.216 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:08 smithi195 ceph-mon[33773]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:32:09.216 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:08 smithi195 ceph-mon[33773]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T10:32:09.216 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:08 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:09.216 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:08 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:08 smithi148 ceph-mon[33568]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:09.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:08 smithi148 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:32:09.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:08 smithi148 ceph-mon[33568]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T10:32:09.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:08 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:09.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:08 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:09.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-05-10T10:32:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:32:10.055 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:32:10.055 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] boot 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: osdmap e36: 6 total, 6 up, 6 in 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:10.057 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:10.057 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:10.057 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:09 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] boot 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: osdmap e36: 6 total, 6 up, 6 in 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:10.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:10.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:10.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:10.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:10.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:09 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:32:10.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:32:10.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:10.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: osd.5 [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] boot 2024-05-10T10:32:10.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: osdmap e36: 6 total, 6 up, 6 in 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:10.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:09 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:11.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:10 smithi112 ceph-mon[30100]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:11.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:10 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi195 2024-05-10T10:32:11.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:10 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi195 to 8544M 2024-05-10T10:32:11.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:10 smithi112 ceph-mon[30100]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T10:32:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:10 smithi148 ceph-mon[33568]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:10 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi195 2024-05-10T10:32:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:10 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi195 to 8544M 2024-05-10T10:32:11.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:10 smithi148 ceph-mon[33568]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T10:32:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:10 smithi195 ceph-mon[33773]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-10T10:32:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:10 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi195 2024-05-10T10:32:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:10 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi195 to 8544M 2024-05-10T10:32:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:10 smithi195 ceph-mon[33773]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T10:32:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/3222782362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04103600 msgr2=0x7fbf04103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/3222782362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04103600 0x7fbf04103a60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbef40099c0 tx=0x7fbef4009cd0 comp rx=0 tx=0).stop 2024-05-10T10:32:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/3222782362 shutdown_connections 2024-05-10T10:32:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/3222782362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf04103fa0 0x7fbf0410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/3222782362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf04102400 0x7fbf04102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:11.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/3222782362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04103600 0x7fbf04103a60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/3222782362 >> 172.21.15.112:0/3222782362 conn(0x7fbf040fd9a0 msgr2=0x7fbf040ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/3222782362 shutdown_connections 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.318+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/3222782362 wait complete. 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 Processor -- start 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 -- start start 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04102400 0x7fbf041984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf04103600 0x7fbf041989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf04103fa0 0x7fbf0419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0410dd50 con 0x7fbf04102400 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0410dbd0 con 0x7fbf04103600 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf0b4f6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0410ded0 con 0x7fbf04103fa0 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04102400 0x7fbf041984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:11.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04102400 0x7fbf041984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42340/0 (socket says 172.21.15.112:42340) 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 -- 172.21.15.112:0/4254291637 learned_addr learned my addr 172.21.15.112:0/4254291637 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09a93700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf04103fa0 0x7fbf0419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf08a91700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf04103600 0x7fbf041989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 -- 172.21.15.112:0/4254291637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf04103fa0 msgr2=0x7fbf0419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf04103fa0 0x7fbf0419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 -- 172.21.15.112:0/4254291637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf04103600 msgr2=0x7fbf041989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf04103600 0x7fbf041989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 -- 172.21.15.112:0/4254291637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0406b280 con 0x7fbf04102400 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.319+0000 7fbf09292700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04102400 0x7fbf041984a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007f10 tx=0x7fbf0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.320+0000 7fbefa7fc700 1 -- 172.21.15.112:0/4254291637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000dc90 con 0x7fbf04102400 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.320+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0406b570 con 0x7fbf04102400 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.320+0000 7fbefa7fc700 1 -- 172.21.15.112:0/4254291637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf0000de30 con 0x7fbf04102400 2024-05-10T10:32:11.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.320+0000 7fbefa7fc700 1 -- 172.21.15.112:0/4254291637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000133f0 con 0x7fbf04102400 2024-05-10T10:32:11.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.320+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0406bab0 con 0x7fbf04102400 2024-05-10T10:32:11.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.321+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0406a190 con 0x7fbf04102400 2024-05-10T10:32:11.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.323+0000 7fbefa7fc700 1 -- 172.21.15.112:0/4254291637 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbf0001a020 con 0x7fbf04102400 2024-05-10T10:32:11.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.323+0000 7fbefa7fc700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbef00b2880 0x7fbef00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:11.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.324+0000 7fbf08a91700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbef00b2880 0x7fbef00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:11.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.324+0000 7fbefa7fc700 1 -- 172.21.15.112:0/4254291637 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fbf000e1130 con 0x7fbf04102400 2024-05-10T10:32:11.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.324+0000 7fbf08a91700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbef00b2880 0x7fbef00b4d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:32:11.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.327+0000 7fbefa7fc700 1 -- 172.21.15.112:0/4254291637 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbf00010070 con 0x7fbf04102400 2024-05-10T10:32:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:11.538+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbf0402e650 con 0x7fbef00b2880 2024-05-10T10:32:12.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:12 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:32:12.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:12 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:32:12.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:12 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:12.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:12 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:32:12.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:12 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:32:12.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:12 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:12.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:12 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:32:12.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:12 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:32:12.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:12 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:13.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:13 smithi148 ceph-mon[33568]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:13.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:13 smithi148 ceph-mon[33568]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:32:13.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:13 smithi195 ceph-mon[33773]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:13.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:13 smithi195 ceph-mon[33773]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:32:13.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:13 smithi112 ceph-mon[30100]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:13 smithi112 ceph-mon[30100]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:32:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:15 smithi148 ceph-mon[33568]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-10T10:32:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:15 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1449425855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "333835c8-bba4-4777-b7f9-ed8e6bb54ec0"}]: dispatch 2024-05-10T10:32:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:15 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1449425855' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "333835c8-bba4-4777-b7f9-ed8e6bb54ec0"}]': finished 2024-05-10T10:32:15.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:15 smithi148 ceph-mon[33568]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T10:32:15.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:15 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:15.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:15 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2258348315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:32:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:15 smithi195 ceph-mon[33773]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-10T10:32:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:15 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1449425855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "333835c8-bba4-4777-b7f9-ed8e6bb54ec0"}]: dispatch 2024-05-10T10:32:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:15 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1449425855' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "333835c8-bba4-4777-b7f9-ed8e6bb54ec0"}]': finished 2024-05-10T10:32:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:15 smithi195 ceph-mon[33773]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T10:32:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:15 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:15 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2258348315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:32:15.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:15 smithi112 ceph-mon[30100]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-10T10:32:15.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:15 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1449425855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "333835c8-bba4-4777-b7f9-ed8e6bb54ec0"}]: dispatch 2024-05-10T10:32:15.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:15 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1449425855' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "333835c8-bba4-4777-b7f9-ed8e6bb54ec0"}]': finished 2024-05-10T10:32:15.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:15 smithi112 ceph-mon[30100]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T10:32:15.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:15 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:15.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:15 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2258348315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:32:17.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:17 smithi148 ceph-mon[33568]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:32:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:17 smithi195 ceph-mon[33773]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:32:17.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:17 smithi112 ceph-mon[30100]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:32:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:19 smithi148 ceph-mon[33568]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:32:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:19 smithi195 ceph-mon[33773]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:32:19.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:19 smithi112 ceph-mon[30100]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:32:21.201 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:21 smithi112 ceph-mon[30100]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:32:21.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:21 smithi148 ceph-mon[33568]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:32:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:21 smithi195 ceph-mon[33773]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:32:22.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T10:32:22.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:22.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T10:32:22.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:22.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T10:32:22.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:23 smithi148 ceph-mon[33568]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:32:23.297 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:23 smithi148 ceph-mon[33568]: Deploying daemon osd.6 on smithi112 2024-05-10T10:32:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:23 smithi195 ceph-mon[33773]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:32:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:23 smithi195 ceph-mon[33773]: Deploying daemon osd.6 on smithi112 2024-05-10T10:32:23.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:23 smithi112 ceph-mon[30100]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:32:23.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:23 smithi112 ceph-mon[30100]: Deploying daemon osd.6 on smithi112 2024-05-10T10:32:25.200 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:25 smithi112 ceph-mon[30100]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:25 smithi148 ceph-mon[33568]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:25.318 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:25 smithi195 ceph-mon[33773]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:27.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:27 smithi148 ceph-mon[33568]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:27.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:27 smithi195 ceph-mon[33773]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:27.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:27 smithi112 ceph-mon[30100]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:29.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:28 smithi148 ceph-mon[33568]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:29.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:29.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:29.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:28 smithi195 ceph-mon[33773]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:29.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:28 smithi112 ceph-mon[30100]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:29.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:29.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:29.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:31.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:30 smithi148 ceph-mon[33568]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:31.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:30 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:30 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:30 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:31.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:30 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:31.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:30 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:30 smithi195 ceph-mon[33773]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:30 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:30 smithi112 ceph-mon[30100]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:30 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.649+0000 7fbefa7fc700 1 -- 172.21.15.112:0/4254291637 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf0402e650 con 0x7fbef00b2880 2024-05-10T10:32:31.650 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 6 on host 'smithi112' 2024-05-10T10:32:31.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbef00b2880 msgr2=0x7fbef00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbef00b2880 0x7fbef00b4d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef401a040 comp rx=0 tx=0).stop 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04102400 msgr2=0x7fbf041984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04102400 0x7fbf041984a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007f10 tx=0x7fbf0000d3b0 comp rx=0 tx=0).stop 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 shutdown_connections 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbef00b2880 0x7fbef00b4d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf04103fa0 0x7fbf0419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf04102400 0x7fbf041984a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 --2- 172.21.15.112:0/4254291637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf04103600 0x7fbf041989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 >> 172.21.15.112:0/4254291637 conn(0x7fbf040fd9a0 msgr2=0x7fbf04106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 shutdown_connections 2024-05-10T10:32:31.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:32:31.651+0000 7fbf0b4f6700 1 -- 172.21.15.112:0/4254291637 wait complete. 2024-05-10T10:32:31.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:31 smithi112 ceph-mon[30100]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T10:32:31.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:31.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:31 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:32.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:31 smithi148 ceph-mon[33568]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T10:32:32.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:32.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:32.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:31 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:32.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:31 smithi195 ceph-mon[33773]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T10:32:32.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:32.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:32.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:31 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:32.455 DEBUG:teuthology.orchestra.run.smithi112:osd.6> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.6.service 2024-05-10T10:32:32.459 INFO:tasks.cephadm:Deploying osd.7 on smithi148 with /dev/vg_nvme/lv_2... 2024-05-10T10:32:32.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T10:32:32.710 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:32:32.713 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:32 smithi112 ceph-mon[30100]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:32.713 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:32 smithi112 ceph-mon[30100]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T10:32:32.713 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:32 smithi112 ceph-mon[30100]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T10:32:32.713 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:32 smithi112 ceph-mon[30100]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:32:32.713 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:32 smithi148 ceph-mon[33568]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:32 smithi148 ceph-mon[33568]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T10:32:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:32 smithi148 ceph-mon[33568]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T10:32:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:32 smithi148 ceph-mon[33568]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:32:32.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:32 smithi195 ceph-mon[33773]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:32 smithi195 ceph-mon[33773]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T10:32:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:32 smithi195 ceph-mon[33773]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T10:32:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:32 smithi195 ceph-mon[33773]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:32:33.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:33.654 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:32:33 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:32:33.340+0000 7fcce9852700 -1 osd.6 0 waiting for initial osdmap 2024-05-10T10:32:33.654 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:32:33 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:32:33.350+0000 7fcce0e3c700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:32:33.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:33 smithi148 ceph-mon[33568]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:32:33.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:33 smithi148 ceph-mon[33568]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T10:32:33.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:33.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:33.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:33 smithi148 ceph-mon[33568]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' 2024-05-10T10:32:33.919 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:33.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:33 smithi112 ceph-mon[30100]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:32:33.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:33 smithi112 ceph-mon[30100]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T10:32:33.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:33.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:33.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:33 smithi112 ceph-mon[30100]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' 2024-05-10T10:32:33.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:34.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:33 smithi195 ceph-mon[33773]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:32:34.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:33 smithi195 ceph-mon[33773]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T10:32:34.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:34.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:34.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:33 smithi195 ceph-mon[33773]: from='osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261]' entity='osd.6' 2024-05-10T10:32:34.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:34.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:34 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:32:34.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:34 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:32:34.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:34 smithi148 ceph-mon[33568]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:34.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:34 smithi148 ceph-mon[33568]: osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] boot 2024-05-10T10:32:34.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:34 smithi148 ceph-mon[33568]: osdmap e42: 7 total, 7 up, 7 in 2024-05-10T10:32:34.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:34 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:32:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:34 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:32:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:34 smithi112 ceph-mon[30100]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:34 smithi112 ceph-mon[30100]: osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] boot 2024-05-10T10:32:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:34 smithi112 ceph-mon[30100]: osdmap e42: 7 total, 7 up, 7 in 2024-05-10T10:32:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:34.918 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T10:32:34.918 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T10:32:34.919 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-05-10T10:32:34.919 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-05-10T10:32:34.919 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114946 s, 912 MB/s 2024-05-10T10:32:34.919 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-05-10T10:32:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:34 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:32:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:34 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:32:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:34 smithi195 ceph-mon[33773]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-10T10:32:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:34 smithi195 ceph-mon[33773]: osd.6 [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] boot 2024-05-10T10:32:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:34 smithi195 ceph-mon[33773]: osdmap e42: 7 total, 7 up, 7 in 2024-05-10T10:32:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T10:32:35.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-05-10T10:32:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:32:36.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi112 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi112 to 5696M 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:36.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T10:32:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi112 2024-05-10T10:32:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi112 to 5696M 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:36.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi112 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:32:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi112 to 5696M 2024-05-10T10:32:36.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:36.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:36.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:36.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 -- 172.21.15.148:0/64815325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14101bd0 msgr2=0x7f6e14101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 --2- 172.21.15.148:0/64815325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14101bd0 0x7f6e14101ff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009990 tx=0x7f6e04009ca0 comp rx=0 tx=0).stop 2024-05-10T10:32:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 -- 172.21.15.148:0/64815325 shutdown_connections 2024-05-10T10:32:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 --2- 172.21.15.148:0/64815325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e14103770 0x7f6e1410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 --2- 172.21.15.148:0/64815325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14101bd0 0x7f6e14101ff0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 --2- 172.21.15.148:0/64815325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e14102dd0 0x7f6e14103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 -- 172.21.15.148:0/64815325 >> 172.21.15.148:0/64815325 conn(0x7f6e140fd170 msgr2=0x7f6e140ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:32:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 -- 172.21.15.148:0/64815325 shutdown_connections 2024-05-10T10:32:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.407+0000 7f6e190a0700 1 -- 172.21.15.148:0/64815325 wait complete. 2024-05-10T10:32:37.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 Processor -- start 2024-05-10T10:32:37.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 -- start start 2024-05-10T10:32:37.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e14101bd0 0x7f6e14199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:37.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e14102dd0 0x7f6e14199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:37.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14103770 0x7f6e141a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:37.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410ef40 con 0x7f6e14101bd0 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410edc0 con 0x7f6e14103770 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e190a0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410f0c0 con 0x7f6e14102dd0 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e1359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14103770 0x7f6e141a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e1359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14103770 0x7f6e141a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48156/0 (socket says 172.21.15.148:48156) 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e1359e700 1 -- 172.21.15.148:0/1662572126 learned_addr learned my addr 172.21.15.148:0/1662572126 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.408+0000 7f6e1259c700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e14102dd0 0x7f6e14199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e12d9d700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e14101bd0 0x7f6e14199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e1359e700 1 -- 172.21.15.148:0/1662572126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e14102dd0 msgr2=0x7f6e14199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e1359e700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e14102dd0 0x7f6e14199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e1359e700 1 -- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e14101bd0 msgr2=0x7f6e14199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e1359e700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e14101bd0 0x7f6e14199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e1359e700 1 -- 172.21.15.148:0/1662572126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e141a11b0 con 0x7f6e14103770 2024-05-10T10:32:37.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e1359e700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14103770 0x7f6e141a0b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00b970 tx=0x7f6e0c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:32:37.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6dfbfff700 1 -- 172.21.15.148:0/1662572126 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c019070 con 0x7f6e14103770 2024-05-10T10:32:37.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e141a14a0 con 0x7f6e14103770 2024-05-10T10:32:37.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6dfbfff700 1 -- 172.21.15.148:0/1662572126 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e0c00cdc0 con 0x7f6e14103770 2024-05-10T10:32:37.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6dfbfff700 1 -- 172.21.15.148:0/1662572126 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c014e60 con 0x7f6e14103770 2024-05-10T10:32:37.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.409+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141a1ab0 con 0x7f6e14103770 2024-05-10T10:32:37.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.410+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1406a190 con 0x7f6e14103770 2024-05-10T10:32:37.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.411+0000 7f6dfbfff700 1 -- 172.21.15.148:0/1662572126 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6e0c01c070 con 0x7f6e14103770 2024-05-10T10:32:37.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.412+0000 7f6dfbfff700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6dfc0b2880 0x7f6dfc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:32:37.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.412+0000 7f6dfbfff700 1 -- 172.21.15.148:0/1662572126 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f6e0c0e0cd0 con 0x7f6e14103770 2024-05-10T10:32:37.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.412+0000 7f6e12d9d700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6dfc0b2880 0x7f6dfc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:32:37.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.413+0000 7f6e12d9d700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6dfc0b2880 0x7f6dfc0b4d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e04003820 tx=0x7f6e04005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:32:37.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.416+0000 7f6dfbfff700 1 -- 172.21.15.148:0/1662572126 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6e0c0a9700 con 0x7f6e14103770 2024-05-10T10:32:37.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:37.627+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6e140638e0 con 0x7f6dfc0b2880 2024-05-10T10:32:38.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:38 smithi148 ceph-mon[33568]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:38.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:38 smithi148 ceph-mon[33568]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:32:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:32:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:32:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:38.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:38 smithi195 ceph-mon[33773]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:38.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:38 smithi195 ceph-mon[33773]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:32:38.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:32:38.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:32:38.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:38.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:38 smithi112 ceph-mon[30100]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:38.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:38 smithi112 ceph-mon[30100]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:32:38.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:32:38.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:32:38.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:40.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:40 smithi148 ceph-mon[33568]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:40.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:40 smithi195 ceph-mon[33773]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:40.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:40 smithi112 ceph-mon[30100]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:41.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:41 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/821998272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]: dispatch 2024-05-10T10:32:41.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:41 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]: dispatch 2024-05-10T10:32:41.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:41 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]': finished 2024-05-10T10:32:41.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:41 smithi148 ceph-mon[33568]: osdmap e44: 8 total, 7 up, 8 in 2024-05-10T10:32:41.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:41 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:32:41.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:41 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/2255217844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:32:41.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:41 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/821998272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]: dispatch 2024-05-10T10:32:41.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:41 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]: dispatch 2024-05-10T10:32:41.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:41 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]': finished 2024-05-10T10:32:41.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:41 smithi195 ceph-mon[33773]: osdmap e44: 8 total, 7 up, 8 in 2024-05-10T10:32:41.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:41 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:32:41.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:41 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/2255217844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:32:41.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:41 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/821998272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]: dispatch 2024-05-10T10:32:41.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:41 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]: dispatch 2024-05-10T10:32:41.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:41 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57dc7935-dd5e-47f4-87e0-8aae25736340"}]': finished 2024-05-10T10:32:41.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:41 smithi112 ceph-mon[30100]: osdmap e44: 8 total, 7 up, 8 in 2024-05-10T10:32:41.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:41 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:32:41.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:41 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/2255217844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:32:42.718 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:42 smithi195 ceph-mon[33773]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:42.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:42 smithi148 ceph-mon[33568]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:42 smithi112 ceph-mon[30100]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:44.712 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:44 smithi148 ceph-mon[33568]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:44.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:44 smithi195 ceph-mon[33773]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:44.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:44 smithi112 ceph-mon[30100]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:46.653 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:46 smithi148 ceph-mon[33568]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:46.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:46 smithi195 ceph-mon[33773]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:46.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:46 smithi112 ceph-mon[30100]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:48.747 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:48 smithi148 ceph-mon[33568]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:48.747 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T10:32:48.747 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:48.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:48 smithi195 ceph-mon[33773]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:48.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T10:32:48.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:48.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:48 smithi112 ceph-mon[30100]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:48.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T10:32:48.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:49.767 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:49 smithi148 ceph-mon[33568]: Deploying daemon osd.7 on smithi148 2024-05-10T10:32:49.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:49 smithi195 ceph-mon[33773]: Deploying daemon osd.7 on smithi148 2024-05-10T10:32:49.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:49 smithi112 ceph-mon[30100]: Deploying daemon osd.7 on smithi148 2024-05-10T10:32:50.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:50 smithi148 ceph-mon[33568]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:50.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:50 smithi195 ceph-mon[33773]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:50 smithi112 ceph-mon[30100]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:52.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:52 smithi148 ceph-mon[33568]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:52.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:52 smithi195 ceph-mon[33773]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:52.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:52 smithi112 ceph-mon[30100]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:55.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:54 smithi148 ceph-mon[33568]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:55.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:54 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:55.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:54 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:55.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:54 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:54 smithi195 ceph-mon[33773]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:54 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:54 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:54 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:55.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:54 smithi112 ceph-mon[30100]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:55.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:54 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:55.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:54 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:55.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:54 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:57.216 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:56 smithi148 ceph-mon[33568]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:56 smithi195 ceph-mon[33773]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:57.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:56 smithi112 ceph-mon[30100]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:57.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.994+0000 7f6dfbfff700 1 -- 172.21.15.148:0/1662572126 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6e140638e0 con 0x7f6dfc0b2880 2024-05-10T10:32:57.995 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 7 on host 'smithi148' 2024-05-10T10:32:57.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6dfc0b2880 msgr2=0x7f6dfc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:57.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6dfc0b2880 0x7f6dfc0b4d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e04003820 tx=0x7f6e04005fb0 comp rx=0 tx=0).stop 2024-05-10T10:32:57.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14103770 msgr2=0x7f6e141a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:32:57.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14103770 0x7f6e141a0b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00b970 tx=0x7f6e0c00bc80 comp rx=0 tx=0).stop 2024-05-10T10:32:57.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 shutdown_connections 2024-05-10T10:32:57.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e14103770 0x7f6e141a0b10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:57.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e14101bd0 0x7f6e14199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:57.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6dfc0b2880 0x7f6dfc0b4d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:57.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 --2- 172.21.15.148:0/1662572126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e14102dd0 0x7f6e14199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:32:57.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.996+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 >> 172.21.15.148:0/1662572126 conn(0x7f6e140fd170 msgr2=0x7f6e14106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:32:57.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.997+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 shutdown_connections 2024-05-10T10:32:57.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:32:57.997+0000 7f6e190a0700 1 -- 172.21.15.148:0/1662572126 wait complete. 2024-05-10T10:32:58.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:58.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:58.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:58.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:58.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T10:32:58.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T10:32:58.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:58.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:32:58.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:32:58.762 DEBUG:teuthology.orchestra.run.smithi148:osd.7> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.7.service 2024-05-10T10:32:58.765 INFO:tasks.cephadm:Deploying osd.8 on smithi195 with /dev/vg_nvme/lv_2... 2024-05-10T10:32:58.765 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T10:32:59.014 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:32:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:59 smithi148 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T10:32:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:59 smithi148 ceph-mon[33568]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T10:32:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:59 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:32:59.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:59 smithi148 ceph-mon[33568]: from='osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:32:59.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:32:59 smithi148 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:32:59.388 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:59 smithi195 ceph-mon[33773]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T10:32:59.388 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:59 smithi195 ceph-mon[33773]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T10:32:59.388 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:59 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:32:59.388 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:59 smithi195 ceph-mon[33773]: from='osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:32:59.388 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:32:59 smithi195 ceph-mon[33773]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:32:59.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:59 smithi112 ceph-mon[30100]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T10:32:59.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:59 smithi112 ceph-mon[30100]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T10:32:59.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:59 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:32:59.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:59 smithi112 ceph-mon[30100]: from='osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:32:59.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:32:59 smithi112 ceph-mon[30100]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:33:00.133 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:32:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:32:59.845+0000 7ff6019a9700 -1 osd.7 0 waiting for initial osdmap 2024-05-10T10:33:00.133 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:32:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:32:59.856+0000 7ff5f9794700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:00.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:33:00.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T10:33:00.415 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:33:00.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T10:33:00.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.416 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:00 smithi195 ceph-mon[33773]: from='osd.7 ' entity='osd.7' 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:00 smithi148 ceph-mon[33568]: from='osd.7 ' entity='osd.7' 2024-05-10T10:33:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:33:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T10:33:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-10T10:33:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T10:33:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:00.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:00 smithi112 ceph-mon[30100]: from='osd.7 ' entity='osd.7' 2024-05-10T10:33:01.100 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T10:33:01.100 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T10:33:01.100 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-10T10:33:01.100 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-10T10:33:01.100 INFO:teuthology.orchestra.run.smithi195.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00864083 s, 1.2 GB/s 2024-05-10T10:33:01.100 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-10T10:33:01.398 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:01 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:33:01.399 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:01 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:33:01.399 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:01.399 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:01 smithi195 ceph-mon[33773]: osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] boot 2024-05-10T10:33:01.399 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:01 smithi195 ceph-mon[33773]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T10:33:01.399 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:01 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:33:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:01 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:33:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:01 smithi148 ceph-mon[33568]: osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] boot 2024-05-10T10:33:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:01 smithi148 ceph-mon[33568]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T10:33:01.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:01 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:33:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:01 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:33:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:01 smithi112 ceph-mon[30100]: osd.7 [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] boot 2024-05-10T10:33:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:01 smithi112 ceph-mon[30100]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T10:33:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T10:33:01.823 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_2 2024-05-10T10:33:02.069 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:33:02.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:02.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T10:33:02.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi148 2024-05-10T10:33:02.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi148 to 5696M 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:02 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:02.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T10:33:02.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi148 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi148 to 5696M 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:02.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:02 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi148 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi148 to 5696M 2024-05-10T10:33:02.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:02.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:02.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:02.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:02 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:03.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.721+0000 7f173b872700 1 -- 172.21.15.195:0/3587103764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 msgr2=0x7f1734103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:03.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.721+0000 7f173b872700 1 --2- 172.21.15.195:0/3587103764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734103a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1724009990 tx=0x7f1724009ca0 comp rx=0 tx=0).stop 2024-05-10T10:33:03.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.721+0000 7f173b872700 1 -- 172.21.15.195:0/3587103764 shutdown_connections 2024-05-10T10:33:03.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.721+0000 7f173b872700 1 --2- 172.21.15.195:0/3587103764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1734103fb0 0x7f173410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:03.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.721+0000 7f173b872700 1 --2- 172.21.15.195:0/3587103764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734102410 0x7f1734102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:03.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.721+0000 7f173b872700 1 --2- 172.21.15.195:0/3587103764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734103a70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.721+0000 7f173b872700 1 -- 172.21.15.195:0/3587103764 >> 172.21.15.195:0/3587103764 conn(0x7f17340fd9d0 msgr2=0x7f17340ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 -- 172.21.15.195:0/3587103764 shutdown_connections 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 -- 172.21.15.195:0/3587103764 wait complete. 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 Processor -- start 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 -- start start 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1734102410 0x7f1734199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734103fb0 0x7f17341a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f173410de50 con 0x7f1734103fb0 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f173410dcd0 con 0x7f1734102410 2024-05-10T10:33:03.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f173b872700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f173410dfd0 con 0x7f1734103610 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f1738e0d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f1738e0d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33642/0 (socket says 172.21.15.195:33642) 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.722+0000 7f1738e0d700 1 -- 172.21.15.195:0/69960482 learned_addr learned my addr 172.21.15.195:0/69960482 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f1739e0f700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734103fb0 0x7f17341a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f173960e700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1734102410 0x7f1734199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f1738e0d700 1 -- 172.21.15.195:0/69960482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1734102410 msgr2=0x7f1734199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f1738e0d700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1734102410 0x7f1734199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f1738e0d700 1 -- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734103fb0 msgr2=0x7f17341a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f1738e0d700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734103fb0 0x7f17341a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f1738e0d700 1 -- 172.21.15.195:0/69960482 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17341a13e0 con 0x7f1734103610 2024-05-10T10:33:03.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f1738e0d700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734199cc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17240057f0 tx=0x7f1724005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:03.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f172a7fc700 1 -- 172.21.15.195:0/69960482 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724027070 con 0x7f1734103610 2024-05-10T10:33:03.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17341a1670 con 0x7f1734103610 2024-05-10T10:33:03.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f172a7fc700 1 -- 172.21.15.195:0/69960482 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17240040a0 con 0x7f1734103610 2024-05-10T10:33:03.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f172a7fc700 1 -- 172.21.15.195:0/69960482 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724022d00 con 0x7f1734103610 2024-05-10T10:33:03.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.723+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17341a1b40 con 0x7f1734103610 2024-05-10T10:33:03.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.724+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173406a190 con 0x7f1734103610 2024-05-10T10:33:03.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.726+0000 7f172a7fc700 1 -- 172.21.15.195:0/69960482 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f172402a070 con 0x7f1734103610 2024-05-10T10:33:03.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.727+0000 7f172a7fc700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f17200b2880 0x7f17200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:03.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.727+0000 7f172a7fc700 1 -- 172.21.15.195:0/69960482 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f17240e2d30 con 0x7f1734103610 2024-05-10T10:33:03.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.727+0000 7f173960e700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f17200b2880 0x7f17200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:03.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.728+0000 7f173960e700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f17200b2880 0x7f17200b4d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1734103470 tx=0x7f1730009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:03.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.731+0000 7f172a7fc700 1 -- 172.21.15.195:0/69960482 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1724031020 con 0x7f1734103610 2024-05-10T10:33:03.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:03 smithi148 ceph-mon[33568]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T10:33:03.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:03 smithi195 ceph-mon[33773]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T10:33:03.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:03 smithi112 ceph-mon[30100]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T10:33:03.946 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:03.945+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f173406c300 con 0x7f17200b2880 2024-05-10T10:33:04.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:04 smithi148 ceph-mon[33568]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:04.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:04 smithi148 ceph-mon[33568]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:04.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:04 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:04.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:04 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:04.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:04 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:04.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:04 smithi195 ceph-mon[33773]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:04.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:04 smithi195 ceph-mon[33773]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:04.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:04 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:04.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:04 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:04.803 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:04 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:04.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:04 smithi112 ceph-mon[30100]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:04.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:04 smithi112 ceph-mon[30100]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:04.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:04.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:04.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:04 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:06.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:06 smithi195 ceph-mon[33773]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:06.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:06 smithi112 ceph-mon[30100]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:06 smithi148 ceph-mon[33568]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:07 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/3651773134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]: dispatch 2024-05-10T10:33:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:07 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]: dispatch 2024-05-10T10:33:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:07 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]': finished 2024-05-10T10:33:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:07 smithi195 ceph-mon[33773]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T10:33:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:07 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:07 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/3335991974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:33:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:07 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/3651773134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]: dispatch 2024-05-10T10:33:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:07 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]: dispatch 2024-05-10T10:33:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:07 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]': finished 2024-05-10T10:33:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:07 smithi112 ceph-mon[30100]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T10:33:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:07 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:07.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:07 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/3335991974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:33:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:07 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/3651773134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]: dispatch 2024-05-10T10:33:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:07 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]: dispatch 2024-05-10T10:33:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:07 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e756afdf-f8df-4bda-ad15-a943ba0527d6"}]': finished 2024-05-10T10:33:08.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:07 smithi148 ceph-mon[33568]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T10:33:08.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:07 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:08.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:07 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/3335991974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:33:09.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:08 smithi148 ceph-mon[33568]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-10T10:33:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:08 smithi195 ceph-mon[33773]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-10T10:33:09.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:08 smithi112 ceph-mon[30100]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-10T10:33:10.934 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:10 smithi195 ceph-mon[33773]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-05-10T10:33:11.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:10 smithi148 ceph-mon[33568]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-05-10T10:33:11.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:10 smithi112 ceph-mon[30100]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-05-10T10:33:12.902 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:12 smithi195 ceph-mon[33773]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-05-10T10:33:13.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:12 smithi148 ceph-mon[33568]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-05-10T10:33:13.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:12 smithi112 ceph-mon[30100]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-05-10T10:33:14.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:14 smithi195 ceph-mon[33773]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-10T10:33:14.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T10:33:14.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:14.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:14 smithi195 ceph-mon[33773]: Deploying daemon osd.8 on smithi195 2024-05-10T10:33:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:14 smithi148 ceph-mon[33568]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-10T10:33:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T10:33:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:14 smithi148 ceph-mon[33568]: Deploying daemon osd.8 on smithi195 2024-05-10T10:33:15.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:14 smithi112 ceph-mon[30100]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-10T10:33:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T10:33:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:14 smithi112 ceph-mon[30100]: Deploying daemon osd.8 on smithi195 2024-05-10T10:33:16.989 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:16 smithi195 ceph-mon[33773]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-10T10:33:17.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:16 smithi148 ceph-mon[33568]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-10T10:33:17.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:16 smithi112 ceph-mon[30100]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-10T10:33:18.933 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:18 smithi195 ceph-mon[33773]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:18 smithi148 ceph-mon[33568]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:19.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:18 smithi112 ceph-mon[30100]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:20 smithi195 ceph-mon[33773]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:20 smithi112 ceph-mon[30100]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:20.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:21.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:20 smithi148 ceph-mon[33568]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:21.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:21.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:21.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:22 smithi195 ceph-mon[33773]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:22.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:22 smithi112 ceph-mon[30100]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:23.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:22 smithi148 ceph-mon[33568]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:23.459 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.457+0000 7f172a7fc700 1 -- 172.21.15.195:0/69960482 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f173406c300 con 0x7f17200b2880 2024-05-10T10:33:23.459 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 8 on host 'smithi195' 2024-05-10T10:33:23.461 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f17200b2880 msgr2=0x7f17200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:23.461 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f17200b2880 0x7f17200b4d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1734103470 tx=0x7f1730009190 comp rx=0 tx=0).stop 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 msgr2=0x7f1734199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734199cc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17240057f0 tx=0x7f1724005e80 comp rx=0 tx=0).stop 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 shutdown_connections 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734103fb0 0x7f17341a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1734102410 0x7f1734199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f17200b2880 0x7f17200b4d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 --2- 172.21.15.195:0/69960482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1734103610 0x7f1734199cc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 >> 172.21.15.195:0/69960482 conn(0x7f17340fd9d0 msgr2=0x7f173406bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 shutdown_connections 2024-05-10T10:33:23.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:33:23.460+0000 7f173b872700 1 -- 172.21.15.195:0/69960482 wait complete. 2024-05-10T10:33:23.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:23.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:23.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:23.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T10:33:23.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:23 smithi195 ceph-mon[33773]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T10:33:23.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T10:33:23.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:23 smithi112 ceph-mon[30100]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T10:33:24.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:24.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:24.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:24.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:24.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:24.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:24.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:24.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:24.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T10:33:24.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:23 smithi148 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T10:33:24.133 DEBUG:teuthology.orchestra.run.smithi195:osd.8> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.8.service 2024-05-10T10:33:24.135 INFO:tasks.cephadm:Deploying osd.9 on smithi112 with /dev/vg_nvme/lv_1... 2024-05-10T10:33:24.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T10:33:24.374 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:33:24.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:24 smithi112 ceph-mon[30100]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:24.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:24 smithi112 ceph-mon[30100]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T10:33:24.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:24 smithi112 ceph-mon[30100]: from='osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:33:24.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:24 smithi112 ceph-mon[30100]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T10:33:24.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:24.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:24 smithi112 ceph-mon[30100]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:33:25.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:24 smithi148 ceph-mon[33568]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:25.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:24 smithi148 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T10:33:25.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:24 smithi148 ceph-mon[33568]: from='osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:33:25.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:24 smithi148 ceph-mon[33568]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T10:33:25.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:25.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:24 smithi148 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:33:25.050 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:24 smithi195 ceph-mon[33773]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:25.050 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:24 smithi195 ceph-mon[33773]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T10:33:25.050 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:24 smithi195 ceph-mon[33773]: from='osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:33:25.050 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:24 smithi195 ceph-mon[33773]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T10:33:25.050 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:25.050 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:24 smithi195 ceph-mon[33773]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:33:25.651 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:33:25 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:33:25.448+0000 7f62cb5a1700 -1 osd.8 0 waiting for initial osdmap 2024-05-10T10:33:25.651 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:33:25 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:33:25.460+0000 7f62c7394700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T10:33:26.474 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T10:33:26.474 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-10T10:33:26.474 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-10T10:33:26.474 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118114 s, 888 MB/s 2024-05-10T10:33:26.474 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-10T10:33:26.757 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:26 smithi195 ceph-mon[33773]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:26.757 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:26 smithi195 ceph-mon[33773]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:33:26.758 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:26 smithi195 ceph-mon[33773]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T10:33:26.758 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:26 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:26.758 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:26 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:26.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:26 smithi148 ceph-mon[33568]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:26.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:26 smithi148 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:33:26.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:26 smithi148 ceph-mon[33568]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T10:33:26.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:26 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:26.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:26 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:26.795 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:26 smithi112 ceph-mon[30100]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:26.795 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:26 smithi112 ceph-mon[30100]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:33:26.795 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:26 smithi112 ceph-mon[30100]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T10:33:26.795 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:26 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:26.795 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:26 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:27.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-05-10T10:33:27.457 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:33:27.700 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:33:27.700 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] boot 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: osdmap e54: 9 total, 9 up, 9 in 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:27.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:27 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] boot 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: osdmap e54: 9 total, 9 up, 9 in 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:27.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:27.796 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:27 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:33:27.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:33:27.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: osd.8 [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] boot 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: osdmap e54: 9 total, 9 up, 9 in 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:27.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:27.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:27.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:27.818 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:27 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:28.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:28 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi195 2024-05-10T10:33:28.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:28 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi195 to 5696M 2024-05-10T10:33:28.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:28 smithi112 ceph-mon[30100]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:28.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:28 smithi112 ceph-mon[30100]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T10:33:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:28 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi195 2024-05-10T10:33:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:28 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi195 to 5696M 2024-05-10T10:33:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:28 smithi148 ceph-mon[33568]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:28 smithi148 ceph-mon[33568]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T10:33:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:28 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi195 2024-05-10T10:33:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:28 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi195 to 5696M 2024-05-10T10:33:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:28 smithi195 ceph-mon[33773]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-10T10:33:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:28 smithi195 ceph-mon[33773]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 -- 172.21.15.112:0/2402096656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0103630 msgr2=0x7f9ad0103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 --2- 172.21.15.112:0/2402096656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0103630 0x7f9ad0103a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80099c0 tx=0x7f9ab8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 -- 172.21.15.112:0/2402096656 shutdown_connections 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 --2- 172.21.15.112:0/2402096656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad0103fd0 0x7f9ad010a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 --2- 172.21.15.112:0/2402096656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0102430 0x7f9ad0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 --2- 172.21.15.112:0/2402096656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0103630 0x7f9ad0103a90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 -- 172.21.15.112:0/2402096656 >> 172.21.15.112:0/2402096656 conn(0x7f9ad00fd9f0 msgr2=0x7f9ad00ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 -- 172.21.15.112:0/2402096656 shutdown_connections 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 -- 172.21.15.112:0/2402096656 wait complete. 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.061+0000 7f9ad5111700 1 Processor -- start 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ad5111700 1 -- start start 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ad5111700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0102430 0x7f9ad006c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ad5111700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0103630 0x7f9ad019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ad5111700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad0103fd0 0x7f9ad01a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ad5111700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010de70 con 0x7f9ad0102430 2024-05-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ad5111700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010dcf0 con 0x7f9ad0103fd0 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ad5111700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010dff0 con 0x7f9ad0103630 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0102430 0x7f9ad006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0102430 0x7f9ad006c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59538/0 (socket says 172.21.15.112:59538) 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 -- 172.21.15.112:0/3507330499 learned_addr learned my addr 172.21.15.112:0/3507330499 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9ac659c700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0103630 0x7f9ad019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9acf59e700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad0103fd0 0x7f9ad01a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 -- 172.21.15.112:0/3507330499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0103630 msgr2=0x7f9ad019e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0103630 0x7f9ad019e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 -- 172.21.15.112:0/3507330499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad0103fd0 msgr2=0x7f9ad01a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad0103fd0 0x7f9ad01a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.062+0000 7f9aced9d700 1 -- 172.21.15.112:0/3507330499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad01a12a0 con 0x7f9ad0102430 2024-05-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.063+0000 7f9aced9d700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0102430 0x7f9ad006c730 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000b500 tx=0x7f9ac000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.063+0000 7f9accd99700 1 -- 172.21.15.112:0/3507330499 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0015020 con 0x7f9ad0102430 2024-05-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.063+0000 7f9accd99700 1 -- 172.21.15.112:0/3507330499 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ac0011cc0 con 0x7f9ad0102430 2024-05-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.063+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad01a1590 con 0x7f9ad0102430 2024-05-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.063+0000 7f9accd99700 1 -- 172.21.15.112:0/3507330499 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0010ca0 con 0x7f9ad0102430 2024-05-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.063+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad01982e0 con 0x7f9ad0102430 2024-05-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.064+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad006a190 con 0x7f9ad0102430 2024-05-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.065+0000 7f9accd99700 1 -- 172.21.15.112:0/3507330499 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9ac0018020 con 0x7f9ad0102430 2024-05-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.066+0000 7f9accd99700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9abc0b2880 0x7f9abc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.066+0000 7f9ac659c700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9abc0b2880 0x7f9abc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.067+0000 7f9accd99700 1 -- 172.21.15.112:0/3507330499 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f9ac00e1720 con 0x7f9ad0102430 2024-05-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.067+0000 7f9ac659c700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9abc0b2880 0x7f9abc0b4d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8000c00 tx=0x7f9ab801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.070+0000 7f9accd99700 1 -- 172.21.15.112:0/3507330499 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9ac001f020 con 0x7f9ad0102430 2024-05-10T10:33:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:29.284+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9ad00638e0 con 0x7f9abc0b2880 2024-05-10T10:33:29.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:29 smithi148 ceph-mon[33568]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T10:33:29.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:29 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:29.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:29 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:29.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:29 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:29.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:29 smithi195 ceph-mon[33773]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T10:33:29.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:29 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:29.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:29 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:29.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:29 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:29.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:29 smithi112 ceph-mon[30100]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T10:33:29.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:29 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:29.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:29 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:29.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:29 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:30 smithi148 ceph-mon[33568]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:30 smithi148 ceph-mon[33568]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:30.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:30 smithi195 ceph-mon[33773]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:30.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:30 smithi195 ceph-mon[33773]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:30.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:30 smithi112 ceph-mon[30100]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:30.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:30 smithi112 ceph-mon[30100]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:32 smithi148 ceph-mon[33568]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-10T10:33:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:32 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1694489535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1111b3c0-269c-451e-8e48-bc2ae180960c"}]: dispatch 2024-05-10T10:33:32.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:32 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1694489535' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1111b3c0-269c-451e-8e48-bc2ae180960c"}]': finished 2024-05-10T10:33:32.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:32 smithi148 ceph-mon[33568]: osdmap e57: 10 total, 9 up, 10 in 2024-05-10T10:33:32.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:32 smithi195 ceph-mon[33773]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-10T10:33:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:32 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1694489535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1111b3c0-269c-451e-8e48-bc2ae180960c"}]: dispatch 2024-05-10T10:33:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:32 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1694489535' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1111b3c0-269c-451e-8e48-bc2ae180960c"}]': finished 2024-05-10T10:33:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:32 smithi195 ceph-mon[33773]: osdmap e57: 10 total, 9 up, 10 in 2024-05-10T10:33:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:32.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:32 smithi112 ceph-mon[30100]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-10T10:33:32.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:32 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1694489535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1111b3c0-269c-451e-8e48-bc2ae180960c"}]: dispatch 2024-05-10T10:33:32.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:32 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1694489535' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1111b3c0-269c-451e-8e48-bc2ae180960c"}]': finished 2024-05-10T10:33:32.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:32 smithi112 ceph-mon[30100]: osdmap e57: 10 total, 9 up, 10 in 2024-05-10T10:33:32.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:33.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:33 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3947658017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:33:33.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:33 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3947658017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:33:33.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:33 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3947658017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:33:34.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:34 smithi148 ceph-mon[33568]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:33:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:34 smithi195 ceph-mon[33773]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:33:34.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:34 smithi112 ceph-mon[30100]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:33:36.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:36 smithi148 ceph-mon[33568]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:33:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:36 smithi195 ceph-mon[33773]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:33:36.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:36 smithi112 ceph-mon[30100]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:33:38.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:38 smithi148 ceph-mon[33568]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:33:38.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:38 smithi195 ceph-mon[33773]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:33:38.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:38 smithi112 ceph-mon[30100]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:33:40.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:40 smithi112 ceph-mon[30100]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:33:40.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T10:33:40.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:40.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:40 smithi112 ceph-mon[30100]: Deploying daemon osd.9 on smithi112 2024-05-10T10:33:40.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:40 smithi148 ceph-mon[33568]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:33:40.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T10:33:40.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:40.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:40 smithi148 ceph-mon[33568]: Deploying daemon osd.9 on smithi112 2024-05-10T10:33:40.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:40 smithi195 ceph-mon[33773]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T10:33:40.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T10:33:40.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:40.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:40 smithi195 ceph-mon[33773]: Deploying daemon osd.9 on smithi112 2024-05-10T10:33:42.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:42 smithi112 ceph-mon[30100]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:42.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:42 smithi148 ceph-mon[33568]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:42.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:42 smithi195 ceph-mon[33773]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:44.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:44 smithi148 ceph-mon[33568]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:44.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:44 smithi195 ceph-mon[33773]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:44.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:44 smithi112 ceph-mon[30100]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:46.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:46 smithi112 ceph-mon[30100]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:46.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:46.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:46.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:47.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:46 smithi148 ceph-mon[33568]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:47.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:47.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:47.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:46 smithi195 ceph-mon[33773]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:49.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:48 smithi148 ceph-mon[33568]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:48 smithi195 ceph-mon[33773]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:49.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:48 smithi112 ceph-mon[30100]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.640+0000 7f9accd99700 1 -- 172.21.15.112:0/3507330499 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ad00638e0 con 0x7f9abc0b2880 2024-05-10T10:33:49.642 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 9 on host 'smithi112' 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9abc0b2880 msgr2=0x7f9abc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9abc0b2880 0x7f9abc0b4d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8000c00 tx=0x7f9ab801a040 comp rx=0 tx=0).stop 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0102430 msgr2=0x7f9ad006c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0102430 0x7f9ad006c730 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000b500 tx=0x7f9ac000b8c0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 shutdown_connections 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad0103fd0 0x7f9ad01a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad0102430 0x7f9ad006c730 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9abc0b2880 0x7f9abc0b4d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 --2- 172.21.15.112:0/3507330499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0103630 0x7f9ad019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.642+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 >> 172.21.15.112:0/3507330499 conn(0x7f9ad00fd9f0 msgr2=0x7f9ad0106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.643+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 shutdown_connections 2024-05-10T10:33:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:33:49.643+0000 7f9ad5111700 1 -- 172.21.15.112:0/3507330499 wait complete. 2024-05-10T10:33:49.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:49.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:50.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:50.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T10:33:50.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:50.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:50.325 DEBUG:teuthology.orchestra.run.smithi112:osd.9> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.9.service 2024-05-10T10:33:50.328 INFO:tasks.cephadm:Deploying osd.10 on smithi148 with /dev/vg_nvme/lv_1... 2024-05-10T10:33:50.328 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T10:33:50.578 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:33:50.953 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:50 smithi148 ceph-mon[33568]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:50.953 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:50 smithi148 ceph-mon[33568]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T10:33:50.953 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:50 smithi148 ceph-mon[33568]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T10:33:50.953 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:50 smithi148 ceph-mon[33568]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:33:50.953 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:50 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:50 smithi195 ceph-mon[33773]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:50 smithi195 ceph-mon[33773]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T10:33:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:50 smithi195 ceph-mon[33773]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T10:33:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:50 smithi195 ceph-mon[33773]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:33:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:50 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:51.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:50 smithi112 ceph-mon[30100]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:51.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:50 smithi112 ceph-mon[30100]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T10:33:51.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:50 smithi112 ceph-mon[30100]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T10:33:51.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:50 smithi112 ceph-mon[30100]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-10T10:33:51.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:50 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:51.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:33:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:33:50.707+0000 7f5216eab700 -1 osd.9 0 waiting for initial osdmap 2024-05-10T10:33:51.081 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:33:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:33:50.717+0000 7f5212c9e700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:51.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:51 smithi112 ceph-mon[30100]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:33:51.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:51 smithi112 ceph-mon[30100]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T10:33:51.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:51.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:52.032 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:51 smithi148 ceph-mon[33568]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:33:52.032 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:51 smithi148 ceph-mon[33568]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T10:33:52.032 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:52.032 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:51 smithi195 ceph-mon[33773]: from='osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-10T10:33:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:51 smithi195 ceph-mon[33773]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T10:33:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:52.611 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T10:33:52.611 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T10:33:52.611 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-05-10T10:33:52.611 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-05-10T10:33:52.612 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140499 s, 746 MB/s 2024-05-10T10:33:52.612 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-05-10T10:33:53.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:52 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:33:53.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:52 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:33:53.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:52 smithi148 ceph-mon[33568]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:53.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:52 smithi148 ceph-mon[33568]: osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] boot 2024-05-10T10:33:53.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:52 smithi148 ceph-mon[33568]: osdmap e60: 10 total, 10 up, 10 in 2024-05-10T10:33:53.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:52 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:53.060 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:52 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:33:53.060 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:52 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:33:53.061 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:52 smithi112 ceph-mon[30100]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:53.061 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:52 smithi112 ceph-mon[30100]: osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] boot 2024-05-10T10:33:53.061 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:52 smithi112 ceph-mon[30100]: osdmap e60: 10 total, 10 up, 10 in 2024-05-10T10:33:53.061 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:52 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:52 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:33:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:52 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:33:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:52 smithi195 ceph-mon[33773]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-10T10:33:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:52 smithi195 ceph-mon[33773]: osd.9 [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] boot 2024-05-10T10:33:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:52 smithi195 ceph-mon[33773]: osdmap e60: 10 total, 10 up, 10 in 2024-05-10T10:33:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:52 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T10:33:53.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2024-05-10T10:33:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:33:53.973 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:53.974 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:53 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:54.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:53 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T10:33:54.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:33:54.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:53 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:33:55.036 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:54 smithi148 ceph-mon[33568]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:55.036 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:54 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi112 2024-05-10T10:33:55.036 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:54 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi112 to 4272M 2024-05-10T10:33:55.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:54 smithi195 ceph-mon[33773]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:55.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:54 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi112 2024-05-10T10:33:55.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:54 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi112 to 4272M 2024-05-10T10:33:55.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:54 smithi112 ceph-mon[30100]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:55.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:54 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi112 2024-05-10T10:33:55.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:54 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi112 to 4272M 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3063723706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 msgr2=0x7fb6a41035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3063723706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41035a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00b0a0 tx=0x7fb69c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3063723706 shutdown_connections 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3063723706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41035a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3063723706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6a4101e00 0x7fb6a4102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3063723706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a4102760 0x7fb6a4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3063723706 >> 172.21.15.148:0/3063723706 conn(0x7fb6a40fd9a0 msgr2=0x7fb6a40ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3063723706 shutdown_connections 2024-05-10T10:33:55.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.269+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3063723706 wait complete. 2024-05-10T10:33:55.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 Processor -- start 2024-05-10T10:33:55.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 -- start start 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6a4101e00 0x7fb6a419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a4102760 0x7fb6a419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4111010 con 0x7fb6a4101e00 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4110e90 con 0x7fb6a4103120 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6ab27f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4111190 con 0x7fb6a4102760 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57992/0 (socket says 172.21.15.148:57992) 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 -- 172.21.15.148:0/3697699032 learned_addr learned my addr 172.21.15.148:0/3697699032 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a881a700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a4102760 0x7fb6a419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a901b700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6a4101e00 0x7fb6a419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 -- 172.21.15.148:0/3697699032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a4102760 msgr2=0x7fb6a419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a4102760 0x7fb6a419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 -- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6a4101e00 msgr2=0x7fb6a419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6a4101e00 0x7fb6a419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 -- 172.21.15.148:0/3697699032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a41a3350 con 0x7fb6a4103120 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a901b700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6a4101e00 0x7fb6a419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.270+0000 7fb6a981c700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41a2cb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00b830 tx=0x7fb69c009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.271+0000 7fb69a7fc700 1 -- 172.21.15.148:0/3697699032 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c021070 con 0x7fb6a4103120 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.271+0000 7fb69a7fc700 1 -- 172.21.15.148:0/3697699032 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb69c007c00 con 0x7fb6a4103120 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.271+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a41a35e0 con 0x7fb6a4103120 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.271+0000 7fb69a7fc700 1 -- 172.21.15.148:0/3697699032 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c01b720 con 0x7fb6a4103120 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.271+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a41a3a90 con 0x7fb6a4103120 2024-05-10T10:33:55.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.272+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a406a190 con 0x7fb6a4103120 2024-05-10T10:33:55.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.273+0000 7fb69a7fc700 1 -- 172.21.15.148:0/3697699032 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb69c024070 con 0x7fb6a4103120 2024-05-10T10:33:55.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.274+0000 7fb69a7fc700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6900b2900 0x7fb6900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.274+0000 7fb69a7fc700 1 -- 172.21.15.148:0/3697699032 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7fb69c00f020 con 0x7fb6a4103120 2024-05-10T10:33:55.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.274+0000 7fb6a901b700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6900b2900 0x7fb6900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.275+0000 7fb6a901b700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6900b2900 0x7fb6900b4dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0005950 tx=0x7fb6a00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.277+0000 7fb69a7fc700 1 -- 172.21.15.148:0/3697699032 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb69c02b020 con 0x7fb6a4103120 2024-05-10T10:33:55.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:33:55.492+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb6a40639a0 con 0x7fb6900b2900 2024-05-10T10:33:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:55.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:56.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:56.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:56.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:56.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:33:56.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:33:56.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:33:57.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:56 smithi148 ceph-mon[33568]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:57.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:56 smithi148 ceph-mon[33568]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:56 smithi195 ceph-mon[33773]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:56 smithi195 ceph-mon[33773]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:57.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:56 smithi112 ceph-mon[30100]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:57.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:56 smithi112 ceph-mon[30100]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:33:59.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:58 smithi148 ceph-mon[33568]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:59.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:58 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/3619284774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]: dispatch 2024-05-10T10:33:59.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:58 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]: dispatch 2024-05-10T10:33:59.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:58 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]': finished 2024-05-10T10:33:59.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:58 smithi148 ceph-mon[33568]: osdmap e62: 11 total, 10 up, 11 in 2024-05-10T10:33:59.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:33:59.065 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:58 smithi195 ceph-mon[33773]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:58 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/3619284774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]: dispatch 2024-05-10T10:33:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:58 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]: dispatch 2024-05-10T10:33:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:58 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]': finished 2024-05-10T10:33:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:58 smithi195 ceph-mon[33773]: osdmap e62: 11 total, 10 up, 11 in 2024-05-10T10:33:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:33:59.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:58 smithi112 ceph-mon[30100]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:33:59.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:58 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/3619284774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]: dispatch 2024-05-10T10:33:59.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:58 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]: dispatch 2024-05-10T10:33:59.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:58 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "841869b8-c760-4899-9e09-33c930cb81da"}]': finished 2024-05-10T10:33:59.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:58 smithi112 ceph-mon[30100]: osdmap e62: 11 total, 10 up, 11 in 2024-05-10T10:33:59.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:00.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:33:59 smithi148 ceph-mon[33568]: from='client.? 172.21.15.148:0/1565262769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:34:00.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:33:59 smithi195 ceph-mon[33773]: from='client.? 172.21.15.148:0/1565262769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:34:00.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:33:59 smithi112 ceph-mon[30100]: from='client.? 172.21.15.148:0/1565262769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:34:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:00 smithi148 ceph-mon[33568]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:00 smithi195 ceph-mon[33773]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:01.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:00 smithi112 ceph-mon[30100]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:02 smithi148 ceph-mon[33568]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:02 smithi195 ceph-mon[33773]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:03.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:02 smithi112 ceph-mon[30100]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:05.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:04 smithi148 ceph-mon[33568]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:05.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:04 smithi195 ceph-mon[33773]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:05.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:04 smithi112 ceph-mon[30100]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:07.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:06 smithi148 ceph-mon[33568]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:07.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T10:34:07.026 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:06 smithi195 ceph-mon[33773]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T10:34:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:07.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:06 smithi112 ceph-mon[30100]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:07.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T10:34:07.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:07 smithi148 ceph-mon[33568]: Deploying daemon osd.10 on smithi148 2024-05-10T10:34:08.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:07 smithi195 ceph-mon[33773]: Deploying daemon osd.10 on smithi148 2024-05-10T10:34:08.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:07 smithi112 ceph-mon[30100]: Deploying daemon osd.10 on smithi148 2024-05-10T10:34:08.962 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:08 smithi148 ceph-mon[33568]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:08 smithi195 ceph-mon[33773]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:09.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:08 smithi112 ceph-mon[30100]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:11.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:10 smithi148 ceph-mon[33568]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:11.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:10 smithi195 ceph-mon[33773]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:11.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:10 smithi112 ceph-mon[30100]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:12.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:12 smithi148 ceph-mon[33568]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:12.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:12 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:12.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:12 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:12.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:12 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:12 smithi195 ceph-mon[33773]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:12 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:12 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:12 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:13.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:12 smithi112 ceph-mon[30100]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:13.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:12 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:13.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:12 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:13.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:12 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:15.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:14 smithi148 ceph-mon[33568]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:14 smithi195 ceph-mon[33773]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:15.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:14 smithi112 ceph-mon[30100]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.112+0000 7fb69a7fc700 1 -- 172.21.15.148:0/3697699032 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb6a40639a0 con 0x7fb6900b2900 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 10 on host 'smithi148' 2024-05-10T10:34:16.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6900b2900 msgr2=0x7fb6900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:16.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6900b2900 0x7fb6900b4dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0005950 tx=0x7fb6a00058e0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 msgr2=0x7fb6a41a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:16.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41a2cb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00b830 tx=0x7fb69c009bf0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 shutdown_connections 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6a4103120 0x7fb6a41a2cb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6a4101e00 0x7fb6a419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6900b2900 0x7fb6900b4dc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 --2- 172.21.15.148:0/3697699032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6a4102760 0x7fb6a419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 >> 172.21.15.148:0/3697699032 conn(0x7fb6a40fd9a0 msgr2=0x7fb6a40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 shutdown_connections 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-05-10T10:34:16.115+0000 7fb6ab27f700 1 -- 172.21.15.148:0/3697699032 wait complete. 2024-05-10T10:34:16.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:16.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T10:34:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:16.821 DEBUG:teuthology.orchestra.run.smithi148:osd.10> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.10.service 2024-05-10T10:34:16.823 INFO:tasks.cephadm:Deploying osd.11 on smithi195 with /dev/vg_nvme/lv_1... 2024-05-10T10:34:16.823 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T10:34:16.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:16.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:17.075 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:34:18.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:17 smithi148 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T10:34:18.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:17 smithi148 ceph-mon[33568]: from='osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:34:18.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:17 smithi148 ceph-mon[33568]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T10:34:18.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:17 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:18.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:17 smithi148 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:34:18.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:34:17 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:34:17.784+0000 7fcaf4e64700 -1 osd.10 0 waiting for initial osdmap 2024-05-10T10:34:18.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:34:17 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:34:17.796+0000 7fcaf0c57700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:34:18.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:17 smithi195 ceph-mon[33773]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T10:34:18.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:17 smithi195 ceph-mon[33773]: from='osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:34:18.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:17 smithi195 ceph-mon[33773]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T10:34:18.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:17 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:18.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:17 smithi195 ceph-mon[33773]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:34:18.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:17 smithi112 ceph-mon[30100]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T10:34:18.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:17 smithi112 ceph-mon[30100]: from='osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:34:18.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:17 smithi112 ceph-mon[30100]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T10:34:18.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:17 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:18.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:17 smithi112 ceph-mon[30100]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-05-10T10:34:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:18 smithi148 ceph-mon[33568]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:18 smithi148 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:34:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:18 smithi148 ceph-mon[33568]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T10:34:19.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:18 smithi195 ceph-mon[33773]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:18 smithi195 ceph-mon[33773]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:34:19.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:18 smithi195 ceph-mon[33773]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T10:34:19.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:18 smithi112 ceph-mon[30100]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-10T10:34:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:18 smithi112 ceph-mon[30100]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-05-10T10:34:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:18 smithi112 ceph-mon[30100]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T10:34:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.191 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T10:34:19.192 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T10:34:19.192 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-10T10:34:19.192 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-10T10:34:19.192 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00941809 s, 1.1 GB/s 2024-05-10T10:34:19.192 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-10T10:34:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:19 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:34:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:19 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:34:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:19 smithi195 ceph-mon[33773]: osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] boot 2024-05-10T10:34:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:19 smithi195 ceph-mon[33773]: osdmap e65: 11 total, 11 up, 11 in 2024-05-10T10:34:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:19 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:19 smithi195 ceph-mon[33773]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T10:34:19.904 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:19 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:34:19.905 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:19 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:34:19.905 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:19 smithi148 ceph-mon[33568]: osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] boot 2024-05-10T10:34:19.905 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:19 smithi148 ceph-mon[33568]: osdmap e65: 11 total, 11 up, 11 in 2024-05-10T10:34:19.905 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:19 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:19.905 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:19 smithi148 ceph-mon[33568]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T10:34:19.963 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_1 2024-05-10T10:34:20.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:19 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:34:20.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:19 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:34:20.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:19 smithi112 ceph-mon[30100]: osd.10 [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] boot 2024-05-10T10:34:20.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:19 smithi112 ceph-mon[30100]: osdmap e65: 11 total, 11 up, 11 in 2024-05-10T10:34:20.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:19 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T10:34:20.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:19 smithi112 ceph-mon[30100]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T10:34:20.205 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:34:21.292 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:21.292 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi148 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi148 to 4272M 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:20 smithi195 ceph-mon[33773]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi148 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi148 to 4272M 2024-05-10T10:34:21.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:21.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:21.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:20 smithi148 ceph-mon[33568]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi148 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi148 to 4272M 2024-05-10T10:34:21.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:21.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:21.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:21.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:20 smithi112 ceph-mon[30100]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T10:34:21.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.826+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/2971018402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c941017c0 msgr2=0x7f4c94101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.826+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/2971018402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c941017c0 0x7f4c94101be0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009990 tx=0x7f4c84009ca0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/2971018402 shutdown_connections 2024-05-10T10:34:21.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/2971018402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c94105140 0x7f4c9410b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/2971018402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c941017c0 0x7f4c94101be0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/2971018402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c941047a0 0x7f4c94104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/2971018402 >> 172.21.15.195:0/2971018402 conn(0x7f4c940fd100 msgr2=0x7f4c940ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/2971018402 shutdown_connections 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/2971018402 wait complete. 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 Processor -- start 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.827+0000 7f4c9a4a9700 1 -- start start 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c941017c0 0x7f4c94199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c941047a0 0x7f4c94199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c94105140 0x7f4c941a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410ee50 con 0x7f4c941017c0 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410ecd0 con 0x7f4c941047a0 2024-05-10T10:34:21.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410efd0 con 0x7f4c94105140 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c94105140 0x7f4c941a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c94105140 0x7f4c941a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:34076/0 (socket says 172.21.15.195:34076) 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 -- 172.21.15.195:0/3487012632 learned_addr learned my addr 172.21.15.195:0/3487012632 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c93fff700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c941017c0 0x7f4c94199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c937fe700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c941047a0 0x7f4c94199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 -- 172.21.15.195:0/3487012632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c941047a0 msgr2=0x7f4c94199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c941047a0 0x7f4c94199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 -- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c941017c0 msgr2=0x7f4c94199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c941017c0 0x7f4c94199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 -- 172.21.15.195:0/3487012632 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c941a11c0 con 0x7f4c94105140 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c937fe700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c941047a0 0x7f4c94199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c93fff700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c941017c0 0x7f4c94199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c98a46700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c94105140 0x7f4c941a0b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c007d40 tx=0x7f4c8c00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c917fa700 1 -- 172.21.15.195:0/3487012632 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c016020 con 0x7f4c94105140 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c941a14b0 con 0x7f4c94105140 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c917fa700 1 -- 172.21.15.195:0/3487012632 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c8c00ad60 con 0x7f4c94105140 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c917fa700 1 -- 172.21.15.195:0/3487012632 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0115e0 con 0x7f4c94105140 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.828+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c941a19f0 con 0x7f4c94105140 2024-05-10T10:34:21.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.829+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9406a190 con 0x7f4c94105140 2024-05-10T10:34:21.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.831+0000 7f4c917fa700 1 -- 172.21.15.195:0/3487012632 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4c8c019020 con 0x7f4c94105140 2024-05-10T10:34:21.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.831+0000 7f4c917fa700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4c7c0b2880 0x7f4c7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.832+0000 7f4c917fa700 1 -- 172.21.15.195:0/3487012632 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f4c8c0e2090 con 0x7f4c94105140 2024-05-10T10:34:21.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.832+0000 7f4c93fff700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4c7c0b2880 0x7f4c7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.836 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.832+0000 7f4c93fff700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4c7c0b2880 0x7f4c7c0b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4c84003820 tx=0x7f4c84005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:21.836 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:21.835+0000 7f4c917fa700 1 -- 172.21.15.195:0/3487012632 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4c8c020020 con 0x7f4c94105140 2024-05-10T10:34:22.047 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:22.045+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4c940638e0 con 0x7f4c7c0b2880 2024-05-10T10:34:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:22 smithi148 ceph-mon[33568]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T10:34:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:22 smithi148 ceph-mon[33568]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:34:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:34:23.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:34:23.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:22 smithi195 ceph-mon[33773]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T10:34:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:22 smithi195 ceph-mon[33773]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:34:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:34:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:34:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:23.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:22 smithi112 ceph-mon[30100]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T10:34:23.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:22 smithi112 ceph-mon[30100]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:34:23.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T10:34:23.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T10:34:23.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:24 smithi148 ceph-mon[33568]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-05-10T10:34:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:24 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/2159045062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]: dispatch 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:24 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]: dispatch 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:24 smithi148 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]': finished 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:24 smithi148 ceph-mon[33568]: osdmap e68: 12 total, 11 up, 12 in 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:24 smithi195 ceph-mon[33773]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:24 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/2159045062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]: dispatch 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:24 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]: dispatch 2024-05-10T10:34:25.295 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:24 smithi195 ceph-mon[33773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]': finished 2024-05-10T10:34:25.296 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:24 smithi195 ceph-mon[33773]: osdmap e68: 12 total, 11 up, 12 in 2024-05-10T10:34:25.296 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:25.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:24 smithi112 ceph-mon[30100]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-05-10T10:34:25.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:24 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/2159045062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]: dispatch 2024-05-10T10:34:25.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:24 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]: dispatch 2024-05-10T10:34:25.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:24 smithi112 ceph-mon[30100]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e59ef38d-f7e9-4721-9779-75eb13dfddcc"}]': finished 2024-05-10T10:34:25.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:24 smithi112 ceph-mon[30100]: osdmap e68: 12 total, 11 up, 12 in 2024-05-10T10:34:25.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:26.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:26 smithi148 ceph-mon[33568]: from='client.? 172.21.15.195:0/2008823846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:34:26.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:26 smithi195 ceph-mon[33773]: from='client.? 172.21.15.195:0/2008823846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:34:26.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:25 smithi112 ceph-mon[30100]: from='client.? 172.21.15.195:0/2008823846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T10:34:27.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:27 smithi148 ceph-mon[33568]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:34:27.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:27 smithi195 ceph-mon[33773]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:34:27.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:27 smithi112 ceph-mon[30100]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T10:34:29.278 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:29 smithi195 ceph-mon[33773]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:34:29.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:29 smithi148 ceph-mon[33568]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:34:29.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:29 smithi112 ceph-mon[30100]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T10:34:31.238 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:31 smithi195 ceph-mon[33773]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:34:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:31 smithi148 ceph-mon[33568]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:34:31.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:31 smithi112 ceph-mon[30100]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-10T10:34:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:33 smithi148 ceph-mon[33568]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T10:34:33.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:33.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:33 smithi195 ceph-mon[33773]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:33.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T10:34:33.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:33.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:33 smithi112 ceph-mon[30100]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:33.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T10:34:33.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:34.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:34 smithi195 ceph-mon[33773]: Deploying daemon osd.11 on smithi195 2024-05-10T10:34:34.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:34 smithi148 ceph-mon[33568]: Deploying daemon osd.11 on smithi195 2024-05-10T10:34:34.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:34 smithi112 ceph-mon[30100]: Deploying daemon osd.11 on smithi195 2024-05-10T10:34:35.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:35 smithi148 ceph-mon[33568]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:35.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:35 smithi195 ceph-mon[33773]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:35.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:35 smithi112 ceph-mon[30100]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:37.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:37 smithi148 ceph-mon[33568]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:37.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:37 smithi195 ceph-mon[33773]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:37.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:37 smithi112 ceph-mon[30100]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:39.054 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:39 smithi195 ceph-mon[33773]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:39.054 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:39.054 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:39.054 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:39.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:39 smithi148 ceph-mon[33568]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:39.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:39.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:39.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:39.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:39 smithi112 ceph-mon[30100]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:39.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:39.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:39.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:41 smithi195 ceph-mon[33773]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:41.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:41 smithi148 ceph-mon[33568]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:41.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:41 smithi112 ceph-mon[30100]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:42.317 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 11 on host 'smithi195' 2024-05-10T10:34:42.318 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.315+0000 7f4c917fa700 1 -- 172.21.15.195:0/3487012632 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4c940638e0 con 0x7f4c7c0b2880 2024-05-10T10:34:42.319 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4c7c0b2880 msgr2=0x7f4c7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4c7c0b2880 0x7f4c7c0b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4c84003820 tx=0x7f4c84005fb0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c94105140 msgr2=0x7f4c941a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c94105140 0x7f4c941a0b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c007d40 tx=0x7f4c8c00a340 comp rx=0 tx=0).stop 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 shutdown_connections 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4c7c0b2880 0x7f4c7c0b4d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c94105140 0x7f4c941a0b20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c941017c0 0x7f4c94199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 --2- 172.21.15.195:0/3487012632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c941047a0 0x7f4c94199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 >> 172.21.15.195:0/3487012632 conn(0x7f4c940fd100 msgr2=0x7f4c94107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 shutdown_connections 2024-05-10T10:34:42.320 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:42.318+0000 7f4c9a4a9700 1 -- 172.21.15.195:0/3487012632 wait complete. 2024-05-10T10:34:42.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:42.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:42.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:42.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T10:34:42.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T10:34:42.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:42.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:43.015 DEBUG:teuthology.orchestra.run.smithi195:osd.11> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.11.service 2024-05-10T10:34:43.017 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-10T10:34:43.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T10:34:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:34:44.223 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:44 smithi195 ceph-mon[33773]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T10:34:44.223 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:44 smithi195 ceph-mon[33773]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T10:34:44.224 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:44.224 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:44 smithi195 ceph-mon[33773]: from='osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:34:44.224 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:44 smithi195 ceph-mon[33773]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:34:44.224 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:34:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:34:44.035+0000 7f848f109700 -1 osd.11 0 waiting for initial osdmap 2024-05-10T10:34:44.224 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:34:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:34:44.049+0000 7f848aefc700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:34:44.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:44 smithi148 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T10:34:44.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:44 smithi148 ceph-mon[33568]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T10:34:44.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:44.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:44 smithi148 ceph-mon[33568]: from='osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:34:44.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:44 smithi148 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:34:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:44 smithi112 ceph-mon[30100]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T10:34:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:44 smithi112 ceph-mon[30100]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T10:34:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:44 smithi112 ceph-mon[30100]: from='osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:34:44.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:44 smithi112 ceph-mon[30100]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-10T10:34:45.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.041+0000 7f01836c3700 1 -- 172.21.15.112:0/1940749846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103fa0 msgr2=0x7f017c10a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.041+0000 7f01836c3700 1 --2- 172.21.15.112:0/1940749846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103fa0 0x7f017c10a850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f017400b0d0 tx=0x7f017400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.041+0000 7f01836c3700 1 -- 172.21.15.112:0/1940749846 shutdown_connections 2024-05-10T10:34:45.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.041+0000 7f01836c3700 1 --2- 172.21.15.112:0/1940749846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103fa0 0x7f017c10a850 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.042+0000 7f01836c3700 1 --2- 172.21.15.112:0/1940749846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f017c102400 0x7f017c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.042+0000 7f01836c3700 1 --2- 172.21.15.112:0/1940749846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017c103600 0x7f017c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.042+0000 7f01836c3700 1 -- 172.21.15.112:0/1940749846 >> 172.21.15.112:0/1940749846 conn(0x7f017c0fd9a0 msgr2=0x7f017c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:45.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.042+0000 7f01836c3700 1 -- 172.21.15.112:0/1940749846 shutdown_connections 2024-05-10T10:34:45.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.042+0000 7f01836c3700 1 -- 172.21.15.112:0/1940749846 wait complete. 2024-05-10T10:34:45.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.042+0000 7f01836c3700 1 Processor -- start 2024-05-10T10:34:45.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f01836c3700 1 -- start start 2024-05-10T10:34:45.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f01836c3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f017c102400 0x7f017c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f01836c3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103600 0x7f017c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f01836c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017c103fa0 0x7f017c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f01836c3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f017c10cd20 con 0x7f017c103600 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f01836c3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f017c10cba0 con 0x7f017c102400 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f01836c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f017c10cea0 con 0x7f017c103fa0 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103600 0x7f017c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103600 0x7f017c197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55506/0 (socket says 172.21.15.112:55506) 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f018145f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f017c102400 0x7f017c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 -- 172.21.15.112:0/76318922 learned_addr learned my addr 172.21.15.112:0/76318922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0181c60700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017c103fa0 0x7f017c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 -- 172.21.15.112:0/76318922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017c103fa0 msgr2=0x7f017c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017c103fa0 0x7f017c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 -- 172.21.15.112:0/76318922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f017c102400 msgr2=0x7f017c1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f017c102400 0x7f017c1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f0180c5e700 1 -- 172.21.15.112:0/76318922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c19eff0 con 0x7f017c103600 2024-05-10T10:34:45.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.043+0000 7f018145f700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f017c102400 0x7f017c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:45.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.044+0000 7f0180c5e700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103600 0x7f017c197920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f016c00d9b0 tx=0x7f016c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.045+0000 7f01727fc700 1 -- 172.21.15.112:0/76318922 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c017070 con 0x7f017c103600 2024-05-10T10:34:45.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.045+0000 7f01727fc700 1 -- 172.21.15.112:0/76318922 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f016c005020 con 0x7f017c103600 2024-05-10T10:34:45.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.045+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c19f2e0 con 0x7f017c103600 2024-05-10T10:34:45.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.045+0000 7f01727fc700 1 -- 172.21.15.112:0/76318922 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c009990 con 0x7f017c103600 2024-05-10T10:34:45.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.045+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c19f820 con 0x7f017c103600 2024-05-10T10:34:45.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.046+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c06a190 con 0x7f017c103600 2024-05-10T10:34:45.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.047+0000 7f01727fc700 1 -- 172.21.15.112:0/76318922 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f016c0051d0 con 0x7f017c103600 2024-05-10T10:34:45.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.048+0000 7f01727fc700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01680b2900 0x7f01680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.048+0000 7f018145f700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01680b2900 0x7f01680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.048+0000 7f01727fc700 1 -- 172.21.15.112:0/76318922 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f016c0e1b30 con 0x7f017c103600 2024-05-10T10:34:45.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.049+0000 7f018145f700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01680b2900 0x7f01680b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f017800bfd0 tx=0x7f017800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.053+0000 7f01727fc700 1 -- 172.21.15.112:0/76318922 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f016c020020 con 0x7f017c103600 2024-05-10T10:34:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.259+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f017c02e510 con 0x7f017c103600 2024-05-10T10:34:45.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:45 smithi112 ceph-mon[30100]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:45.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:45 smithi112 ceph-mon[30100]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:34:45.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:45 smithi112 ceph-mon[30100]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T10:34:45.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.259+0000 7f01727fc700 1 -- 172.21.15.112:0/76318922 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f016c0a9cf0 con 0x7f017c103600 2024-05-10T10:34:45.262 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:34:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01680b2900 msgr2=0x7f01680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01680b2900 0x7f01680b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f017800bfd0 tx=0x7f017800d040 comp rx=0 tx=0).stop 2024-05-10T10:34:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103600 msgr2=0x7f017c197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103600 0x7f017c197920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f016c00d9b0 tx=0x7f016c00dcc0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 shutdown_connections 2024-05-10T10:34:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01680b2900 0x7f01680b4dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017c103fa0 0x7f017c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f017c102400 0x7f017c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 --2- 172.21.15.112:0/76318922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c103600 0x7f017c197920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 >> 172.21.15.112:0/76318922 conn(0x7f017c0fd9a0 msgr2=0x7f017c106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 shutdown_connections 2024-05-10T10:34:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:45.262+0000 7f01836c3700 1 -- 172.21.15.112:0/76318922 wait complete. 2024-05-10T10:34:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:45 smithi148 ceph-mon[33568]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:45 smithi148 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:34:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:45 smithi148 ceph-mon[33568]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T10:34:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:45 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:45 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:45 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:45 smithi195 ceph-mon[33773]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:45 smithi195 ceph-mon[33773]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-10T10:34:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:45 smithi195 ceph-mon[33773]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T10:34:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:45 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:45 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:45 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:46.059 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1715337285,"num_in_osds":12,"osd_in_since":1715337264,"num_remapped_pgs":0} 2024-05-10T10:34:46.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:34:46.310 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:34:46.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:46 smithi195 ceph-mon[33773]: purged_snaps scrub starts 2024-05-10T10:34:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:46 smithi195 ceph-mon[33773]: purged_snaps scrub ok 2024-05-10T10:34:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:46 smithi195 ceph-mon[33773]: osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] boot 2024-05-10T10:34:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:46 smithi195 ceph-mon[33773]: osdmap e71: 12 total, 12 up, 12 in 2024-05-10T10:34:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:46.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:46 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/76318922' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T10:34:46.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:46 smithi112 ceph-mon[30100]: purged_snaps scrub starts 2024-05-10T10:34:46.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:46 smithi112 ceph-mon[30100]: purged_snaps scrub ok 2024-05-10T10:34:46.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:46 smithi112 ceph-mon[30100]: osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] boot 2024-05-10T10:34:46.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:46 smithi112 ceph-mon[30100]: osdmap e71: 12 total, 12 up, 12 in 2024-05-10T10:34:46.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:46.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:46 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/76318922' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T10:34:46.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:46 smithi148 ceph-mon[33568]: purged_snaps scrub starts 2024-05-10T10:34:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:46 smithi148 ceph-mon[33568]: purged_snaps scrub ok 2024-05-10T10:34:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:46 smithi148 ceph-mon[33568]: osd.11 [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] boot 2024-05-10T10:34:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:46 smithi148 ceph-mon[33568]: osdmap e71: 12 total, 12 up, 12 in 2024-05-10T10:34:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T10:34:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:46 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/76318922' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T10:34:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T10:34:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: Detected new or changed devices on smithi195 2024-05-10T10:34:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi195 to 4272M 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:47.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: Detected new or changed devices on smithi195 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi195 to 4272M 2024-05-10T10:34:47.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:47.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:47.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: Detected new or changed devices on smithi195 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi195 to 4272M 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:47.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:47.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 -- 172.21.15.112:0/829644848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334108450 msgr2=0x7f6334108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 --2- 172.21.15.112:0/829644848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334108450 0x7f6334108830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6328009990 tx=0x7f6328009ca0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 -- 172.21.15.112:0/829644848 shutdown_connections 2024-05-10T10:34:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 --2- 172.21.15.112:0/829644848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6334102d90 0x7f633410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 --2- 172.21.15.112:0/829644848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334108450 0x7f6334108830 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 --2- 172.21.15.112:0/829644848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63341023d0 0x7f6334102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 -- 172.21.15.112:0/829644848 >> 172.21.15.112:0/829644848 conn(0x7f6334078410 msgr2=0x7f6334078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 -- 172.21.15.112:0/829644848 shutdown_connections 2024-05-10T10:34:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.004+0000 7f6338b16700 1 -- 172.21.15.112:0/829644848 wait complete. 2024-05-10T10:34:48.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 Processor -- start 2024-05-10T10:34:48.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 -- start start 2024-05-10T10:34:48.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341023d0 0x7f6334199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6334102d90 0x7f633419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6334108450 0x7f633419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6334113630 con 0x7f63341023d0 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f63341134b0 con 0x7f6334102d90 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6338b16700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63341137b0 con 0x7f6334108450 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341023d0 0x7f6334199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341023d0 0x7f6334199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58492/0 (socket says 172.21.15.112:58492) 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 -- 172.21.15.112:0/2511006950 learned_addr learned my addr 172.21.15.112:0/2511006950 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 -- 172.21.15.112:0/2511006950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6334108450 msgr2=0x7f633419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6334108450 0x7f633419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 -- 172.21.15.112:0/2511006950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6334102d90 msgr2=0x7f633419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6331d9b700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6334102d90 0x7f633419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6334102d90 0x7f633419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 -- 172.21.15.112:0/2511006950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633419ed30 con 0x7f63341023d0 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f6331d9b700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6334102d90 0x7f633419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.005+0000 7f633259c700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341023d0 0x7f6334199d70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6328005910 tx=0x7f6328009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.006+0000 7f63237fe700 1 -- 172.21.15.112:0/2511006950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328027070 con 0x7f63341023d0 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.006+0000 7f63237fe700 1 -- 172.21.15.112:0/2511006950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f632800fce0 con 0x7f63341023d0 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.006+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633419efc0 con 0x7f63341023d0 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.006+0000 7f63237fe700 1 -- 172.21.15.112:0/2511006950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280173f0 con 0x7f63341023d0 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.006+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63341a68a0 con 0x7f63341023d0 2024-05-10T10:34:48.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.007+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633410a910 con 0x7f63341023d0 2024-05-10T10:34:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.008+0000 7f63237fe700 1 -- 172.21.15.112:0/2511006950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f632802a070 con 0x7f63341023d0 2024-05-10T10:34:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.008+0000 7f63237fe700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f631c0b2880 0x7f631c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.009+0000 7f6331d9b700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f631c0b2880 0x7f631c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.009+0000 7f63237fe700 1 -- 172.21.15.112:0/2511006950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f63280e3530 con 0x7f63341023d0 2024-05-10T10:34:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.009+0000 7f6331d9b700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f631c0b2880 0x7f631c0b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6334078c30 tx=0x7f6324008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.012+0000 7f63237fe700 1 -- 172.21.15.112:0/2511006950 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6328031070 con 0x7f63341023d0 2024-05-10T10:34:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.222+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f633419b080 con 0x7f63341023d0 2024-05-10T10:34:48.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.223+0000 7f63237fe700 1 -- 172.21.15.112:0/2511006950 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18492 (secure 0 0 0) 0x7f6328013070 con 0x7f63341023d0 2024-05-10T10:34:48.225 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:34:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":72,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:34:46.043024+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f631c0b2880 msgr2=0x7f631c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f631c0b2880 0x7f631c0b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6334078c30 tx=0x7f6324008040 comp rx=0 tx=0).stop 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341023d0 msgr2=0x7f6334199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341023d0 0x7f6334199d70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6328005910 tx=0x7f6328009f90 comp rx=0 tx=0).stop 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 shutdown_connections 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f631c0b2880 0x7f631c0b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6334108450 0x7f633419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341023d0 0x7f6334199d70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 --2- 172.21.15.112:0/2511006950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6334102d90 0x7f633419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 >> 172.21.15.112:0/2511006950 conn(0x7f6334078410 msgr2=0x7f6334101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 shutdown_connections 2024-05-10T10:34:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:48.226+0000 7f6338b16700 1 -- 172.21.15.112:0/2511006950 wait complete. 2024-05-10T10:34:48.872 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:30:55.269138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T10:34:48.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-10T10:34:49.115 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:34:49.236 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:49 smithi112 ceph-mon[30100]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:49.237 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:49 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2511006950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:34:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:49 smithi195 ceph-mon[33773]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:49 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2511006950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:34:49.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:49 smithi148 ceph-mon[33568]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:49.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:49 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2511006950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:34:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.056+0000 7f6492c58700 1 -- 172.21.15.112:0/4160113890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c108410 msgr2=0x7f648c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.056+0000 7f6492c58700 1 --2- 172.21.15.112:0/4160113890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c108410 0x7f648c1087f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6480009990 tx=0x7f6480009ca0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- 172.21.15.112:0/4160113890 shutdown_connections 2024-05-10T10:34:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 --2- 172.21.15.112:0/4160113890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102d60 0x7f648c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 --2- 172.21.15.112:0/4160113890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c108410 0x7f648c1087f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 --2- 172.21.15.112:0/4160113890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c1023a0 0x7f648c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- 172.21.15.112:0/4160113890 >> 172.21.15.112:0/4160113890 conn(0x7f648c079350 msgr2=0x7f648c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- 172.21.15.112:0/4160113890 shutdown_connections 2024-05-10T10:34:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- 172.21.15.112:0/4160113890 wait complete. 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 Processor -- start 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- start start 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c1023a0 0x7f648c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102d60 0x7f648c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c108410 0x7f648c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1123e0 con 0x7f648c1023a0 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f648c112260 con 0x7f648c108410 2024-05-10T10:34:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.057+0000 7f6492c58700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f648c112560 con 0x7f648c102d60 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c1023a0 0x7f648c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c1023a0 0x7f648c19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58508/0 (socket says 172.21.15.112:58508) 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 -- 172.21.15.112:0/1953833192 learned_addr learned my addr 172.21.15.112:0/1953833192 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f648bfff700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102d60 0x7f648c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64911f5700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c108410 0x7f648c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 -- 172.21.15.112:0/1953833192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102d60 msgr2=0x7f648c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102d60 0x7f648c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 -- 172.21.15.112:0/1953833192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c108410 msgr2=0x7f648c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c108410 0x7f648c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 -- 172.21.15.112:0/1953833192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c19a010 con 0x7f648c1023a0 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64911f5700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c108410 0x7f648c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f64909f4700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c1023a0 0x7f648c19f6b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64800057c0 tx=0x7f64800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f6489ffb700 1 -- 172.21.15.112:0/1953833192 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480027070 con 0x7f648c1023a0 2024-05-10T10:34:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c19a2a0 con 0x7f648c1023a0 2024-05-10T10:34:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f6489ffb700 1 -- 172.21.15.112:0/1953833192 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f648000fc00 con 0x7f648c1023a0 2024-05-10T10:34:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f6489ffb700 1 -- 172.21.15.112:0/1953833192 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480022b30 con 0x7f648c1023a0 2024-05-10T10:34:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.058+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c103fb0 con 0x7f648c1023a0 2024-05-10T10:34:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.059+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648c103ac0 con 0x7f648c1023a0 2024-05-10T10:34:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.061+0000 7f6489ffb700 1 -- 172.21.15.112:0/1953833192 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f648002a070 con 0x7f648c1023a0 2024-05-10T10:34:51.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:51 smithi112 ceph-mon[30100]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:51.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.062+0000 7f6489ffb700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f647c0b2880 0x7f647c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.062+0000 7f648bfff700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f647c0b2880 0x7f647c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.062+0000 7f6489ffb700 1 -- 172.21.15.112:0/1953833192 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f64800e2ea0 con 0x7f648c1023a0 2024-05-10T10:34:51.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.062+0000 7f648bfff700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f647c0b2880 0x7f647c0b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f648c0798e0 tx=0x7f647800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:51.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.065+0000 7f6489ffb700 1 -- 172.21.15.112:0/1953833192 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6480031080 con 0x7f648c1023a0 2024-05-10T10:34:51.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.265+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f648c1042d0 con 0x7f648c1023a0 2024-05-10T10:34:51.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.265+0000 7f6489ffb700 1 -- 172.21.15.112:0/1953833192 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f6480013070 con 0x7f648c1023a0 2024-05-10T10:34:51.267 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f647c0b2880 msgr2=0x7f647c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f647c0b2880 0x7f647c0b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f648c0798e0 tx=0x7f647800b410 comp rx=0 tx=0).stop 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c1023a0 msgr2=0x7f648c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c1023a0 0x7f648c19f6b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64800057c0 tx=0x7f64800057f0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 shutdown_connections 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c108410 0x7f648c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f648c1023a0 0x7f648c19f6b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f647c0b2880 0x7f647c0b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 --2- 172.21.15.112:0/1953833192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102d60 0x7f648c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.267+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 >> 172.21.15.112:0/1953833192 conn(0x7f648c079350 msgr2=0x7f648c06b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.268+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 shutdown_connections 2024-05-10T10:34:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:51.268+0000 7f6492c58700 1 -- 172.21.15.112:0/1953833192 wait complete. 2024-05-10T10:34:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:51 smithi195 ceph-mon[33773]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:51 smithi148 ceph-mon[33568]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:52.082 INFO:tasks.cephadm:Adding mds.a on smithi112 2024-05-10T10:34:52.082 INFO:tasks.cephadm:Adding mds.d on smithi112 2024-05-10T10:34:52.082 INFO:tasks.cephadm:Adding mds.g on smithi112 2024-05-10T10:34:52.082 INFO:tasks.cephadm:Adding mds.j on smithi112 2024-05-10T10:34:52.082 INFO:tasks.cephadm:Adding mds.b on smithi148 2024-05-10T10:34:52.082 INFO:tasks.cephadm:Adding mds.e on smithi148 2024-05-10T10:34:52.082 INFO:tasks.cephadm:Adding mds.h on smithi148 2024-05-10T10:34:52.083 INFO:tasks.cephadm:Adding mds.k on smithi148 2024-05-10T10:34:52.083 INFO:tasks.cephadm:Adding mds.c on smithi195 2024-05-10T10:34:52.083 INFO:tasks.cephadm:Adding mds.f on smithi195 2024-05-10T10:34:52.083 INFO:tasks.cephadm:Adding mds.i on smithi195 2024-05-10T10:34:52.083 INFO:tasks.cephadm:Adding mds.l on smithi195 2024-05-10T10:34:52.083 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l' 2024-05-10T10:34:52.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:52 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1953833192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:34:52.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:52 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1953833192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:34:52.335 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:34:52.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:52 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1953833192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:34:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:53 smithi195 ceph-mon[33773]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:53.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:53 smithi112 ceph-mon[30100]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:53.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:53 smithi148 ceph-mon[33568]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:53.990 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 -- 172.21.15.195:0/2307264349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 msgr2=0x7fc9ac1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/2307264349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac1031d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc994009990 tx=0x7fc994009ca0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 -- 172.21.15.195:0/2307264349 shutdown_connections 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/2307264349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ac103710 0x7fc9ac109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/2307264349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ac101b70 0x7fc9ac101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/2307264349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac1031d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 -- 172.21.15.195:0/2307264349 >> 172.21.15.195:0/2307264349 conn(0x7fc9ac0fd130 msgr2=0x7fc9ac0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 -- 172.21.15.195:0/2307264349 shutdown_connections 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.988+0000 7fc9b16ad700 1 -- 172.21.15.195:0/2307264349 wait complete. 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 Processor -- start 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 -- start start 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ac101b70 0x7fc9ac1102f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac10e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ac103710 0x7fc9ac10ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10ddb0 con 0x7fc9ac101b70 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10dc30 con 0x7fc9ac103710 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9b16ad700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10df30 con 0x7fc9ac102d70 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac10e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac10e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:41844/0 (socket says 172.21.15.195:41844) 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 -- 172.21.15.195:0/652937997 learned_addr learned my addr 172.21.15.195:0/652937997 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9aaffd700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ac101b70 0x7fc9ac1102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9ab7fe700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ac103710 0x7fc9ac10ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 -- 172.21.15.195:0/652937997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ac103710 msgr2=0x7fc9ac10ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ac103710 0x7fc9ac10ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 -- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ac101b70 msgr2=0x7fc9ac1102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ac101b70 0x7fc9ac1102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.993 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9a27fc700 1 -- 172.21.15.195:0/652937997 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac10f730 con 0x7fc9ac102d70 2024-05-10T10:34:53.993 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.989+0000 7fc9ab7fe700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ac103710 0x7fc9ac10ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:53.993 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.990+0000 7fc9a27fc700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac10e940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9940098e0 tx=0x7fc9940057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:53.993 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.990+0000 7fc9a8ff9700 1 -- 172.21.15.195:0/652937997 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994027070 con 0x7fc9ac102d70 2024-05-10T10:34:53.993 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.990+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac1a36e0 con 0x7fc9ac102d70 2024-05-10T10:34:53.993 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.990+0000 7fc9a8ff9700 1 -- 172.21.15.195:0/652937997 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc994003fe0 con 0x7fc9ac102d70 2024-05-10T10:34:53.993 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.990+0000 7fc9a8ff9700 1 -- 172.21.15.195:0/652937997 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994022d90 con 0x7fc9ac102d70 2024-05-10T10:34:53.994 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.990+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1a3b60 con 0x7fc9ac102d70 2024-05-10T10:34:53.994 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.991+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac06a190 con 0x7fc9ac102d70 2024-05-10T10:34:53.994 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.992+0000 7fc9a8ff9700 1 -- 172.21.15.195:0/652937997 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc99402a070 con 0x7fc9ac102d70 2024-05-10T10:34:53.994 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.993+0000 7fc9a8ff9700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc9980b2880 0x7fc9980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.994 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.993+0000 7fc9a8ff9700 1 -- 172.21.15.195:0/652937997 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc9940e34e0 con 0x7fc9ac102d70 2024-05-10T10:34:53.994 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.993+0000 7fc9aaffd700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc9980b2880 0x7fc9980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.995 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.994+0000 7fc9aaffd700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc9980b2880 0x7fc9980b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac102bd0 tx=0x7fc99c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:53.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:53.997+0000 7fc9a8ff9700 1 -- 172.21.15.195:0/652937997 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc994031020 con 0x7fc9ac102d70 2024-05-10T10:34:54.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.209+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}) -- 0x7fc9ac0fdfe0 con 0x7fc9980b2880 2024-05-10T10:34:54.219 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.219+0000 7fc9a8ff9700 1 -- 172.21.15.195:0/652937997 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc9ac0fdfe0 con 0x7fc9980b2880 2024-05-10T10:34:54.219 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mds.all update... 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc9980b2880 msgr2=0x7fc9980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc9980b2880 0x7fc9980b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac102bd0 tx=0x7fc99c009190 comp rx=0 tx=0).stop 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 msgr2=0x7fc9ac10e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac10e940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9940098e0 tx=0x7fc9940057c0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 shutdown_connections 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ac103710 0x7fc9ac10ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ac101b70 0x7fc9ac1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc9980b2880 0x7fc9980b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 --2- 172.21.15.195:0/652937997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102d70 0x7fc9ac10e940 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 >> 172.21.15.195:0/652937997 conn(0x7fc9ac0fd130 msgr2=0x7fc9ac0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:54.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 shutdown_connections 2024-05-10T10:34:54.223 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-10T10:34:54.221+0000 7fc9b16ad700 1 -- 172.21.15.195:0/652937997 wait complete. 2024-05-10T10:34:54.896 DEBUG:teuthology.orchestra.run.smithi112:mds.a> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.a.service 2024-05-10T10:34:54.899 DEBUG:teuthology.orchestra.run.smithi112:mds.d> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.d.service 2024-05-10T10:34:54.942 DEBUG:teuthology.orchestra.run.smithi112:mds.g> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.g.service 2024-05-10T10:34:54.985 DEBUG:teuthology.orchestra.run.smithi112:mds.j> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.j.service 2024-05-10T10:34:55.028 DEBUG:teuthology.orchestra.run.smithi148:mds.b> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.b.service 2024-05-10T10:34:55.031 DEBUG:teuthology.orchestra.run.smithi148:mds.e> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.e.service 2024-05-10T10:34:55.075 DEBUG:teuthology.orchestra.run.smithi148:mds.h> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.h.service 2024-05-10T10:34:55.118 DEBUG:teuthology.orchestra.run.smithi148:mds.k> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.k.service 2024-05-10T10:34:55.160 DEBUG:teuthology.orchestra.run.smithi195:mds.c> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.c.service 2024-05-10T10:34:55.163 DEBUG:teuthology.orchestra.run.smithi195:mds.f> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.f.service 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:34:55.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:55 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:55.186 DEBUG:teuthology.orchestra.run.smithi195:mds.i> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.i.service 2024-05-10T10:34:55.229 DEBUG:teuthology.orchestra.run.smithi195:mds.l> sudo journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.l.service 2024-05-10T10:34:55.272 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-10T10:34:55.272 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:34:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:55 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:55.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:34:55.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:55.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:34:55.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:55.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:34:55.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:34:55.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:55 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:34:56.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:56 smithi195 ceph-mon[33773]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:34:56.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:56 smithi195 ceph-mon[33773]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-10T10:34:56.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:56 smithi195 ceph-mon[33773]: Deploying daemon mds.l on smithi195 2024-05-10T10:34:56.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:56 smithi112 ceph-mon[30100]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:34:56.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:56 smithi112 ceph-mon[30100]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-10T10:34:56.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:56 smithi112 ceph-mon[30100]: Deploying daemon mds.l on smithi195 2024-05-10T10:34:56.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:56 smithi148 ceph-mon[33568]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:34:56.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:56 smithi148 ceph-mon[33568]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-10T10:34:56.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:56 smithi148 ceph-mon[33568]: Deploying daemon mds.l on smithi195 2024-05-10T10:34:57.066 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:56 smithi195 systemd[1]: Starting Ceph mds.l for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:34:57.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:57 smithi112 ceph-mon[30100]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:57.326 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:57 smithi195 ceph-mon[33773]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:57.327 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 podman[63164]: 2024-05-10 10:34:57.105605395 +0000 UTC m=+0.018194783 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:34:57.327 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 podman[63164]: 2024-05-10 10:34:57.211344801 +0000 UTC m=+0.123934189 container create 61f4e1feb90372b4fc1536ac1388085fd41e979b1241fb9c27e8069b05ea6e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 -- 172.21.15.112:0/1145338145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840108850 msgr2=0x7ff8401103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 --2- 172.21.15.112:0/1145338145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840108850 0x7ff8401103d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff83800b0d0 tx=0x7ff83800b3e0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 -- 172.21.15.112:0/1145338145 shutdown_connections 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 --2- 172.21.15.112:0/1145338145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840108850 0x7ff8401103d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 --2- 172.21.15.112:0/1145338145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff840103750 0x7ff840103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 --2- 172.21.15.112:0/1145338145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff840104100 0x7ff840108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 -- 172.21.15.112:0/1145338145 >> 172.21.15.112:0/1145338145 conn(0x7ff84007a600 msgr2=0x7ff84007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 -- 172.21.15.112:0/1145338145 shutdown_connections 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.289+0000 7ff8471f2700 1 -- 172.21.15.112:0/1145338145 wait complete. 2024-05-10T10:34:57.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 Processor -- start 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 -- start start 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840103750 0x7ff84019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff840104100 0x7ff84019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff840108850 0x7ff840199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401134f0 con 0x7ff840103750 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff840113370 con 0x7ff840108850 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff8471f2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff840113670 con 0x7ff840104100 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840103750 0x7ff84019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840103750 0x7ff84019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60960/0 (socket says 172.21.15.112:60960) 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 -- 172.21.15.112:0/3759368362 learned_addr learned my addr 172.21.15.112:0/3759368362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff83ffff700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff840104100 0x7ff84019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 -- 172.21.15.112:0/3759368362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff840104100 msgr2=0x7ff84019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff840104100 0x7ff84019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 -- 172.21.15.112:0/3759368362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff840108850 msgr2=0x7ff840199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff84578f700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff840108850 0x7ff840199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff840108850 0x7ff840199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.290+0000 7ff844f8e700 1 -- 172.21.15.112:0/3759368362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84019a050 con 0x7ff840103750 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.291+0000 7ff844f8e700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840103750 0x7ff84019f6a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff83400d980 tx=0x7ff83400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.291+0000 7ff83dffb700 1 -- 172.21.15.112:0/3759368362 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834017070 con 0x7ff840103750 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.291+0000 7ff83dffb700 1 -- 172.21.15.112:0/3759368362 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff834004830 con 0x7ff840103750 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.291+0000 7ff83dffb700 1 -- 172.21.15.112:0/3759368362 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8340053a0 con 0x7ff840103750 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.291+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84019a340 con 0x7ff840103750 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.291+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a6620 con 0x7ff840103750 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.292+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84010ba20 con 0x7ff840103750 2024-05-10T10:34:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.293+0000 7ff83dffb700 1 -- 172.21.15.112:0/3759368362 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff8340049e0 con 0x7ff840103750 2024-05-10T10:34:57.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.294+0000 7ff83dffb700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8300b2880 0x7ff8300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.294+0000 7ff83ffff700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8300b2880 0x7ff8300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.294+0000 7ff83dffb700 1 -- 172.21.15.112:0/3759368362 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7ff8340e2100 con 0x7ff840103750 2024-05-10T10:34:57.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.294+0000 7ff83ffff700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8300b2880 0x7ff8300b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff82c005ea0 tx=0x7ff82c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:57.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.297+0000 7ff83dffb700 1 -- 172.21.15.112:0/3759368362 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff834020020 con 0x7ff840103750 2024-05-10T10:34:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:57 smithi148 ceph-mon[33568]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.554+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff840104be0 con 0x7ff840103750 2024-05-10T10:34:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.554+0000 7ff83dffb700 1 -- 172.21.15.112:0/3759368362 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7ff8340aa2c0 con 0x7ff840103750 2024-05-10T10:34:57.556 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:34:57.556 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T10:34:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8300b2880 msgr2=0x7ff8300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8300b2880 0x7ff8300b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff82c005ea0 tx=0x7ff82c005e30 comp rx=0 tx=0).stop 2024-05-10T10:34:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840103750 msgr2=0x7ff84019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840103750 0x7ff84019f6a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff83400d980 tx=0x7ff83400dd40 comp rx=0 tx=0).stop 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 shutdown_connections 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff840108850 0x7ff840199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff840103750 0x7ff84019f6a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8300b2880 0x7ff8300b4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 --2- 172.21.15.112:0/3759368362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff840104100 0x7ff84019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 >> 172.21.15.112:0/3759368362 conn(0x7ff84007a600 msgr2=0x7ff840101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 shutdown_connections 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:34:57.557+0000 7ff8471f2700 1 -- 172.21.15.112:0/3759368362 wait complete. 2024-05-10T10:34:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-05-10T10:34:57.816 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 podman[63164]: 2024-05-10 10:34:57.449987102 +0000 UTC m=+0.362576490 container init 61f4e1feb90372b4fc1536ac1388085fd41e979b1241fb9c27e8069b05ea6e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-10T10:34:57.816 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 podman[63164]: 2024-05-10 10:34:57.452835125 +0000 UTC m=+0.365424522 container start 61f4e1feb90372b4fc1536ac1388085fd41e979b1241fb9c27e8069b05ea6e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-10T10:34:57.816 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 ceph-mds[63183]: -- 172.21.15.195:0/899923616 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55667bbfba40 con 0x55667bbce800 2024-05-10T10:34:57.816 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l[63179]: starting mds.l at 2024-05-10T10:34:57.816 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 bash[63164]: 61f4e1feb90372b4fc1536ac1388085fd41e979b1241fb9c27e8069b05ea6e59 2024-05-10T10:34:57.817 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:34:57 smithi195 systemd[1]: Started Ceph mds.l for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6832/38979975,v1:172.21.15.195:6833/38979975] up:boot 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: fsmap 1 up:standby 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3759368362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:34:58.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:34:58.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:58.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:34:58 smithi148 ceph-mon[33568]: Deploying daemon mds.k on smithi148 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6832/38979975,v1:172.21.15.195:6833/38979975] up:boot 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: fsmap 1 up:standby 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3759368362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:34:58.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:34:58.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:58.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:34:58 smithi195 ceph-mon[33773]: Deploying daemon mds.k on smithi148 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6832/38979975,v1:172.21.15.195:6833/38979975] up:boot 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: fsmap 1 up:standby 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3759368362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:34:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:34:58.831 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:34:58 smithi112 ceph-mon[30100]: Deploying daemon mds.k on smithi148 2024-05-10T10:35:00.044 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:34:59 smithi148 systemd[1]: Starting Ceph mds.k for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:00.224 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:00.474 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:00.487 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 podman[62927]: 2024-05-10 10:35:00.066794359 +0000 UTC m=+0.018470127 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:00.487 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 podman[62927]: 2024-05-10 10:35:00.255949202 +0000 UTC m=+0.207624994 container create 3e3306709d8f20aed6ec03e13af570d01b11b962cf3db8de1d5b6b417beaca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-10T10:35:00.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:00 smithi112 ceph-mon[30100]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:00.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:00 smithi148 ceph-mon[33568]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:00.794 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 podman[62927]: 2024-05-10 10:35:00.500332392 +0000 UTC m=+0.452008150 container init 3e3306709d8f20aed6ec03e13af570d01b11b962cf3db8de1d5b6b417beaca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-10T10:35:00.794 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 podman[62927]: 2024-05-10 10:35:00.503549034 +0000 UTC m=+0.455224787 container start 3e3306709d8f20aed6ec03e13af570d01b11b962cf3db8de1d5b6b417beaca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD) 2024-05-10T10:35:00.795 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 ceph-mds[62946]: -- 172.21.15.148:0/540095455 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582810e1a40 con 0x5582810b5000 2024-05-10T10:35:00.795 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k[62942]: starting mds.k at 2024-05-10T10:35:00.795 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 bash[62927]: 3e3306709d8f20aed6ec03e13af570d01b11b962cf3db8de1d5b6b417beaca40 2024-05-10T10:35:00.795 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:35:00 smithi148 systemd[1]: Started Ceph mds.k for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:00 smithi195 ceph-mon[33773]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:01.895 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:01.895 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: Deploying daemon mds.j on smithi112 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6832/3726517324,v1:172.21.15.148:6833/3726517324] up:boot 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: fsmap 2 up:standby 2024-05-10T10:35:01.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:01 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T10:35:01.920 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:01.920 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:01.920 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:01.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:01.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:01.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:01.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: Deploying daemon mds.j on smithi112 2024-05-10T10:35:01.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6832/3726517324,v1:172.21.15.148:6833/3726517324] up:boot 2024-05-10T10:35:01.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: fsmap 2 up:standby 2024-05-10T10:35:01.921 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:01 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T10:35:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: Deploying daemon mds.j on smithi112 2024-05-10T10:35:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6832/3726517324,v1:172.21.15.148:6833/3726517324] up:boot 2024-05-10T10:35:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: fsmap 2 up:standby 2024-05-10T10:35:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:01 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T10:35:02.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:02 smithi112 ceph-mon[30100]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:02 smithi148 ceph-mon[33568]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:02 smithi195 ceph-mon[33773]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 -- 172.21.15.112:0/344966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06c3d0 msgr2=0x7f0a1c110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 --2- 172.21.15.112:0/344966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06c3d0 0x7f0a1c110320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400b0a0 tx=0x7f0a1400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 -- 172.21.15.112:0/344966017 shutdown_connections 2024-05-10T10:35:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 --2- 172.21.15.112:0/344966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06c3d0 0x7f0a1c110320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 --2- 172.21.15.112:0/344966017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a1c109500 0x7f0a1c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 --2- 172.21.15.112:0/344966017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a1c06ba10 0x7f0a1c06be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 -- 172.21.15.112:0/344966017 >> 172.21.15.112:0/344966017 conn(0x7f0a1c079310 msgr2=0x7f0a1c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 -- 172.21.15.112:0/344966017 shutdown_connections 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 -- 172.21.15.112:0/344966017 wait complete. 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 Processor -- start 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.503+0000 7f0a2329e700 1 -- start start 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06ba10 0x7f0a1c102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a1c06c3d0 0x7f0a1c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a1c109500 0x7f0a1c102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c113540 con 0x7f0a1c06ba10 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1133c0 con 0x7f0a1c109500 2024-05-10T10:35:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1136c0 con 0x7f0a1c06c3d0 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06ba10 0x7f0a1c102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06ba10 0x7f0a1c102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60968/0 (socket says 172.21.15.112:60968) 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 -- 172.21.15.112:0/1810646112 learned_addr learned my addr 172.21.15.112:0/1810646112 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a20839700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a1c06c3d0 0x7f0a1c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2183b700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a1c109500 0x7f0a1c102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 -- 172.21.15.112:0/1810646112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a1c06c3d0 msgr2=0x7f0a1c1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a1c06c3d0 0x7f0a1c1029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 -- 172.21.15.112:0/1810646112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a1c109500 msgr2=0x7f0a1c102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a1c109500 0x7f0a1c102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 -- 172.21.15.112:0/1810646112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c1a7fe0 con 0x7f0a1c06ba10 2024-05-10T10:35:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2103a700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06ba10 0x7f0a1c102460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c920 tx=0x7f0a1800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a127fc700 1 -- 172.21.15.112:0/1810646112 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18007b00 con 0x7f0a1c06ba10 2024-05-10T10:35:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a127fc700 1 -- 172.21.15.112:0/1810646112 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a18004cc0 con 0x7f0a1c06ba10 2024-05-10T10:35:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a127fc700 1 -- 172.21.15.112:0/1810646112 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a180054f0 con 0x7f0a1c06ba10 2024-05-10T10:35:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c1a8270 con 0x7f0a1c06ba10 2024-05-10T10:35:03.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.504+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c1a8760 con 0x7f0a1c06ba10 2024-05-10T10:35:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.505+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1c06a190 con 0x7f0a1c06ba10 2024-05-10T10:35:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.507+0000 7f0a127fc700 1 -- 172.21.15.112:0/1810646112 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0a18007ca0 con 0x7f0a1c06ba10 2024-05-10T10:35:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.507+0000 7f0a127fc700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0a080b2880 0x7f0a080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.507+0000 7f0a20839700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0a080b2880 0x7f0a080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.508+0000 7f0a127fc700 1 -- 172.21.15.112:0/1810646112 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0a180e19b0 con 0x7f0a1c06ba10 2024-05-10T10:35:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.508+0000 7f0a20839700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0a080b2880 0x7f0a080b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006fd0 tx=0x7f0a0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.512+0000 7f0a127fc700 1 -- 172.21.15.112:0/1810646112 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0a18014030 con 0x7f0a1c06ba10 2024-05-10T10:35:03.758 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:03 smithi112 systemd[1]: Starting Ceph mds.j for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:03.790 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:03.790 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T10:35:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.787+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a1c02e6d0 con 0x7f0a1c06ba10 2024-05-10T10:35:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.788+0000 7f0a127fc700 1 -- 172.21.15.112:0/1810646112 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f0a180a9b70 con 0x7f0a1c06ba10 2024-05-10T10:35:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0a080b2880 msgr2=0x7f0a080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0a080b2880 0x7f0a080b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006fd0 tx=0x7f0a0c008040 comp rx=0 tx=0).stop 2024-05-10T10:35:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06ba10 msgr2=0x7f0a1c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06ba10 0x7f0a1c102460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c920 tx=0x7f0a1800cce0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 shutdown_connections 2024-05-10T10:35:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a1c109500 0x7f0a1c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a1c06ba10 0x7f0a1c102460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0a080b2880 0x7f0a080b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 --2- 172.21.15.112:0/1810646112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a1c06c3d0 0x7f0a1c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 >> 172.21.15.112:0/1810646112 conn(0x7f0a1c079310 msgr2=0x7f0a1c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 shutdown_connections 2024-05-10T10:35:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:03.791+0000 7f0a2329e700 1 -- 172.21.15.112:0/1810646112 wait complete. 2024-05-10T10:35:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-05-10T10:35:04.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:04 smithi112 ceph-mon[30100]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:04.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:04 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1810646112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:04 smithi195 ceph-mon[33773]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:04 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1810646112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:04 smithi148 ceph-mon[33568]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:04 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1810646112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:05.080 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:04 smithi112 podman[67873]: 2024-05-10 10:35:04.717086524 +0000 UTC m=+0.018426116 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:05.080 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:04 smithi112 podman[67873]: 2024-05-10 10:35:04.847176058 +0000 UTC m=+0.148515634 container create 40dc4b06be95cb7d5bf266f4f1a4feff12caedff47889332a94595fe9b9a20e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-10T10:35:05.580 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:05 smithi112 podman[67873]: 2024-05-10 10:35:05.11864829 +0000 UTC m=+0.419987869 container init 40dc4b06be95cb7d5bf266f4f1a4feff12caedff47889332a94595fe9b9a20e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:35:05.580 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:05 smithi112 podman[67873]: 2024-05-10 10:35:05.121570453 +0000 UTC m=+0.422910030 container start 40dc4b06be95cb7d5bf266f4f1a4feff12caedff47889332a94595fe9b9a20e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-10T10:35:05.580 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:05 smithi112 ceph-mds[67893]: -- 172.21.15.112:0/1485543239 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56067e257a40 con 0x56067e22ac00 2024-05-10T10:35:05.580 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:05 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j[67889]: starting mds.j at 2024-05-10T10:35:05.580 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:05 smithi112 bash[67873]: 40dc4b06be95cb7d5bf266f4f1a4feff12caedff47889332a94595fe9b9a20e3 2024-05-10T10:35:05.580 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:35:05 smithi112 systemd[1]: Started Ceph mds.j for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:06.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: Deploying daemon mds.d on smithi112 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.112:6834/441557684,v1:172.21.15.112:6835/441557684] up:boot 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: fsmap 3 up:standby 2024-05-10T10:35:06.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:06 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T10:35:06.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:06.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:06.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:06.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:06.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: Deploying daemon mds.d on smithi112 2024-05-10T10:35:06.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.112:6834/441557684,v1:172.21.15.112:6835/441557684] up:boot 2024-05-10T10:35:06.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: fsmap 3 up:standby 2024-05-10T10:35:06.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:06 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T10:35:06.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: Deploying daemon mds.d on smithi112 2024-05-10T10:35:06.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.112:6834/441557684,v1:172.21.15.112:6835/441557684] up:boot 2024-05-10T10:35:06.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: fsmap 3 up:standby 2024-05-10T10:35:06.581 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:06 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T10:35:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:08.401 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:08 smithi112 ceph-mon[30100]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:08.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:08 smithi148 ceph-mon[33568]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:08.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:08 smithi195 ceph-mon[33773]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2377416585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103f00 msgr2=0x7f73f81103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2377416585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103f00 0x7f73f81103b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f73f000b0d0 tx=0x7f73f000b3e0 comp rx=0 tx=0).stop 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2377416585 shutdown_connections 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2377416585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103f00 0x7f73f81103b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2377416585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73f81095e0 0x7f73f81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2377416585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f8103540 0x7f73f81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2377416585 >> 172.21.15.112:0/2377416585 conn(0x7f73f807a650 msgr2=0x7f73f807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2377416585 shutdown_connections 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.983+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2377416585 wait complete. 2024-05-10T10:35:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 Processor -- start 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 -- start start 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103540 0x7f73f80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f8103f00 0x7f73f8075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73f81095e0 0x7f73f8076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8113670 con 0x7f73f8103540 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81134f0 con 0x7f73f81095e0 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.984+0000 7f73fe6c4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81137f0 con 0x7f73f8103f00 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103540 0x7f73f80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103540 0x7f73f80757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35050/0 (socket says 172.21.15.112:35050) 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 -- 172.21.15.112:0/2934881656 learned_addr learned my addr 172.21.15.112:0/2934881656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73effff700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f8103f00 0x7f73f8075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73fcc61700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73f81095e0 0x7f73f8076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 -- 172.21.15.112:0/2934881656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f8103f00 msgr2=0x7f73f8075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f8103f00 0x7f73f8075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 -- 172.21.15.112:0/2934881656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73f81095e0 msgr2=0x7f73f8076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73f81095e0 0x7f73f8076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 -- 172.21.15.112:0/2934881656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f81a8030 con 0x7f73f8103540 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f7fff700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103540 0x7f73f80757a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b530 tx=0x7f73e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f5ffb700 1 -- 172.21.15.112:0/2934881656 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8015020 con 0x7f73f8103540 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f5ffb700 1 -- 172.21.15.112:0/2934881656 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73e8011cc0 con 0x7f73f8103540 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f81a8320 con 0x7f73f8103540 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73f5ffb700 1 -- 172.21.15.112:0/2934881656 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8010c80 con 0x7f73f8103540 2024-05-10T10:35:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.985+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f81a88b0 con 0x7f73f8103540 2024-05-10T10:35:08.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.986+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f810ba70 con 0x7f73f8103540 2024-05-10T10:35:08.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.988+0000 7f73f5ffb700 1 -- 172.21.15.112:0/2934881656 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f73e8018020 con 0x7f73f8103540 2024-05-10T10:35:08.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.989+0000 7f73f5ffb700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73e40b2900 0x7f73e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.989+0000 7f73effff700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73e40b2900 0x7f73e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.989+0000 7f73f5ffb700 1 -- 172.21.15.112:0/2934881656 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f73e80e1960 con 0x7f73f8103540 2024-05-10T10:35:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.989+0000 7f73effff700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73e40b2900 0x7f73e40b4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73f807ac50 tx=0x7f73e0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:08.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:08.992+0000 7f73f5ffb700 1 -- 172.21.15.112:0/2934881656 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f73e801f020 con 0x7f73f8103540 2024-05-10T10:35:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.249+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73f806a190 con 0x7f73f8103540 2024-05-10T10:35:09.251 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:35:09 smithi112 podman[68288]: 2024-05-10 10:35:09.156207662 +0000 UTC m=+0.699715775 container create 1f93a651e3f46cef0712f063730ffa51c946cfea4b9b5700144eac59cf1ee7e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-10T10:35:09.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.250+0000 7f73f5ffb700 1 -- 172.21.15.112:0/2934881656 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f73e80a9b20 con 0x7f73f8103540 2024-05-10T10:35:09.252 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:09.252 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T10:35:09.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73e40b2900 msgr2=0x7f73e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73e40b2900 0x7f73e40b4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73f807ac50 tx=0x7f73e0009190 comp rx=0 tx=0).stop 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103540 msgr2=0x7f73f80757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103540 0x7f73f80757a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b530 tx=0x7f73e800b840 comp rx=0 tx=0).stop 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 shutdown_connections 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73f81095e0 0x7f73f8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f8103540 0x7f73f80757a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73e40b2900 0x7f73e40b4dc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 --2- 172.21.15.112:0/2934881656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f8103f00 0x7f73f8075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 >> 172.21.15.112:0/2934881656 conn(0x7f73f807a650 msgr2=0x7f73f8102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:09.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 shutdown_connections 2024-05-10T10:35:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:09.252+0000 7f73fe6c4700 1 -- 172.21.15.112:0/2934881656 wait complete. 2024-05-10T10:35:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 4 2024-05-10T10:35:09.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:09 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2934881656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:09.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:09 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2934881656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:09.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:09 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2934881656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:10.276 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:35:10 smithi112 podman[68288]: 2024-05-10 10:35:10.030272068 +0000 UTC m=+1.573780186 container init 1f93a651e3f46cef0712f063730ffa51c946cfea4b9b5700144eac59cf1ee7e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:35:10.276 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:35:10 smithi112 podman[68288]: 2024-05-10 10:35:10.033281143 +0000 UTC m=+1.576789255 container start 1f93a651e3f46cef0712f063730ffa51c946cfea4b9b5700144eac59cf1ee7e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507) 2024-05-10T10:35:10.276 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mds[68358]: -- 172.21.15.112:0/3845072148 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56116717ba40 con 0x56116714ec00 2024-05-10T10:35:10.276 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:35:10 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d[68346]: starting mds.d at 2024-05-10T10:35:10.276 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:35:10 smithi112 bash[68288]: 1f93a651e3f46cef0712f063730ffa51c946cfea4b9b5700144eac59cf1ee7e1 2024-05-10T10:35:10.276 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:35:10 smithi112 systemd[1]: Started Ceph mds.d for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:10 smithi148 ceph-mon[33568]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:10 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:10 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:10 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:10 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:10 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:10 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:10 smithi195 ceph-mon[33773]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:10 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:10 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:10 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:10 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:10 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:10 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mon[30100]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:10 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:11 smithi148 ceph-mon[33568]: Deploying daemon mds.e on smithi148 2024-05-10T10:35:11.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:11 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.112:6836/2497530637,v1:172.21.15.112:6837/2497530637] up:boot 2024-05-10T10:35:11.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:11 smithi148 ceph-mon[33568]: fsmap 4 up:standby 2024-05-10T10:35:11.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:11 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T10:35:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:11 smithi195 ceph-mon[33773]: Deploying daemon mds.e on smithi148 2024-05-10T10:35:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:11 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.112:6836/2497530637,v1:172.21.15.112:6837/2497530637] up:boot 2024-05-10T10:35:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:11 smithi195 ceph-mon[33773]: fsmap 4 up:standby 2024-05-10T10:35:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:11 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T10:35:11.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:11 smithi112 ceph-mon[30100]: Deploying daemon mds.e on smithi148 2024-05-10T10:35:11.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:11 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.112:6836/2497530637,v1:172.21.15.112:6837/2497530637] up:boot 2024-05-10T10:35:11.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:11 smithi112 ceph-mon[30100]: fsmap 4 up:standby 2024-05-10T10:35:11.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:11 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T10:35:11.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:12.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:12 smithi148 ceph-mon[33568]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:12.544 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:12 smithi148 systemd[1]: Starting Ceph mds.e for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:12.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:12 smithi195 ceph-mon[33773]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:12.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:12 smithi112 ceph-mon[30100]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:13.045 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:12 smithi148 podman[63249]: 2024-05-10 10:35:12.653601769 +0000 UTC m=+0.018261180 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:13.045 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:12 smithi148 podman[63249]: 2024-05-10 10:35:12.77806426 +0000 UTC m=+0.142723649 container create 567a75ddddc06503480089e31fe45e60676a939cd138bdcfa7a2c1aa8cddf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-10T10:35:13.545 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:13 smithi148 podman[63249]: 2024-05-10 10:35:13.048236257 +0000 UTC m=+0.412895657 container init 567a75ddddc06503480089e31fe45e60676a939cd138bdcfa7a2c1aa8cddf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-10T10:35:13.545 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:13 smithi148 podman[63249]: 2024-05-10 10:35:13.051260605 +0000 UTC m=+0.415920003 container start 567a75ddddc06503480089e31fe45e60676a939cd138bdcfa7a2c1aa8cddf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-10T10:35:13.545 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:13 smithi148 ceph-mds[63269]: -- 172.21.15.148:0/2578149836 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564561e4fa40 con 0x564561e22800 2024-05-10T10:35:13.545 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:13 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e[63264]: starting mds.e at 2024-05-10T10:35:13.545 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:13 smithi148 bash[63249]: 567a75ddddc06503480089e31fe45e60676a939cd138bdcfa7a2c1aa8cddf4fe 2024-05-10T10:35:13.545 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:35:13 smithi148 systemd[1]: Started Ceph mds.e for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:13.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.834+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1683825271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe006bc10 msgr2=0x7fbfe006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.834+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1683825271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe006bc10 0x7fbfe006c090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009990 tx=0x7fbfd4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1683825271 shutdown_connections 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1683825271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe006c5d0 0x7fbfe0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1683825271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfe0109510 0x7fbfe01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1683825271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe006bc10 0x7fbfe006c090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1683825271 >> 172.21.15.112:0/1683825271 conn(0x7fbfe0079350 msgr2=0x7fbfe0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1683825271 shutdown_connections 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1683825271 wait complete. 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 Processor -- start 2024-05-10T10:35:13.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.835+0000 7fbfe5f43700 1 -- start start 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe006bc10 0x7fbfe019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfe006c5d0 0x7fbfe019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe0109510 0x7fbfe0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01134d0 con 0x7fbfe0109510 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe0113350 con 0x7fbfe006bc10 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe0113650 con 0x7fbfe006c5d0 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe0109510 0x7fbfe0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe0109510 0x7fbfe0199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35094/0 (socket says 172.21.15.112:35094) 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfe006c5d0 0x7fbfe019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 -- 172.21.15.112:0/1938927550 learned_addr learned my addr 172.21.15.112:0/1938927550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdf7fe700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe006bc10 0x7fbfe019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 -- 172.21.15.112:0/1938927550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfe006c5d0 msgr2=0x7fbfe019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:13.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfe006c5d0 0x7fbfe019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 -- 172.21.15.112:0/1938927550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe006bc10 msgr2=0x7fbfe019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe006bc10 0x7fbfe019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 -- 172.21.15.112:0/1938927550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe019a010 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdffff700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe0109510 0x7fbfe0199830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8007d70 tx=0x7fbfd800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdcff9700 1 -- 172.21.15.112:0/1938927550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8016020 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe019a300 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdcff9700 1 -- 172.21.15.112:0/1938927550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfd800ad40 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfdcff9700 1 -- 172.21.15.112:0/1938927550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80115e0 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.836+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe01a6510 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.837+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe0104bd0 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.839+0000 7fbfdcff9700 1 -- 172.21.15.112:0/1938927550 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbfd8019020 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.840+0000 7fbfdcff9700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbfc80b2880 0x7fbfc80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.840+0000 7fbfdf7fe700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbfc80b2880 0x7fbfc80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.840+0000 7fbfdcff9700 1 -- 172.21.15.112:0/1938927550 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbfd80e2200 con 0x7fbfe0109510 2024-05-10T10:35:13.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.840+0000 7fbfdf7fe700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbfc80b2880 0x7fbfc80b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0005950 tx=0x7fbfd00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:13.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:13.844+0000 7fbfdcff9700 1 -- 172.21.15.112:0/1938927550 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbfd8020020 con 0x7fbfe0109510 2024-05-10T10:35:14.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.091+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfe002e510 con 0x7fbfe0109510 2024-05-10T10:35:14.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.092+0000 7fbfdcff9700 1 -- 172.21.15.112:0/1938927550 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fbfd8055e20 con 0x7fbfe0109510 2024-05-10T10:35:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:14.093 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T10:35:14.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbfc80b2880 msgr2=0x7fbfc80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:14.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbfc80b2880 0x7fbfc80b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0005950 tx=0x7fbfd00058e0 comp rx=0 tx=0).stop 2024-05-10T10:35:14.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe0109510 msgr2=0x7fbfe0199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe0109510 0x7fbfe0199830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8007d70 tx=0x7fbfd800a340 comp rx=0 tx=0).stop 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 shutdown_connections 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe0109510 0x7fbfe0199830 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe006bc10 0x7fbfe019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbfc80b2880 0x7fbfc80b4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 --2- 172.21.15.112:0/1938927550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfe006c5d0 0x7fbfe019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 >> 172.21.15.112:0/1938927550 conn(0x7fbfe0079350 msgr2=0x7fbfe0101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 shutdown_connections 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:14.094+0000 7fbfe5f43700 1 -- 172.21.15.112:0/1938927550 wait complete. 2024-05-10T10:35:14.096 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 6 2024-05-10T10:35:14.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:14.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:14.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:14.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: Deploying daemon mds.f on smithi195 2024-05-10T10:35:14.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:14.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6834/2643629917,v1:172.21.15.148:6835/2643629917] up:boot 2024-05-10T10:35:14.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: fsmap 5 up:standby 2024-05-10T10:35:14.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T10:35:14.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:14 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1938927550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:14.382 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.382 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: Deploying daemon mds.f on smithi195 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6834/2643629917,v1:172.21.15.148:6835/2643629917] up:boot 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: fsmap 5 up:standby 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T10:35:14.383 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:14 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1938927550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:14.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: Deploying daemon mds.f on smithi195 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6834/2643629917,v1:172.21.15.148:6835/2643629917] up:boot 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: fsmap 5 up:standby 2024-05-10T10:35:14.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T10:35:14.460 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:14 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1938927550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:15.566 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 systemd[1]: Starting Ceph mds.f for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:15.830 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 podman[63485]: 2024-05-10 10:35:15.478342692 +0000 UTC m=+0.018356146 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:15.831 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 podman[63485]: 2024-05-10 10:35:15.593708804 +0000 UTC m=+0.133722237 container create 3f98fe838b65941f25f7580a5aec9693e60b8ba9bc5ca46e53a7977de4ff3c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T10:35:15.831 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 podman[63485]: 2024-05-10 10:35:15.830036857 +0000 UTC m=+0.370050298 container init 3f98fe838b65941f25f7580a5aec9693e60b8ba9bc5ca46e53a7977de4ff3c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-10T10:35:16.316 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 podman[63485]: 2024-05-10 10:35:15.833108512 +0000 UTC m=+0.373121954 container start 3f98fe838b65941f25f7580a5aec9693e60b8ba9bc5ca46e53a7977de4ff3c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-05-10T10:35:16.316 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 ceph-mds[63504]: -- 172.21.15.195:0/632026770 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f79aaaba40 con 0x55f79aa7e800 2024-05-10T10:35:16.316 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f[63500]: starting mds.f at 2024-05-10T10:35:16.316 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 bash[63485]: 3f98fe838b65941f25f7580a5aec9693e60b8ba9bc5ca46e53a7977de4ff3c9c 2024-05-10T10:35:16.316 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:35:15 smithi195 systemd[1]: Started Ceph mds.f for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:16.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:17.198 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: Deploying daemon mds.c on smithi195 2024-05-10T10:35:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] up:boot 2024-05-10T10:35:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: fsmap 6 up:standby 2024-05-10T10:35:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: Deploying daemon mds.c on smithi195 2024-05-10T10:35:17.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] up:boot 2024-05-10T10:35:17.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: fsmap 6 up:standby 2024-05-10T10:35:17.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T10:35:17.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:17.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:17.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:17.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:17.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:17.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: Deploying daemon mds.c on smithi195 2024-05-10T10:35:17.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] up:boot 2024-05-10T10:35:17.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: fsmap 6 up:standby 2024-05-10T10:35:17.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T10:35:18.316 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 systemd[1]: Starting Ceph mds.c for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:18.588 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 podman[63795]: 2024-05-10 10:35:18.33529708 +0000 UTC m=+0.104438518 container create edd25aea5ac33d12594dfde00b260f41f348e063fffa0484dfb8b1d2641471b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-10T10:35:18.589 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 podman[63795]: 2024-05-10 10:35:18.248927629 +0000 UTC m=+0.018069094 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:18.589 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 podman[63795]: 2024-05-10 10:35:18.561277023 +0000 UTC m=+0.330418471 container init edd25aea5ac33d12594dfde00b260f41f348e063fffa0484dfb8b1d2641471b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-10T10:35:18.589 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 podman[63795]: 2024-05-10 10:35:18.564046827 +0000 UTC m=+0.333188264 container start edd25aea5ac33d12594dfde00b260f41f348e063fffa0484dfb8b1d2641471b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-05-10T10:35:18.589 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mds[63815]: -- 172.21.15.195:0/2570872385 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572a6809a40 con 0x5572a67dd000 2024-05-10T10:35:18.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 -- 172.21.15.112:0/726346452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14102390 msgr2=0x7f5d14102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/726346452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14102390 0x7f5d14102810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009990 tx=0x7f5d08009ca0 comp rx=0 tx=0).stop 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 -- 172.21.15.112:0/726346452 shutdown_connections 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/726346452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d14102d50 0x7f5d1410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/726346452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14108400 0x7f5d141087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/726346452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14102390 0x7f5d14102810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 -- 172.21.15.112:0/726346452 >> 172.21.15.112:0/726346452 conn(0x7f5d14079310 msgr2=0x7f5d14079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 -- 172.21.15.112:0/726346452 shutdown_connections 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.894+0000 7f5d18ae8700 1 -- 172.21.15.112:0/726346452 wait complete. 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 Processor -- start 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 -- start start 2024-05-10T10:35:18.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14102390 0x7f5d1419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d14102d50 0x7f5d1419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14108400 0x7f5d141998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112400 con 0x7f5d14108400 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112280 con 0x7f5d14102390 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d18ae8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112580 con 0x7f5d14102d50 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d12d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14108400 0x7f5d141998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d12d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14108400 0x7f5d141998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34506/0 (socket says 172.21.15.112:34506) 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d12d9d700 1 -- 172.21.15.112:0/381026855 learned_addr learned my addr 172.21.15.112:0/381026855 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.895+0000 7f5d11d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d14102d50 0x7f5d1419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d1259c700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14102390 0x7f5d1419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d12d9d700 1 -- 172.21.15.112:0/381026855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d14102d50 msgr2=0x7f5d1419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d12d9d700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d14102d50 0x7f5d1419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:18.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d12d9d700 1 -- 172.21.15.112:0/381026855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14102390 msgr2=0x7f5d1419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d12d9d700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14102390 0x7f5d1419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d12d9d700 1 -- 172.21.15.112:0/381026855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1419a030 con 0x7f5d14108400 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d12d9d700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14108400 0x7f5d141998d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c007ee0 tx=0x7f5d0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d037fe700 1 -- 172.21.15.112:0/381026855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c004bb0 con 0x7f5d14108400 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d037fe700 1 -- 172.21.15.112:0/381026855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d0c004d50 con 0x7f5d14108400 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1419a320 con 0x7f5d14108400 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d037fe700 1 -- 172.21.15.112:0/381026855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0058b0 con 0x7f5d14108400 2024-05-10T10:35:18.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.896+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d141a6590 con 0x7f5d14108400 2024-05-10T10:35:18.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.897+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d1406a190 con 0x7f5d14108400 2024-05-10T10:35:18.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.898+0000 7f5d037fe700 1 -- 172.21.15.112:0/381026855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f5d0c01a020 con 0x7f5d14108400 2024-05-10T10:35:18.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.899+0000 7f5d037fe700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5cfc0b2880 0x7f5cfc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:18.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.899+0000 7f5d1259c700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5cfc0b2880 0x7f5cfc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:18.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.899+0000 7f5d037fe700 1 -- 172.21.15.112:0/381026855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f5d0c0e1400 con 0x7f5d14108400 2024-05-10T10:35:18.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.899+0000 7f5d1259c700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5cfc0b2880 0x7f5cfc0b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d14100c30 tx=0x7f5d0400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:18.903+0000 7f5d037fe700 1 -- 172.21.15.112:0/381026855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5d0c010070 con 0x7f5d14108400 2024-05-10T10:35:18.967 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: starting mds.c at 2024-05-10T10:35:18.967 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 bash[63795]: edd25aea5ac33d12594dfde00b260f41f348e063fffa0484dfb8b1d2641471b6 2024-05-10T10:35:18.967 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:35:18 smithi195 systemd[1]: Started Ceph mds.c for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:19.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.166+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d141a6760 con 0x7f5d14108400 2024-05-10T10:35:19.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.166+0000 7f5d037fe700 1 -- 172.21.15.112:0/381026855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f5d0c0a95c0 con 0x7f5d14108400 2024-05-10T10:35:19.168 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:19.168 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-10T10:35:19.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5cfc0b2880 msgr2=0x7f5cfc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:19.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5cfc0b2880 0x7f5cfc0b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d14100c30 tx=0x7f5d0400b410 comp rx=0 tx=0).stop 2024-05-10T10:35:19.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14108400 msgr2=0x7f5d141998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14108400 0x7f5d141998d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c007ee0 tx=0x7f5d0c00d3b0 comp rx=0 tx=0).stop 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 shutdown_connections 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d14108400 0x7f5d141998d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14102390 0x7f5d1419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5cfc0b2880 0x7f5cfc0b4d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 --2- 172.21.15.112:0/381026855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d14102d50 0x7f5d1419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 >> 172.21.15.112:0/381026855 conn(0x7f5d14079310 msgr2=0x7f5d14101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 shutdown_connections 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:19.169+0000 7f5d18ae8700 1 -- 172.21.15.112:0/381026855 wait complete. 2024-05-10T10:35:19.171 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-05-10T10:35:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:18 smithi148 ceph-mon[33568]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:18 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mon[33773]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:18 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:19.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:18 smithi112 ceph-mon[30100]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:19.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:19.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:19.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:19.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:18 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:20.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:19 smithi148 ceph-mon[33568]: Deploying daemon mds.h on smithi148 2024-05-10T10:35:20.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:19 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] up:boot 2024-05-10T10:35:20.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:19 smithi148 ceph-mon[33568]: fsmap 7 up:standby 2024-05-10T10:35:20.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:19 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T10:35:20.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:19 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/381026855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:20.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:19 smithi195 ceph-mon[33773]: Deploying daemon mds.h on smithi148 2024-05-10T10:35:20.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:19 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] up:boot 2024-05-10T10:35:20.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:19 smithi195 ceph-mon[33773]: fsmap 7 up:standby 2024-05-10T10:35:20.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:19 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T10:35:20.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:19 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/381026855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:20.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:19 smithi112 ceph-mon[30100]: Deploying daemon mds.h on smithi148 2024-05-10T10:35:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:19 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] up:boot 2024-05-10T10:35:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:19 smithi112 ceph-mon[30100]: fsmap 7 up:standby 2024-05-10T10:35:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:19 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T10:35:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:19 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/381026855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:20.980 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:20 smithi148 ceph-mon[33568]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:21.294 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 systemd[1]: Starting Ceph mds.h for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:20 smithi195 ceph-mon[33773]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:21.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:20 smithi112 ceph-mon[30100]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:21.715 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 podman[63574]: 2024-05-10 10:35:21.332278217 +0000 UTC m=+0.018128996 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:21.715 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 podman[63574]: 2024-05-10 10:35:21.43663669 +0000 UTC m=+0.122487441 container create 3c8d2f32cc7ecdf5be9b6973774c93d062b968164798062426df33b278e923e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-10T10:35:21.715 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 podman[63574]: 2024-05-10 10:35:21.71458911 +0000 UTC m=+0.400439861 container init 3c8d2f32cc7ecdf5be9b6973774c93d062b968164798062426df33b278e923e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-10T10:35:21.894 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:22.044 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 podman[63574]: 2024-05-10 10:35:21.71762413 +0000 UTC m=+0.403474885 container start 3c8d2f32cc7ecdf5be9b6973774c93d062b968164798062426df33b278e923e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:35:22.045 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 ceph-mds[63592]: -- 172.21.15.148:0/1695364137 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a21245a40 con 0x555a21219000 2024-05-10T10:35:22.045 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h[63588]: starting mds.h at 2024-05-10T10:35:22.045 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 bash[63574]: 3c8d2f32cc7ecdf5be9b6973774c93d062b968164798062426df33b278e923e5 2024-05-10T10:35:22.045 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:35:21 smithi148 systemd[1]: Started Ceph mds.h for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:22.146 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:23.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:23.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:23.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: Deploying daemon mds.i on smithi195 2024-05-10T10:35:23.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] up:boot 2024-05-10T10:35:23.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: fsmap 8 up:standby 2024-05-10T10:35:23.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:22 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T10:35:23.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:23.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:23.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:23.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:23.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: Deploying daemon mds.i on smithi195 2024-05-10T10:35:23.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] up:boot 2024-05-10T10:35:23.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: fsmap 8 up:standby 2024-05-10T10:35:23.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:22 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: Deploying daemon mds.i on smithi195 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] up:boot 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: fsmap 8 up:standby 2024-05-10T10:35:23.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:22 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T10:35:23.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 -- 172.21.15.112:0/3652254259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810104100 msgr2=0x7f2810108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:23.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 --2- 172.21.15.112:0/3652254259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810104100 0x7f2810108120 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0099c0 tx=0x7f27fc009cd0 comp rx=0 tx=0).stop 2024-05-10T10:35:23.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 -- 172.21.15.112:0/3652254259 shutdown_connections 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 --2- 172.21.15.112:0/3652254259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2810108850 0x7f28101103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 --2- 172.21.15.112:0/3652254259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2810103750 0x7f2810103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 --2- 172.21.15.112:0/3652254259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810104100 0x7f2810108120 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 -- 172.21.15.112:0/3652254259 >> 172.21.15.112:0/3652254259 conn(0x7f281007a600 msgr2=0x7f281007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 -- 172.21.15.112:0/3652254259 shutdown_connections 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.802+0000 7f2817236700 1 -- 172.21.15.112:0/3652254259 wait complete. 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 Processor -- start 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 -- start start 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810103750 0x7f281019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2810104100 0x7f281019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2810108850 0x7f2810199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2810113540 con 0x7f2810103750 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28101133c0 con 0x7f2810108850 2024-05-10T10:35:23.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2817236700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28101136c0 con 0x7f2810104100 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2814fd2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810103750 0x7f281019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2814fd2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810103750 0x7f281019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34524/0 (socket says 172.21.15.112:34524) 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f2814fd2700 1 -- 172.21.15.112:0/4086125830 learned_addr learned my addr 172.21.15.112:0/4086125830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.803+0000 7f280ffff700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2810104100 0x7f281019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f28157d3700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2810108850 0x7f2810199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2814fd2700 1 -- 172.21.15.112:0/4086125830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2810104100 msgr2=0x7f281019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2814fd2700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2810104100 0x7f281019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2814fd2700 1 -- 172.21.15.112:0/4086125830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2810108850 msgr2=0x7f2810199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2814fd2700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2810108850 0x7f2810199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2814fd2700 1 -- 172.21.15.112:0/4086125830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281019a020 con 0x7f2810103750 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f28157d3700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2810108850 0x7f2810199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:23.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2814fd2700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810103750 0x7f281019f6c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f280400d950 tx=0x7f280400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:23.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f280dffb700 1 -- 172.21.15.112:0/4086125830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804016070 con 0x7f2810103750 2024-05-10T10:35:23.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f280dffb700 1 -- 172.21.15.112:0/4086125830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2804009dd0 con 0x7f2810103750 2024-05-10T10:35:23.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281019a310 con 0x7f2810103750 2024-05-10T10:35:23.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f280dffb700 1 -- 172.21.15.112:0/4086125830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280400f3c0 con 0x7f2810103750 2024-05-10T10:35:23.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.804+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28101a6520 con 0x7f2810103750 2024-05-10T10:35:23.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.805+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2810104be0 con 0x7f2810103750 2024-05-10T10:35:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.806+0000 7f280dffb700 1 -- 172.21.15.112:0/4086125830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f2804019070 con 0x7f2810103750 2024-05-10T10:35:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.807+0000 7f280dffb700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28000b2880 0x7f28000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.807+0000 7f280ffff700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28000b2880 0x7f28000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.807+0000 7f280dffb700 1 -- 172.21.15.112:0/4086125830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f28040e1d80 con 0x7f2810103750 2024-05-10T10:35:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.808+0000 7f280ffff700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28000b2880 0x7f28000b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:23.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:23.811+0000 7f280dffb700 1 -- 172.21.15.112:0/4086125830 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f28040a9f40 con 0x7f2810103750 2024-05-10T10:35:24.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.064+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f281002e510 con 0x7f2810103750 2024-05-10T10:35:24.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.065+0000 7f280dffb700 1 -- 172.21.15.112:0/4086125830 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f28040a9840 con 0x7f2810103750 2024-05-10T10:35:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:24.066 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-10T10:35:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28000b2880 msgr2=0x7f28000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28000b2880 0x7f28000b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc005fb0 comp rx=0 tx=0).stop 2024-05-10T10:35:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810103750 msgr2=0x7f281019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:24.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810103750 0x7f281019f6c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f280400d950 tx=0x7f280400dc60 comp rx=0 tx=0).stop 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 shutdown_connections 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2810108850 0x7f2810199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2810103750 0x7f281019f6c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28000b2880 0x7f28000b4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 --2- 172.21.15.112:0/4086125830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2810104100 0x7f281019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 >> 172.21.15.112:0/4086125830 conn(0x7f281007a600 msgr2=0x7f28101018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 shutdown_connections 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:24.067+0000 7f2817236700 1 -- 172.21.15.112:0/4086125830 wait complete. 2024-05-10T10:35:24.069 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-05-10T10:35:24.290 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 systemd[1]: Starting Ceph mds.i for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:24.560 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 podman[64104]: 2024-05-10 10:35:24.289179109 +0000 UTC m=+0.113807738 container create 8ede9de1eb86562c52fa3971e52e3ef5dde7024cfa7a23689b2c845277845cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-10T10:35:24.560 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 podman[64104]: 2024-05-10 10:35:24.193672348 +0000 UTC m=+0.018300985 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:24.561 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 podman[64104]: 2024-05-10 10:35:24.533684949 +0000 UTC m=+0.358313576 container init 8ede9de1eb86562c52fa3971e52e3ef5dde7024cfa7a23689b2c845277845cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-10T10:35:24.561 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 podman[64104]: 2024-05-10 10:35:24.536462284 +0000 UTC m=+0.361090913 container start 8ede9de1eb86562c52fa3971e52e3ef5dde7024cfa7a23689b2c845277845cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True) 2024-05-10T10:35:24.816 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mds[64123]: -- 172.21.15.195:0/1035782623 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6beba1a40 con 0x55b6beb74c00 2024-05-10T10:35:24.816 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: starting mds.i at 2024-05-10T10:35:24.816 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 bash[64104]: 8ede9de1eb86562c52fa3971e52e3ef5dde7024cfa7a23689b2c845277845cbd 2024-05-10T10:35:24.816 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:24 smithi195 systemd[1]: Started Ceph mds.i for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4086125830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:25.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:25.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:24 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:25.154 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:25.154 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4086125830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:25.154 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.155 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.155 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.155 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:25.155 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:25.155 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:24 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4086125830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:25.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:24 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:26.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:25 smithi148 ceph-mon[33568]: Deploying daemon mds.g on smithi112 2024-05-10T10:35:26.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:25 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] up:boot 2024-05-10T10:35:26.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:25 smithi148 ceph-mon[33568]: fsmap 9 up:standby 2024-05-10T10:35:26.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:25 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T10:35:26.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:25 smithi112 ceph-mon[30100]: Deploying daemon mds.g on smithi112 2024-05-10T10:35:26.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:25 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] up:boot 2024-05-10T10:35:26.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:25 smithi112 ceph-mon[30100]: fsmap 9 up:standby 2024-05-10T10:35:26.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:25 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T10:35:26.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:25 smithi195 ceph-mon[33773]: Deploying daemon mds.g on smithi112 2024-05-10T10:35:26.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:25 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] up:boot 2024-05-10T10:35:26.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:25 smithi195 ceph-mon[33773]: fsmap 9 up:standby 2024-05-10T10:35:26.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:25 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T10:35:26.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:26.985 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:26 smithi195 ceph-mon[33773]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:27.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:26 smithi112 ceph-mon[30100]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:27.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:26 smithi148 ceph-mon[33568]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:27.717 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:35:27 smithi112 systemd[1]: Started Ceph mds.g for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:28.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: Deploying daemon mds.a on smithi112 2024-05-10T10:35:28.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.112:6838/2952588053,v1:172.21.15.112:6839/2952588053] up:boot 2024-05-10T10:35:28.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: fsmap 10 up:standby 2024-05-10T10:35:28.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:28 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T10:35:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: Deploying daemon mds.a on smithi112 2024-05-10T10:35:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.112:6838/2952588053,v1:172.21.15.112:6839/2952588053] up:boot 2024-05-10T10:35:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: fsmap 10 up:standby 2024-05-10T10:35:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:28 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T10:35:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:28.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:28.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: Deploying daemon mds.a on smithi112 2024-05-10T10:35:28.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.112:6838/2952588053,v1:172.21.15.112:6839/2952588053] up:boot 2024-05-10T10:35:28.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: fsmap 10 up:standby 2024-05-10T10:35:28.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:28 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T10:35:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 -- 172.21.15.112:0/2195640613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c105ee0 msgr2=0x7fed6c106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 --2- 172.21.15.112:0/2195640613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c105ee0 0x7fed6c106370 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b0a0 tx=0x7fed6400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 -- 172.21.15.112:0/2195640613 shutdown_connections 2024-05-10T10:35:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 --2- 172.21.15.112:0/2195640613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c105ee0 0x7fed6c106370 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 --2- 172.21.15.112:0/2195640613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed6c1083d0 0x7fed6c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 --2- 172.21.15.112:0/2195640613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed6c06b6f0 0x7fed6c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 -- 172.21.15.112:0/2195640613 >> 172.21.15.112:0/2195640613 conn(0x7fed6c079350 msgr2=0x7fed6c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 -- 172.21.15.112:0/2195640613 shutdown_connections 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.283+0000 7fed72df9700 1 -- 172.21.15.112:0/2195640613 wait complete. 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 Processor -- start 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 -- start start 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c06b6f0 0x7fed6c1a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed6c105ee0 0x7fed6c1a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed6c1083d0 0x7fed6c1a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c119b40 con 0x7fed6c06b6f0 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1199c0 con 0x7fed6c1083d0 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed72df9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c119cc0 con 0x7fed6c105ee0 2024-05-10T10:35:29.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed70b95700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c06b6f0 0x7fed6c1a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed70b95700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c06b6f0 0x7fed6c1a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42178/0 (socket says 172.21.15.112:42178) 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed70b95700 1 -- 172.21.15.112:0/3170298816 learned_addr learned my addr 172.21.15.112:0/3170298816 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.284+0000 7fed6bfff700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed6c105ee0 0x7fed6c1a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed70b95700 1 -- 172.21.15.112:0/3170298816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed6c105ee0 msgr2=0x7fed6c1a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed70b95700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed6c105ee0 0x7fed6c1a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed70b95700 1 -- 172.21.15.112:0/3170298816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed6c1083d0 msgr2=0x7fed6c1a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed70b95700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed6c1083d0 0x7fed6c1a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed70b95700 1 -- 172.21.15.112:0/3170298816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c1a5020 con 0x7fed6c06b6f0 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed6bfff700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed6c105ee0 0x7fed6c1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed70b95700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c06b6f0 0x7fed6c1a0010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c9b0 tx=0x7fed6000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed69ffb700 1 -- 172.21.15.112:0/3170298816 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60007b00 con 0x7fed6c06b6f0 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6c1a5310 con 0x7fed6c06b6f0 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed69ffb700 1 -- 172.21.15.112:0/3170298816 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed60007ca0 con 0x7fed6c06b6f0 2024-05-10T10:35:29.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed69ffb700 1 -- 172.21.15.112:0/3170298816 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60018660 con 0x7fed6c06b6f0 2024-05-10T10:35:29.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.285+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6c1acb90 con 0x7fed6c06b6f0 2024-05-10T10:35:29.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.286+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6c06a190 con 0x7fed6c06b6f0 2024-05-10T10:35:29.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.287+0000 7fed69ffb700 1 -- 172.21.15.112:0/3170298816 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fed6000f450 con 0x7fed6c06b6f0 2024-05-10T10:35:29.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.288+0000 7fed69ffb700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fed5c0b2880 0x7fed5c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:29.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.289+0000 7fed6bfff700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fed5c0b2880 0x7fed5c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:29.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.289+0000 7fed69ffb700 1 -- 172.21.15.112:0/3170298816 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fed600e21a0 con 0x7fed6c06b6f0 2024-05-10T10:35:29.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.289+0000 7fed6bfff700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fed5c0b2880 0x7fed5c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fed6c06b4e0 tx=0x7fed58008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:29.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.292+0000 7fed69ffb700 1 -- 172.21.15.112:0/3170298816 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fed60014030 con 0x7fed6c06b6f0 2024-05-10T10:35:29.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.552+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed6c1a1370 con 0x7fed6c06b6f0 2024-05-10T10:35:29.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.553+0000 7fed69ffb700 1 -- 172.21.15.112:0/3170298816 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7fed600aa350 con 0x7fed6c06b6f0 2024-05-10T10:35:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:29.555 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T10:35:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fed5c0b2880 msgr2=0x7fed5c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fed5c0b2880 0x7fed5c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fed6c06b4e0 tx=0x7fed58008040 comp rx=0 tx=0).stop 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c06b6f0 msgr2=0x7fed6c1a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c06b6f0 0x7fed6c1a0010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c9b0 tx=0x7fed6000cd70 comp rx=0 tx=0).stop 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 shutdown_connections 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed6c1083d0 0x7fed6c1a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed6c06b6f0 0x7fed6c1a0010 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fed5c0b2880 0x7fed5c0b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 --2- 172.21.15.112:0/3170298816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed6c105ee0 0x7fed6c1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 >> 172.21.15.112:0/3170298816 conn(0x7fed6c079350 msgr2=0x7fed6c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 shutdown_connections 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:29.555+0000 7fed72df9700 1 -- 172.21.15.112:0/3170298816 wait complete. 2024-05-10T10:35:29.557 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-05-10T10:35:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:30 smithi148 ceph-mon[33568]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:30 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3170298816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:30.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:30 smithi195 ceph-mon[33773]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:30.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:30 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3170298816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:30.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:30 smithi112 ceph-mon[30100]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:30.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:30 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3170298816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:31.830 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:31 smithi112 systemd[1]: Starting Ceph mds.a for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:32.209 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:31 smithi112 podman[69606]: 2024-05-10 10:35:31.842742029 +0000 UTC m=+0.018274855 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:32.209 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:31 smithi112 podman[69606]: 2024-05-10 10:35:31.964980553 +0000 UTC m=+0.140513371 container create a70829d91c71fe1b1873b616d5e4fb2872f87f1b08a36c988a2fb2afe1309305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0) 2024-05-10T10:35:32.209 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:32 smithi112 podman[69606]: 2024-05-10 10:35:32.209183066 +0000 UTC m=+0.384715885 container init a70829d91c71fe1b1873b616d5e4fb2872f87f1b08a36c988a2fb2afe1309305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-10T10:35:32.514 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:32 smithi112 podman[69606]: 2024-05-10 10:35:32.212097861 +0000 UTC m=+0.387630674 container start a70829d91c71fe1b1873b616d5e4fb2872f87f1b08a36c988a2fb2afe1309305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-10T10:35:32.514 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mds[69625]: -- 172.21.15.112:0/3280328719 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b612caba40 con 0x55b612c7f000 2024-05-10T10:35:32.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a[69621]: starting mds.a at 2024-05-10T10:35:32.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:32 smithi112 bash[69606]: a70829d91c71fe1b1873b616d5e4fb2872f87f1b08a36c988a2fb2afe1309305 2024-05-10T10:35:32.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:35:32 smithi112 systemd[1]: Started Ceph mds.a for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:32.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:32.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mon[30100]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:32.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:32.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:32.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:32 smithi148 ceph-mon[33568]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:32.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:32 smithi195 ceph-mon[33773]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T10:35:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T10:35:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:33.009 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:33.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:33 smithi148 ceph-mon[33568]: Deploying daemon mds.b on smithi148 2024-05-10T10:35:33.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:33 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.112:6840/1213307484,v1:172.21.15.112:6841/1213307484] up:boot 2024-05-10T10:35:33.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:33 smithi148 ceph-mon[33568]: fsmap 11 up:standby 2024-05-10T10:35:33.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:33 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T10:35:33.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:33 smithi195 ceph-mon[33773]: Deploying daemon mds.b on smithi148 2024-05-10T10:35:33.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:33 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.112:6840/1213307484,v1:172.21.15.112:6841/1213307484] up:boot 2024-05-10T10:35:33.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:33 smithi195 ceph-mon[33773]: fsmap 11 up:standby 2024-05-10T10:35:33.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:33 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T10:35:33.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:33 smithi112 ceph-mon[30100]: Deploying daemon mds.b on smithi148 2024-05-10T10:35:33.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:33 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.112:6840/1213307484,v1:172.21.15.112:6841/1213307484] up:boot 2024-05-10T10:35:33.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:33 smithi112 ceph-mon[30100]: fsmap 11 up:standby 2024-05-10T10:35:33.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:33 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T10:35:34.755 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:34 smithi112 ceph-mon[30100]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:34.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 -- 172.21.15.112:0/3266176791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 msgr2=0x7f326c109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:34.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 --2- 172.21.15.112:0/3266176791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c109900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3264009990 tx=0x7f3264009ca0 comp rx=0 tx=0).stop 2024-05-10T10:35:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 -- 172.21.15.112:0/3266176791 shutdown_connections 2024-05-10T10:35:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 --2- 172.21.15.112:0/3266176791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c102450 0x7f326c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 --2- 172.21.15.112:0/3266176791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c109900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 --2- 172.21.15.112:0/3266176791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f326c101a90 0x7f326c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 -- 172.21.15.112:0/3266176791 >> 172.21.15.112:0/3266176791 conn(0x7f326c078a10 msgr2=0x7f326c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 -- 172.21.15.112:0/3266176791 shutdown_connections 2024-05-10T10:35:34.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.773+0000 7f32740da700 1 -- 172.21.15.112:0/3266176791 wait complete. 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 Processor -- start 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 -- start start 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c101a90 0x7f326c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f326c102450 0x7f326c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f326c113560 con 0x7f326c109520 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1133e0 con 0x7f326c101a90 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f32740da700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1136e0 con 0x7f326c102450 2024-05-10T10:35:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42202/0 (socket says 172.21.15.112:42202) 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 -- 172.21.15.112:0/1383431706 learned_addr learned my addr 172.21.15.112:0/1383431706 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3271e76700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c101a90 0x7f326c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3271675700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f326c102450 0x7f326c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 -- 172.21.15.112:0/1383431706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f326c102450 msgr2=0x7f326c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f326c102450 0x7f326c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 -- 172.21.15.112:0/1383431706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c101a90 msgr2=0x7f326c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c101a90 0x7f326c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.774+0000 7f3272677700 1 -- 172.21.15.112:0/1383431706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c19a040 con 0x7f326c109520 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.775+0000 7f3272677700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c1998e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f325c007ee0 tx=0x7f325c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.775+0000 7f326affd700 1 -- 172.21.15.112:0/1383431706 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c00dbd0 con 0x7f326c109520 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.775+0000 7f326affd700 1 -- 172.21.15.112:0/1383431706 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f325c00dd70 con 0x7f326c109520 2024-05-10T10:35:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.775+0000 7f326affd700 1 -- 172.21.15.112:0/1383431706 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c0058d0 con 0x7f326c109520 2024-05-10T10:35:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.775+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c19a330 con 0x7f326c109520 2024-05-10T10:35:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.775+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c1a65f0 con 0x7f326c109520 2024-05-10T10:35:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.776+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326c10ba00 con 0x7f326c109520 2024-05-10T10:35:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.777+0000 7f326affd700 1 -- 172.21.15.112:0/1383431706 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f325c01a020 con 0x7f326c109520 2024-05-10T10:35:34.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.778+0000 7f326affd700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32540b2900 0x7f32540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:34.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.778+0000 7f3271e76700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32540b2900 0x7f32540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:34.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.778+0000 7f326affd700 1 -- 172.21.15.112:0/1383431706 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f325c0e1b60 con 0x7f326c109520 2024-05-10T10:35:34.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.778+0000 7f3271e76700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32540b2900 0x7f32540b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f326400b5c0 tx=0x7f326401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:34.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:34.781+0000 7f326affd700 1 -- 172.21.15.112:0/1383431706 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f325c010070 con 0x7f326c109520 2024-05-10T10:35:34.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:34 smithi148 ceph-mon[33568]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:34.794 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:34 smithi148 systemd[1]: Starting Ceph mds.b for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:35:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:34 smithi195 ceph-mon[33773]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:35.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.028+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f326c06a190 con 0x7f326c109520 2024-05-10T10:35:35.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.029+0000 7f326affd700 1 -- 172.21.15.112:0/1383431706 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9671 (secure 0 0 0) 0x7f325c0a9d20 con 0x7f326c109520 2024-05-10T10:35:35.030 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:35.030 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T10:35:35.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32540b2900 msgr2=0x7f32540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32540b2900 0x7f32540b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f326400b5c0 tx=0x7f326401a040 comp rx=0 tx=0).stop 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 msgr2=0x7f326c1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c1998e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f325c007ee0 tx=0x7f325c00d3b0 comp rx=0 tx=0).stop 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 shutdown_connections 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109520 0x7f326c1998e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c101a90 0x7f326c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32540b2900 0x7f32540b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 --2- 172.21.15.112:0/1383431706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f326c102450 0x7f326c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 >> 172.21.15.112:0/1383431706 conn(0x7f326c078a10 msgr2=0x7f326c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.031+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 shutdown_connections 2024-05-10T10:35:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:35.032+0000 7f32740da700 1 -- 172.21.15.112:0/1383431706 wait complete. 2024-05-10T10:35:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-05-10T10:35:35.200 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:34 smithi148 podman[63886]: 2024-05-10 10:35:34.810633159 +0000 UTC m=+0.018303069 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:35:35.200 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:34 smithi148 podman[63886]: 2024-05-10 10:35:34.935697501 +0000 UTC m=+0.143367398 container create 564e4250fa82a6058aa42dae2dbb1e4c3ec5deda0db7e93781338b191d10a7f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-05-10T10:35:35.200 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:35 smithi148 podman[63886]: 2024-05-10 10:35:35.199991277 +0000 UTC m=+0.407661178 container init 564e4250fa82a6058aa42dae2dbb1e4c3ec5deda0db7e93781338b191d10a7f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:35:35.526 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:35 smithi148 podman[63886]: 2024-05-10 10:35:35.203153474 +0000 UTC m=+0.410823371 container start 564e4250fa82a6058aa42dae2dbb1e4c3ec5deda0db7e93781338b191d10a7f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-10T10:35:35.526 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-mds[63905]: -- 172.21.15.148:0/1264438194 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604e59cda40 con 0x5604e59a1000 2024-05-10T10:35:35.526 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b[63901]: starting mds.b at 2024-05-10T10:35:35.527 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:35 smithi148 bash[63886]: 564e4250fa82a6058aa42dae2dbb1e4c3ec5deda0db7e93781338b191d10a7f2 2024-05-10T10:35:35.527 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:35:35 smithi148 systemd[1]: Started Ceph mds.b for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:35:35.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:35 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1383431706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:35.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.763 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.763 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.763 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.763 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1383431706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:35.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:35 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1383431706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:36.648 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:36 smithi148 ceph-mon[33568]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:36.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:36 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] up:boot 2024-05-10T10:35:36.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:36 smithi148 ceph-mon[33568]: fsmap 12 up:standby 2024-05-10T10:35:36.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T10:35:36.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:35:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:36 smithi195 ceph-mon[33773]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:36 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] up:boot 2024-05-10T10:35:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:36 smithi195 ceph-mon[33773]: fsmap 12 up:standby 2024-05-10T10:35:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T10:35:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:35:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:36 smithi112 ceph-mon[30100]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:36 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] up:boot 2024-05-10T10:35:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:36 smithi112 ceph-mon[30100]: fsmap 12 up:standby 2024-05-10T10:35:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T10:35:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:35:37.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:35:38.037 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:38.413 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:35:38.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:38.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:35:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:35:39.487 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:39.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:39.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 -- 172.21.15.112:0/1865779441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c102d60 msgr2=0x7f339c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/1865779441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c102d60 0x7f339c10f250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f339400b0a0 tx=0x7f339400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 -- 172.21.15.112:0/1865779441 shutdown_connections 2024-05-10T10:35:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/1865779441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c102d60 0x7f339c10f250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/1865779441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339c108410 0x7f339c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/1865779441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f339c1023a0 0x7f339c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 -- 172.21.15.112:0/1865779441 >> 172.21.15.112:0/1865779441 conn(0x7f339c079350 msgr2=0x7f339c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 -- 172.21.15.112:0/1865779441 shutdown_connections 2024-05-10T10:35:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 -- 172.21.15.112:0/1865779441 wait complete. 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.728+0000 7f33a0bb1700 1 Processor -- start 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f33a0bb1700 1 -- start start 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f33a0bb1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f339c1023a0 0x7f339c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f33a0bb1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339c102d60 0x7f339c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f33a0bb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c108410 0x7f339c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f33a0bb1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f339c112410 con 0x7f339c108410 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f33a0bb1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f339c112290 con 0x7f339c102d60 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f33a0bb1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f339c112590 con 0x7f339c1023a0 2024-05-10T10:35:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c108410 0x7f339c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f339c1023a0 0x7f339c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c108410 0x7f339c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50348/0 (socket says 172.21.15.112:50348) 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 -- 172.21.15.112:0/2326153165 learned_addr learned my addr 172.21.15.112:0/2326153165 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339a59c700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f339c1023a0 0x7f339c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:37528/0 (socket says 172.21.15.112:37528) 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f3399d9b700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339c102d60 0x7f339c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 -- 172.21.15.112:0/2326153165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f339c1023a0 msgr2=0x7f339c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f339c1023a0 0x7f339c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 -- 172.21.15.112:0/2326153165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339c102d60 msgr2=0x7f339c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339c102d60 0x7f339c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 -- 172.21.15.112:0/2326153165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c199fe0 con 0x7f339c108410 2024-05-10T10:35:39.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.729+0000 7f339ad9d700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c108410 0x7f339c199800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f339400b780 tx=0x7f3394004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.730+0000 7f338b7fe700 1 -- 172.21.15.112:0/2326153165 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394021070 con 0x7f339c108410 2024-05-10T10:35:39.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.730+0000 7f338b7fe700 1 -- 172.21.15.112:0/2326153165 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3394007b80 con 0x7f339c108410 2024-05-10T10:35:39.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.730+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c19a270 con 0x7f339c108410 2024-05-10T10:35:39.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.730+0000 7f338b7fe700 1 -- 172.21.15.112:0/2326153165 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33940052a0 con 0x7f339c108410 2024-05-10T10:35:39.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.730+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c1a64e0 con 0x7f339c108410 2024-05-10T10:35:39.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.731+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c103ac0 con 0x7f339c108410 2024-05-10T10:35:39.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.732+0000 7f338b7fe700 1 -- 172.21.15.112:0/2326153165 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3394024070 con 0x7f339c108410 2024-05-10T10:35:39.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.733+0000 7f338b7fe700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33840b2900 0x7f33840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.733+0000 7f339a59c700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33840b2900 0x7f33840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.733+0000 7f338b7fe700 1 -- 172.21.15.112:0/2326153165 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f339400f020 con 0x7f339c108410 2024-05-10T10:35:39.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.733+0000 7f339a59c700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33840b2900 0x7f33840b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f338c009be0 tx=0x7f338c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:39.738+0000 7f338b7fe700 1 -- 172.21.15.112:0/2326153165 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f339402b020 con 0x7f339c108410 2024-05-10T10:35:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.007+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f339c1a66b0 con 0x7f339c108410 2024-05-10T10:35:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.008+0000 7f338b7fe700 1 -- 172.21.15.112:0/2326153165 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7f33940ab3e0 con 0x7f339c108410 2024-05-10T10:35:40.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:40.009 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T10:35:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33840b2900 msgr2=0x7f33840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33840b2900 0x7f33840b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f338c009be0 tx=0x7f338c009260 comp rx=0 tx=0).stop 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c108410 msgr2=0x7f339c199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c108410 0x7f339c199800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f339400b780 tx=0x7f3394004d00 comp rx=0 tx=0).stop 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 shutdown_connections 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f339c108410 0x7f339c199800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33840b2900 0x7f33840b4dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f339c1023a0 0x7f339c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 --2- 172.21.15.112:0/2326153165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339c102d60 0x7f339c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 >> 172.21.15.112:0/2326153165 conn(0x7f339c079350 msgr2=0x7f339c101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 shutdown_connections 2024-05-10T10:35:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:40.010+0000 7f33a0bb1700 1 -- 172.21.15.112:0/2326153165 wait complete. 2024-05-10T10:35:40.013 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-05-10T10:35:41.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:40 smithi112 ceph-mon[30100]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:41.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.053 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.053 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.053 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.053 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:40 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2326153165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:41.080 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-10T10:35:41.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:40 smithi148 ceph-mon[33568]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:40 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2326153165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:40 smithi195 ceph-mon[33773]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:40 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2326153165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:35:41.354 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 -- 172.21.15.112:0/625215680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674108850 msgr2=0x7f76741103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 --2- 172.21.15.112:0/625215680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674108850 0x7f76741103d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f766c00b0d0 tx=0x7f766c00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 -- 172.21.15.112:0/625215680 shutdown_connections 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 --2- 172.21.15.112:0/625215680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674108850 0x7f76741103d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 --2- 172.21.15.112:0/625215680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7674103750 0x7f7674103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 --2- 172.21.15.112:0/625215680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7674104100 0x7f7674108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 -- 172.21.15.112:0/625215680 >> 172.21.15.112:0/625215680 conn(0x7f767407a600 msgr2=0x7f767407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 -- 172.21.15.112:0/625215680 shutdown_connections 2024-05-10T10:35:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.067+0000 7f76794f1700 1 -- 172.21.15.112:0/625215680 wait complete. 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.068+0000 7f76794f1700 1 Processor -- start 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.068+0000 7f76794f1700 1 -- start start 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76794f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674103750 0x7f767419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76794f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7674104100 0x7f767419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76794f1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7674108850 0x7f7674199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76794f1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76741135c0 con 0x7f7674103750 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76794f1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7674113440 con 0x7f7674104100 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76794f1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7674113740 con 0x7f7674108850 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674103750 0x7f767419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674103750 0x7f767419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50372/0 (socket says 172.21.15.112:50372) 2024-05-10T10:35:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 -- 172.21.15.112:0/3328084657 learned_addr learned my addr 172.21.15.112:0/3328084657 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76737fe700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7674108850 0x7f7674199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76727fc700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7674104100 0x7f767419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 -- 172.21.15.112:0/3328084657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7674108850 msgr2=0x7f7674199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7674108850 0x7f7674199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 -- 172.21.15.112:0/3328084657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7674104100 msgr2=0x7f767419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7674104100 0x7f767419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 -- 172.21.15.112:0/3328084657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767419a080 con 0x7f7674103750 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f76737fe700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7674108850 0x7f7674199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f7672ffd700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674103750 0x7f767419f760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f766400b530 tx=0x7f766400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.069+0000 7f765bfff700 1 -- 172.21.15.112:0/3328084657 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664015020 con 0x7f7674103750 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.070+0000 7f765bfff700 1 -- 172.21.15.112:0/3328084657 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7664011cc0 con 0x7f7674103750 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.070+0000 7f765bfff700 1 -- 172.21.15.112:0/3328084657 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664010d30 con 0x7f7674103750 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.070+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767419a370 con 0x7f7674103750 2024-05-10T10:35:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.070+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76741a66b0 con 0x7f7674103750 2024-05-10T10:35:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.071+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767410ba90 con 0x7f7674103750 2024-05-10T10:35:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.072+0000 7f765bfff700 1 -- 172.21.15.112:0/3328084657 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7664018020 con 0x7f7674103750 2024-05-10T10:35:43.074 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:42 smithi112 ceph-mon[30100]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:43.074 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:43.074 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.073+0000 7f765bfff700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f765c0b2900 0x7f765c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.073+0000 7f765bfff700 1 -- 172.21.15.112:0/3328084657 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f76640e1b00 con 0x7f7674103750 2024-05-10T10:35:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.073+0000 7f76727fc700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f765c0b2900 0x7f765c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.073+0000 7f76727fc700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f765c0b2900 0x7f765c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7668006fd0 tx=0x7f7668008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.078+0000 7f765bfff700 1 -- 172.21.15.112:0/3328084657 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f766401f020 con 0x7f7674103750 2024-05-10T10:35:43.140 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:42 smithi195 ceph-mon[33773]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:43.140 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:43.140 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:42 smithi148 ceph-mon[33568]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.294+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f767419ae50 con 0x7f7674103750 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.295+0000 7f765bfff700 1 -- 172.21.15.112:0/3328084657 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f76640a9cc0 con 0x7f7674103750 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:35:43.296 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-05-10T10:35:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f765c0b2900 msgr2=0x7f765c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f765c0b2900 0x7f765c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7668006fd0 tx=0x7f7668008040 comp rx=0 tx=0).stop 2024-05-10T10:35:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674103750 msgr2=0x7f767419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674103750 0x7f767419f760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f766400b530 tx=0x7f766400b8f0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 shutdown_connections 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f765c0b2900 0x7f765c0b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7674108850 0x7f7674199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674103750 0x7f767419f760 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 --2- 172.21.15.112:0/3328084657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7674104100 0x7f767419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 >> 172.21.15.112:0/3328084657 conn(0x7f767407a600 msgr2=0x7f7674101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 shutdown_connections 2024-05-10T10:35:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:43.297+0000 7f76794f1700 1 -- 172.21.15.112:0/3328084657 wait complete. 2024-05-10T10:35:43.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:43.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:43 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3328084657' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T10:35:43.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:35:43.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:43.972 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:35:43.972 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:35:44.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:44.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:43 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3328084657' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T10:35:44.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:35:44.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:44.215 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:44.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:35:44.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:43 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3328084657' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T10:35:44.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:35:44.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:35:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:44 smithi148 ceph-mon[33568]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:44 smithi195 ceph-mon[33773]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:45.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:44 smithi112 ceph-mon[30100]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:45.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 -- 172.21.15.112:0/536556401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c102d60 msgr2=0x7fc33c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:45.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 --2- 172.21.15.112:0/536556401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c102d60 0x7fc33c10f250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc33400b0a0 tx=0x7fc33400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 -- 172.21.15.112:0/536556401 shutdown_connections 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 --2- 172.21.15.112:0/536556401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c102d60 0x7fc33c10f250 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 --2- 172.21.15.112:0/536556401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33c108410 0x7fc33c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 --2- 172.21.15.112:0/536556401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c1023a0 0x7fc33c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 -- 172.21.15.112:0/536556401 >> 172.21.15.112:0/536556401 conn(0x7fc33c079350 msgr2=0x7fc33c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 -- 172.21.15.112:0/536556401 shutdown_connections 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.953+0000 7fc340c1d700 1 -- 172.21.15.112:0/536556401 wait complete. 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 Processor -- start 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 -- start start 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c1023a0 0x7fc33c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33c102d60 0x7fc33c1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c108410 0x7fc33c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c112460 con 0x7fc33c108410 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c1122e0 con 0x7fc33c1023a0 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc340c1d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c1125e0 con 0x7fc33c102d60 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc33ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c108410 0x7fc33c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc33ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c108410 0x7fc33c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33288/0 (socket says 172.21.15.112:33288) 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.954+0000 7fc33ad9d700 1 -- 172.21.15.112:0/972733975 learned_addr learned my addr 172.21.15.112:0/972733975 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc331d9b700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33c102d60 0x7fc33c1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc33a59c700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c1023a0 0x7fc33c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc33ad9d700 1 -- 172.21.15.112:0/972733975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33c102d60 msgr2=0x7fc33c1a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc33ad9d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33c102d60 0x7fc33c1a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc33ad9d700 1 -- 172.21.15.112:0/972733975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c1023a0 msgr2=0x7fc33c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc33ad9d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c1023a0 0x7fc33c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc33ad9d700 1 -- 172.21.15.112:0/972733975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c19c410 con 0x7fc33c108410 2024-05-10T10:35:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc33ad9d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c108410 0x7fc33c19bb30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3340062a0 tx=0x7fc334004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc333fff700 1 -- 172.21.15.112:0/972733975 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334021070 con 0x7fc33c108410 2024-05-10T10:35:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc333fff700 1 -- 172.21.15.112:0/972733975 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc334007c00 con 0x7fc33c108410 2024-05-10T10:35:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc333fff700 1 -- 172.21.15.112:0/972733975 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3340052a0 con 0x7fc33c108410 2024-05-10T10:35:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33c19c6a0 con 0x7fc33c108410 2024-05-10T10:35:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.955+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33c1a89e0 con 0x7fc33c108410 2024-05-10T10:35:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.956+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33c19ca00 con 0x7fc33c108410 2024-05-10T10:35:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.958+0000 7fc333fff700 1 -- 172.21.15.112:0/972733975 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc334024070 con 0x7fc33c108410 2024-05-10T10:35:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.958+0000 7fc333fff700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc3280b2900 0x7fc3280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.958+0000 7fc33a59c700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc3280b2900 0x7fc3280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.959+0000 7fc333fff700 1 -- 172.21.15.112:0/972733975 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc33400f020 con 0x7fc33c108410 2024-05-10T10:35:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.959+0000 7fc33a59c700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc3280b2900 0x7fc3280b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005950 tx=0x7fc32c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:45.962+0000 7fc333fff700 1 -- 172.21.15.112:0/972733975 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc33402b020 con 0x7fc33c108410 2024-05-10T10:35:46.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.165+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc33c103ac0 con 0x7fc33c108410 2024-05-10T10:35:46.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.889+0000 7fc333fff700 1 -- 172.21.15.112:0/972733975 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fc3340aba70 con 0x7fc33c108410 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc3280b2900 msgr2=0x7fc3280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc3280b2900 0x7fc3280b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005950 tx=0x7fc32c00a300 comp rx=0 tx=0).stop 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c108410 msgr2=0x7fc33c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c108410 0x7fc33c19bb30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3340062a0 tx=0x7fc334004e20 comp rx=0 tx=0).stop 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 shutdown_connections 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33c108410 0x7fc33c19bb30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c1023a0 0x7fc33c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc3280b2900 0x7fc3280b4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 --2- 172.21.15.112:0/972733975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc33c102d60 0x7fc33c1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 >> 172.21.15.112:0/972733975 conn(0x7fc33c079350 msgr2=0x7fc33c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 shutdown_connections 2024-05-10T10:35:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:46.893+0000 7fc340c1d700 1 -- 172.21.15.112:0/972733975 wait complete. 2024-05-10T10:35:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-05-10T10:35:47.193 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:46 smithi112 ceph-mon[30100]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:47.193 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:46 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/972733975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T10:35:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:46 smithi148 ceph-mon[33568]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:46 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/972733975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T10:35:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:46 smithi195 ceph-mon[33773]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:46 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/972733975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T10:35:47.547 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-10T10:35:47.794 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:48.172 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:47 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/972733975' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T10:35:48.172 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:47 smithi112 ceph-mon[30100]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T10:35:48.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:47 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/972733975' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T10:35:48.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:47 smithi148 ceph-mon[33568]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T10:35:48.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:47 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/972733975' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T10:35:48.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:47 smithi195 ceph-mon[33773]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T10:35:49.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:48 smithi112 ceph-mon[30100]: pgmap v280: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:49.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:48 smithi112 ceph-mon[30100]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T10:35:49.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:48 smithi112 ceph-mon[30100]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T10:35:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:48 smithi148 ceph-mon[33568]: pgmap v280: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:48 smithi148 ceph-mon[33568]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T10:35:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:48 smithi148 ceph-mon[33568]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T10:35:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:48 smithi195 ceph-mon[33773]: pgmap v280: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:48 smithi195 ceph-mon[33773]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T10:35:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:48 smithi195 ceph-mon[33773]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 -- 172.21.15.112:0/2322149789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109660 msgr2=0x7f2df0109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 --2- 172.21.15.112:0/2322149789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109660 0x7f2df0109a40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009990 tx=0x7f2de0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 -- 172.21.15.112:0/2322149789 shutdown_connections 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 --2- 172.21.15.112:0/2322149789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2df0104180 0x7f2df01103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 --2- 172.21.15.112:0/2322149789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0109660 0x7f2df0109a40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 --2- 172.21.15.112:0/2322149789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df01037c0 0x7f2df0103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 -- 172.21.15.112:0/2322149789 >> 172.21.15.112:0/2322149789 conn(0x7f2df007b430 msgr2=0x7f2df007b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 -- 172.21.15.112:0/2322149789 shutdown_connections 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.614+0000 7f2df59ec700 1 -- 172.21.15.112:0/2322149789 wait complete. 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 Processor -- start 2024-05-10T10:35:49.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 -- start start 2024-05-10T10:35:49.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df01037c0 0x7f2df019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0104180 0x7f2df019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2df0109660 0x7f2df01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0113690 con 0x7f2df0104180 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0113510 con 0x7f2df01037c0 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2df59ec700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0113810 con 0x7f2df0109660 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0104180 0x7f2df019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0104180 0x7f2df019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33298/0 (socket says 172.21.15.112:33298) 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 -- 172.21.15.112:0/506070544 learned_addr learned my addr 172.21.15.112:0/506070544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:49.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2deeffd700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df01037c0 0x7f2df019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2def7fe700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2df0109660 0x7f2df01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 -- 172.21.15.112:0/506070544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2df0109660 msgr2=0x7f2df01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2df0109660 0x7f2df01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 -- 172.21.15.112:0/506070544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df01037c0 msgr2=0x7f2df019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df01037c0 0x7f2df019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.615+0000 7f2de67fc700 1 -- 172.21.15.112:0/506070544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df019a130 con 0x7f2df0104180 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.616+0000 7f2def7fe700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2df0109660 0x7f2df01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.616+0000 7f2de67fc700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0104180 0x7f2df019fd00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800b530 tx=0x7f2dd800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.616+0000 7f2df49ea700 1 -- 172.21.15.112:0/506070544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8015020 con 0x7f2df0104180 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.616+0000 7f2df49ea700 1 -- 172.21.15.112:0/506070544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2dd8011cc0 con 0x7f2df0104180 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.616+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df019a420 con 0x7f2df0104180 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.616+0000 7f2df49ea700 1 -- 172.21.15.112:0/506070544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8010d30 con 0x7f2df0104180 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.616+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a6710 con 0x7f2df0104180 2024-05-10T10:35:49.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.617+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df006a190 con 0x7f2df0104180 2024-05-10T10:35:49.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.618+0000 7f2df49ea700 1 -- 172.21.15.112:0/506070544 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f2dd8018020 con 0x7f2df0104180 2024-05-10T10:35:49.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.619+0000 7f2df49ea700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ddc0b2900 0x7f2ddc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.619+0000 7f2deeffd700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ddc0b2900 0x7f2ddc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.619+0000 7f2df49ea700 1 -- 172.21.15.112:0/506070544 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f2dd80e1d10 con 0x7f2df0104180 2024-05-10T10:35:49.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.620+0000 7f2deeffd700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ddc0b2900 0x7f2ddc0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:49.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.623+0000 7f2df49ea700 1 -- 172.21.15.112:0/506070544 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2dd801f020 con 0x7f2df0104180 2024-05-10T10:35:49.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.827+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2df002e510 con 0x7f2df0104180 2024-05-10T10:35:49.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.906+0000 7f2df49ea700 1 -- 172.21.15.112:0/506070544 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f2dd80a9cc0 con 0x7f2df0104180 2024-05-10T10:35:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ddc0b2900 msgr2=0x7f2ddc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ddc0b2900 0x7f2ddc0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de001a040 comp rx=0 tx=0).stop 2024-05-10T10:35:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0104180 msgr2=0x7f2df019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0104180 0x7f2df019fd00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800b530 tx=0x7f2dd800b8f0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 shutdown_connections 2024-05-10T10:35:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2ddc0b2900 0x7f2ddc0b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2df0109660 0x7f2df01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df01037c0 0x7f2df019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 --2- 172.21.15.112:0/506070544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df0104180 0x7f2df019fd00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 >> 172.21.15.112:0/506070544 conn(0x7f2df007b430 msgr2=0x7f2df01024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:49.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.910+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 shutdown_connections 2024-05-10T10:35:49.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:49.911+0000 7f2df59ec700 1 -- 172.21.15.112:0/506070544 wait complete. 2024-05-10T10:35:49.917 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-05-10T10:35:50.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:49 smithi112 ceph-mon[30100]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T10:35:50.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:49 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/506070544' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T10:35:50.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:49 smithi148 ceph-mon[33568]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T10:35:50.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:49 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/506070544' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T10:35:50.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:49 smithi195 ceph-mon[33773]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T10:35:50.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:49 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/506070544' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T10:35:50.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:35:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:51.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:50 smithi112 ceph-mon[30100]: pgmap v283: 33 pgs: 10 creating+peering, 6 active+clean, 17 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:51.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:50 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/506070544' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T10:35:51.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:50 smithi112 ceph-mon[30100]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T10:35:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:50 smithi148 ceph-mon[33568]: pgmap v283: 33 pgs: 10 creating+peering, 6 active+clean, 17 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:50 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/506070544' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T10:35:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:50 smithi148 ceph-mon[33568]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T10:35:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:50 smithi195 ceph-mon[33773]: pgmap v283: 33 pgs: 10 creating+peering, 6 active+clean, 17 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:50 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/506070544' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T10:35:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:50 smithi195 ceph-mon[33773]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T10:35:52.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:51 smithi112 ceph-mon[30100]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T10:35:52.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:51 smithi148 ceph-mon[33568]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T10:35:52.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:51 smithi195 ceph-mon[33773]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T10:35:52.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 -- 172.21.15.112:0/8383051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 msgr2=0x7efe58102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 --2- 172.21.15.112:0/8383051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe58102850 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe48009990 tx=0x7efe48009ca0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 -- 172.21.15.112:0/8383051 shutdown_connections 2024-05-10T10:35:52.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 --2- 172.21.15.112:0/8383051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe58102d90 0x7efe5810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 --2- 172.21.15.112:0/8383051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe581042f0 0x7efe581046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 --2- 172.21.15.112:0/8383051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe58102850 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 -- 172.21.15.112:0/8383051 >> 172.21.15.112:0/8383051 conn(0x7efe58078410 msgr2=0x7efe58078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 -- 172.21.15.112:0/8383051 shutdown_connections 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 -- 172.21.15.112:0/8383051 wait complete. 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 Processor -- start 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.644+0000 7efe5faa7700 1 -- start start 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5faa7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe5819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5faa7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe58102d90 0x7efe5819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5faa7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe581042f0 0x7efe581a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5faa7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe58112500 con 0x7efe581023d0 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5faa7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe58112380 con 0x7efe581042f0 2024-05-10T10:35:52.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5faa7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe58112680 con 0x7efe58102d90 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe5819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe5819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33316/0 (socket says 172.21.15.112:33316) 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 -- 172.21.15.112:0/3093711659 learned_addr learned my addr 172.21.15.112:0/3093711659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d042700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe58102d90 0x7efe5819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5e044700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe581042f0 0x7efe581a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 -- 172.21.15.112:0/3093711659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe58102d90 msgr2=0x7efe5819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe58102d90 0x7efe5819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 -- 172.21.15.112:0/3093711659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe581042f0 msgr2=0x7efe581a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe581042f0 0x7efe581a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 -- 172.21.15.112:0/3093711659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe581a0db0 con 0x7efe581023d0 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe5d843700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe5819bda0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efe5400d950 tx=0x7efe5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.645+0000 7efe4effd700 1 -- 172.21.15.112:0/3093711659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54016070 con 0x7efe581023d0 2024-05-10T10:35:52.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.646+0000 7efe4effd700 1 -- 172.21.15.112:0/3093711659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe54009dd0 con 0x7efe581023d0 2024-05-10T10:35:52.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.646+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe581a10a0 con 0x7efe581023d0 2024-05-10T10:35:52.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.646+0000 7efe4effd700 1 -- 172.21.15.112:0/3093711659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5400f3c0 con 0x7efe581023d0 2024-05-10T10:35:52.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.646+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a8920 con 0x7efe581023d0 2024-05-10T10:35:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.647+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5806a190 con 0x7efe581023d0 2024-05-10T10:35:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.648+0000 7efe4effd700 1 -- 172.21.15.112:0/3093711659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7efe54019070 con 0x7efe581023d0 2024-05-10T10:35:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.649+0000 7efe4effd700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.649+0000 7efe5d042700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.649+0000 7efe4effd700 1 -- 172.21.15.112:0/3093711659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7efe540e1b70 con 0x7efe581023d0 2024-05-10T10:35:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.649+0000 7efe5d042700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efe4800b5c0 tx=0x7efe48005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.652+0000 7efe4effd700 1 -- 172.21.15.112:0/3093711659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efe540a9a40 con 0x7efe581023d0 2024-05-10T10:35:52.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.901+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efe581067d0 con 0x7efe581023d0 2024-05-10T10:35:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.928+0000 7efe4effd700 1 -- 172.21.15.112:0/3093711659 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7efe540a9340 con 0x7efe581023d0 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 msgr2=0x7efe440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efe4800b5c0 tx=0x7efe48005960 comp rx=0 tx=0).stop 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 msgr2=0x7efe5819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe5819bda0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efe5400d950 tx=0x7efe5400dc60 comp rx=0 tx=0).stop 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 shutdown_connections 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe581042f0 0x7efe581a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe581023d0 0x7efe5819bda0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 --2- 172.21.15.112:0/3093711659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe58102d90 0x7efe5819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 >> 172.21.15.112:0/3093711659 conn(0x7efe58078410 msgr2=0x7efe58100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 shutdown_connections 2024-05-10T10:35:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:52.930+0000 7efe5faa7700 1 -- 172.21.15.112:0/3093711659 wait complete. 2024-05-10T10:35:52.933 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T10:35:53.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:52 smithi112 ceph-mon[30100]: pgmap v286: 97 pgs: 21 creating+peering, 38 active+clean, 38 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:53.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:52 smithi112 ceph-mon[30100]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T10:35:53.251 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:52 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3093711659' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T10:35:53.251 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:52 smithi112 ceph-mon[30100]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-10T10:35:53.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:52 smithi148 ceph-mon[33568]: pgmap v286: 97 pgs: 21 creating+peering, 38 active+clean, 38 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:53.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:52 smithi148 ceph-mon[33568]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T10:35:53.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:52 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3093711659' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T10:35:53.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:52 smithi148 ceph-mon[33568]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-10T10:35:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:52 smithi195 ceph-mon[33773]: pgmap v286: 97 pgs: 21 creating+peering, 38 active+clean, 38 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:52 smithi195 ceph-mon[33773]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T10:35:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:52 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3093711659' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T10:35:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:52 smithi195 ceph-mon[33773]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-10T10:35:53.689 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-10T10:35:53.689 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-10T10:35:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:54.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:53 smithi112 ceph-mon[30100]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T10:35:54.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:53 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3093711659' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T10:35:54.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:53 smithi112 ceph-mon[30100]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-10T10:35:54.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:53 smithi112 ceph-mon[30100]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-10T10:35:54.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:53 smithi148 ceph-mon[33568]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T10:35:54.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:53 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3093711659' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T10:35:54.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:53 smithi148 ceph-mon[33568]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-10T10:35:54.250 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:53 smithi148 ceph-mon[33568]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-10T10:35:54.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:53 smithi195 ceph-mon[33773]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T10:35:54.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:53 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3093711659' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T10:35:54.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:53 smithi195 ceph-mon[33773]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-10T10:35:54.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:53 smithi195 ceph-mon[33773]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-10T10:35:55.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:54 smithi112 ceph-mon[30100]: pgmap v289: 97 pgs: 11 creating+peering, 68 active+clean, 18 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:55.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:54 smithi112 ceph-mon[30100]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T10:35:55.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:54 smithi112 ceph-mon[30100]: Cluster is now healthy 2024-05-10T10:35:55.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:54 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] up:active 2024-05-10T10:35:55.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:54 smithi112 ceph-mon[30100]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:35:55.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:54 smithi112 ceph-mon[30100]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:35:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:54 smithi148 ceph-mon[33568]: pgmap v289: 97 pgs: 11 creating+peering, 68 active+clean, 18 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:54 smithi148 ceph-mon[33568]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T10:35:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:54 smithi148 ceph-mon[33568]: Cluster is now healthy 2024-05-10T10:35:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:54 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] up:active 2024-05-10T10:35:55.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:54 smithi148 ceph-mon[33568]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:35:55.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:54 smithi148 ceph-mon[33568]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:35:55.316 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:35:54 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: 2024-05-10T10:35:54.866+0000 7f2b9a8c7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:35:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:54 smithi195 ceph-mon[33773]: pgmap v289: 97 pgs: 11 creating+peering, 68 active+clean, 18 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:35:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:54 smithi195 ceph-mon[33773]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T10:35:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:54 smithi195 ceph-mon[33773]: Cluster is now healthy 2024-05-10T10:35:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:54 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] up:active 2024-05-10T10:35:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:54 smithi195 ceph-mon[33773]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:35:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:54 smithi195 ceph-mon[33773]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 -- 172.21.15.112:0/453405383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa670109510 msgr2=0x7fa6701098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 --2- 172.21.15.112:0/453405383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa670109510 0x7fa6701098f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa664009990 tx=0x7fa664009ca0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 -- 172.21.15.112:0/453405383 shutdown_connections 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 --2- 172.21.15.112:0/453405383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa67006c590 0x7fa670110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 --2- 172.21.15.112:0/453405383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa670109510 0x7fa6701098f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 --2- 172.21.15.112:0/453405383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa67006bbd0 0x7fa67006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 -- 172.21.15.112:0/453405383 >> 172.21.15.112:0/453405383 conn(0x7fa670079350 msgr2=0x7fa670079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 -- 172.21.15.112:0/453405383 shutdown_connections 2024-05-10T10:35:55.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.745+0000 7fa676c29700 1 -- 172.21.15.112:0/453405383 wait complete. 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 Processor -- start 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 -- start start 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa67006bbd0 0x7fa67019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa67006c590 0x7fa67019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa670109510 0x7fa6701998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6701134d0 con 0x7fa67006bbd0 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa670113350 con 0x7fa670109510 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa676c29700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa670113650 con 0x7fa67006c590 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.746+0000 7fa6749c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa67006bbd0 0x7fa67019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa67006bbd0 0x7fa67019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55714/0 (socket says 172.21.15.112:55714) 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 -- 172.21.15.112:0/4140784272 learned_addr learned my addr 172.21.15.112:0/4140784272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa66ffff700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa67006c590 0x7fa67019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6751c6700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa670109510 0x7fa6701998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 -- 172.21.15.112:0/4140784272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa67006c590 msgr2=0x7fa67019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa67006c590 0x7fa67019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 -- 172.21.15.112:0/4140784272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa670109510 msgr2=0x7fa6701998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa670109510 0x7fa6701998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 -- 172.21.15.112:0/4140784272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67019a030 con 0x7fa67006bbd0 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa6749c5700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa67006bbd0 0x7fa67019f710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa664000c00 tx=0x7fa6640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa66dffb700 1 -- 172.21.15.112:0/4140784272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664027070 con 0x7fa67006bbd0 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa66dffb700 1 -- 172.21.15.112:0/4140784272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa66400fb00 con 0x7fa67006bbd0 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa66dffb700 1 -- 172.21.15.112:0/4140784272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664022c40 con 0x7fa67006bbd0 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.747+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67019a2c0 con 0x7fa67006bbd0 2024-05-10T10:35:55.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.748+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6701a65e0 con 0x7fa67006bbd0 2024-05-10T10:35:55.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.749+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa67010b9f0 con 0x7fa67006bbd0 2024-05-10T10:35:55.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.751+0000 7fa66dffb700 1 -- 172.21.15.112:0/4140784272 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa66402a070 con 0x7fa67006bbd0 2024-05-10T10:35:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.752+0000 7fa66dffb700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6600b2880 0x7fa6600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.752+0000 7fa66ffff700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6600b2880 0x7fa6600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.752+0000 7fa66dffb700 1 -- 172.21.15.112:0/4140784272 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fa6640e33c0 con 0x7fa67006bbd0 2024-05-10T10:35:55.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.752+0000 7fa66ffff700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6600b2880 0x7fa6600b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa670079a90 tx=0x7fa65c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.755+0000 7fa66dffb700 1 -- 172.21.15.112:0/4140784272 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa664031070 con 0x7fa67006bbd0 2024-05-10T10:35:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:55.979+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa67006a190 con 0x7fa67006bbd0 2024-05-10T10:35:56.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.933+0000 7fa66dffb700 1 -- 172.21.15.112:0/4140784272 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7fa664013070 con 0x7fa67006bbd0 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6600b2880 msgr2=0x7fa6600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6600b2880 0x7fa6600b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa670079a90 tx=0x7fa65c00b410 comp rx=0 tx=0).stop 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa67006bbd0 msgr2=0x7fa67019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa67006bbd0 0x7fa67019f710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa664000c00 tx=0x7fa6640047b0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 shutdown_connections 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa670109510 0x7fa6701998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa67006bbd0 0x7fa67019f710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6600b2880 0x7fa6600b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 --2- 172.21.15.112:0/4140784272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa67006c590 0x7fa67019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.936+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 >> 172.21.15.112:0/4140784272 conn(0x7fa670079350 msgr2=0x7fa670101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:56.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.937+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 shutdown_connections 2024-05-10T10:35:56.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:56.937+0000 7fa676c29700 1 -- 172.21.15.112:0/4140784272 wait complete. 2024-05-10T10:35:57.268 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:56 smithi112 ceph-mon[30100]: pgmap v290: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 956 B/s wr, 3 op/s 2024-05-10T10:35:57.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:56 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4140784272' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T10:35:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:56 smithi148 ceph-mon[33568]: pgmap v290: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 956 B/s wr, 3 op/s 2024-05-10T10:35:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:56 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4140784272' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T10:35:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:56 smithi195 ceph-mon[33773]: pgmap v290: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 956 B/s wr, 3 op/s 2024-05-10T10:35:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:56 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4140784272' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T10:35:57.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:35:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:35:58.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:57 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4140784272' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T10:35:58.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:57 smithi112 ceph-mon[30100]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T10:35:58.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:57 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4140784272' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T10:35:58.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:57 smithi148 ceph-mon[33568]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T10:35:58.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:57 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4140784272' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T10:35:58.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:57 smithi195 ceph-mon[33773]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T10:35:59.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:58 smithi112 ceph-mon[30100]: pgmap v292: 97 pgs: 97 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-05-10T10:35:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:58 smithi148 ceph-mon[33568]: pgmap v292: 97 pgs: 97 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-05-10T10:35:59.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:58 smithi195 ceph-mon[33773]: pgmap v292: 97 pgs: 97 active+clean; 578 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 -- 172.21.15.112:0/3271245067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 msgr2=0x7f55b8106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 --2- 172.21.15.112:0/3271245067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b8106720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f55a80099c0 tx=0x7f55a8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 -- 172.21.15.112:0/3271245067 shutdown_connections 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 --2- 172.21.15.112:0/3271245067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55b8108b70 0x7f55b810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 --2- 172.21.15.112:0/3271245067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b8106720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 --2- 172.21.15.112:0/3271245067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b81081b0 0x7f55b8108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 -- 172.21.15.112:0/3271245067 >> 172.21.15.112:0/3271245067 conn(0x7f55b80776c0 msgr2=0x7f55b8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 -- 172.21.15.112:0/3271245067 shutdown_connections 2024-05-10T10:35:59.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.707+0000 7f55bcd94700 1 -- 172.21.15.112:0/3271245067 wait complete. 2024-05-10T10:35:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.708+0000 7f55bcd94700 1 Processor -- start 2024-05-10T10:35:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.708+0000 7f55bcd94700 1 -- start start 2024-05-10T10:35:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55b81081b0 0x7f55b819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b8108b70 0x7f55b81a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8112540 con 0x7f55b8106340 2024-05-10T10:35:59.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81123c0 con 0x7f55b8108b70 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81126c0 con 0x7f55b81081b0 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55728/0 (socket says 172.21.15.112:55728) 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 -- 172.21.15.112:0/541741079 learned_addr learned my addr 172.21.15.112:0/541741079 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b5d9b700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55b81081b0 0x7f55b819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 -- 172.21.15.112:0/541741079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55b81081b0 msgr2=0x7f55b819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55b81081b0 0x7f55b819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 -- 172.21.15.112:0/541741079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b8108b70 msgr2=0x7f55b81a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b8108b70 0x7f55b81a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 -- 172.21.15.112:0/541741079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81a5e40 con 0x7f55b8106340 2024-05-10T10:35:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b5d9b700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55b81081b0 0x7f55b819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55b659c700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b819a880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c00 tx=0x7f55a800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55af7fe700 1 -- 172.21.15.112:0/541741079 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a800f940 con 0x7f55b8106340 2024-05-10T10:35:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81a6010 con 0x7f55b8106340 2024-05-10T10:35:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55af7fe700 1 -- 172.21.15.112:0/541741079 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55a8004830 con 0x7f55b8106340 2024-05-10T10:35:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55af7fe700 1 -- 172.21.15.112:0/541741079 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a80054b0 con 0x7f55b8106340 2024-05-10T10:35:59.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.709+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81a6400 con 0x7f55b8106340 2024-05-10T10:35:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.710+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b80519e0 con 0x7f55b8106340 2024-05-10T10:35:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.712+0000 7f55af7fe700 1 -- 172.21.15.112:0/541741079 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f55a8029060 con 0x7f55b8106340 2024-05-10T10:35:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.712+0000 7f55af7fe700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55a40b2930 0x7f55a40b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.713+0000 7f55b5d9b700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55a40b2930 0x7f55a40b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.713+0000 7f55af7fe700 1 -- 172.21.15.112:0/541741079 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7f55a80e2f90 con 0x7f55b8106340 2024-05-10T10:35:59.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.713+0000 7f55b5d9b700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55a40b2930 0x7f55a40b4df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f55b8077bc0 tx=0x7f55a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:59.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.716+0000 7f55af7fe700 1 -- 172.21.15.112:0/541741079 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f55a8013070 con 0x7f55b8106340 2024-05-10T10:35:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:35:59.917+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f55b81a6750 con 0x7f55b8106340 2024-05-10T10:36:00.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:35:59 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/541741079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T10:36:00.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:35:59 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/541741079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T10:36:00.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:35:59 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/541741079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T10:36:00.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.958+0000 7f55af7fe700 1 -- 172.21.15.112:0/541741079 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f55a80aad10 con 0x7f55b8106340 2024-05-10T10:36:00.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55a40b2930 msgr2=0x7f55a40b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55a40b2930 0x7f55a40b4df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f55b8077bc0 tx=0x7f55a0008040 comp rx=0 tx=0).stop 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 msgr2=0x7f55b819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b819a880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c00 tx=0x7f55a800f510 comp rx=0 tx=0).stop 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 shutdown_connections 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b8108b70 0x7f55b81a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b8106340 0x7f55b819a880 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55a40b2930 0x7f55a40b4df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 --2- 172.21.15.112:0/541741079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55b81081b0 0x7f55b819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.962+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 >> 172.21.15.112:0/541741079 conn(0x7f55b80776c0 msgr2=0x7f55b80ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.963+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 shutdown_connections 2024-05-10T10:36:00.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:00.963+0000 7f55bcd94700 1 -- 172.21.15.112:0/541741079 wait complete. 2024-05-10T10:36:00.967 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data_ec' created 2024-05-10T10:36:01.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:00 smithi148 ceph-mon[33568]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-10T10:36:01.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:00 smithi148 ceph-mon[33568]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T10:36:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:00 smithi195 ceph-mon[33773]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-10T10:36:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:00 smithi195 ceph-mon[33773]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T10:36:01.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:00 smithi112 ceph-mon[30100]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-10T10:36:01.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:00 smithi112 ceph-mon[30100]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T10:36:01.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-10T10:36:01.952 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:02.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:01 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/541741079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T10:36:02.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:01 smithi112 ceph-mon[30100]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T10:36:02.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:01 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/541741079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T10:36:02.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:01 smithi148 ceph-mon[33568]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T10:36:02.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:01 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/541741079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T10:36:02.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:01 smithi195 ceph-mon[33773]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T10:36:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:02 smithi112 ceph-mon[30100]: pgmap v296: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-10T10:36:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:02 smithi112 ceph-mon[30100]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T10:36:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:02 smithi112 ceph-mon[30100]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T10:36:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:02 smithi148 ceph-mon[33568]: pgmap v296: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-10T10:36:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:02 smithi148 ceph-mon[33568]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T10:36:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:02 smithi148 ceph-mon[33568]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T10:36:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:02 smithi195 ceph-mon[33773]: pgmap v296: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-10T10:36:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:02 smithi195 ceph-mon[33773]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T10:36:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:02 smithi195 ceph-mon[33773]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T10:36:03.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 -- 172.21.15.112:0/1218915848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124102d60 msgr2=0x7f412410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:03.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 --2- 172.21.15.112:0/1218915848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124102d60 0x7f412410f250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b0a0 tx=0x7f411c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 -- 172.21.15.112:0/1218915848 shutdown_connections 2024-05-10T10:36:03.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 --2- 172.21.15.112:0/1218915848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124102d60 0x7f412410f250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 --2- 172.21.15.112:0/1218915848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4124108410 0x7f41241087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 --2- 172.21.15.112:0/1218915848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41241023a0 0x7f4124102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 -- 172.21.15.112:0/1218915848 >> 172.21.15.112:0/1218915848 conn(0x7f4124079350 msgr2=0x7f4124079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 -- 172.21.15.112:0/1218915848 shutdown_connections 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.701+0000 7f412b172700 1 -- 172.21.15.112:0/1218915848 wait complete. 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 Processor -- start 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 -- start start 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41241023a0 0x7f412419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4124102d60 0x7f412419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:03.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124108410 0x7f4124199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4124112460 con 0x7f4124108410 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41241122e0 con 0x7f41241023a0 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412b172700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41241125e0 con 0x7f4124102d60 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412970f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124108410 0x7f4124199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412970f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124108410 0x7f4124199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55750/0 (socket says 172.21.15.112:55750) 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f412970f700 1 -- 172.21.15.112:0/1551755185 learned_addr learned my addr 172.21.15.112:0/1551755185 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.702+0000 7f4128f0e700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41241023a0 0x7f412419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f4123fff700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4124102d60 0x7f412419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412970f700 1 -- 172.21.15.112:0/1551755185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4124102d60 msgr2=0x7f412419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412970f700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4124102d60 0x7f412419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412970f700 1 -- 172.21.15.112:0/1551755185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41241023a0 msgr2=0x7f412419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412970f700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41241023a0 0x7f412419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412970f700 1 -- 172.21.15.112:0/1551755185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412419a030 con 0x7f4124108410 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f4123fff700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4124102d60 0x7f412419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412970f700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124108410 0x7f4124199850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b830 tx=0x7f411c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f4121ffb700 1 -- 172.21.15.112:0/1551755185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c021070 con 0x7f4124108410 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412419a2c0 con 0x7f4124108410 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f4121ffb700 1 -- 172.21.15.112:0/1551755185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f411c007b80 con 0x7f4124108410 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f4121ffb700 1 -- 172.21.15.112:0/1551755185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c0052a0 con 0x7f4124108410 2024-05-10T10:36:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.703+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a6530 con 0x7f4124108410 2024-05-10T10:36:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.704+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4124103ac0 con 0x7f4124108410 2024-05-10T10:36:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.706+0000 7f4121ffb700 1 -- 172.21.15.112:0/1551755185 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f411c024070 con 0x7f4124108410 2024-05-10T10:36:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.707+0000 7f4121ffb700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41140b2900 0x7f41140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.707+0000 7f4128f0e700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41140b2900 0x7f41140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.707+0000 7f4121ffb700 1 -- 172.21.15.112:0/1551755185 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f411c0e3940 con 0x7f4124108410 2024-05-10T10:36:03.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.707+0000 7f4128f0e700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41140b2900 0x7f41140b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4118009a10 tx=0x7f4118009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.711+0000 7f4121ffb700 1 -- 172.21.15.112:0/1551755185 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f411c0ab4d0 con 0x7f4124108410 2024-05-10T10:36:03.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.919+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f41241a6700 con 0x7f4124108410 2024-05-10T10:36:03.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.979+0000 7f4121ffb700 1 -- 172.21.15.112:0/1551755185 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f411c0aadd0 con 0x7f4124108410 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41140b2900 msgr2=0x7f41140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41140b2900 0x7f41140b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4118009a10 tx=0x7f4118009190 comp rx=0 tx=0).stop 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124108410 msgr2=0x7f4124199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124108410 0x7f4124199850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b830 tx=0x7f411c004d00 comp rx=0 tx=0).stop 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 shutdown_connections 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4124108410 0x7f4124199850 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41241023a0 0x7f412419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41140b2900 0x7f41140b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 --2- 172.21.15.112:0/1551755185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4124102d60 0x7f412419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 >> 172.21.15.112:0/1551755185 conn(0x7f4124079350 msgr2=0x7f4124101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 shutdown_connections 2024-05-10T10:36:03.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:03.981+0000 7f412b172700 1 -- 172.21.15.112:0/1551755185 wait complete. 2024-05-10T10:36:03.984 INFO:teuthology.orchestra.run.smithi112.stderr:set pool 4 allow_ec_overwrites to true 2024-05-10T10:36:04.244 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:03 smithi112 ceph-mon[30100]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T10:36:04.244 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:03 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1551755185' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T10:36:04.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:03 smithi148 ceph-mon[33568]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T10:36:04.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:03 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1551755185' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T10:36:04.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:03 smithi195 ceph-mon[33773]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T10:36:04.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:03 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1551755185' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T10:36:04.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-10T10:36:05.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:04 smithi112 ceph-mon[30100]: pgmap v299: 129 pgs: 4 creating+peering, 20 unknown, 105 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:05.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:04 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1551755185' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T10:36:05.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:04 smithi112 ceph-mon[30100]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T10:36:05.106 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:05.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:04 smithi148 ceph-mon[33568]: pgmap v299: 129 pgs: 4 creating+peering, 20 unknown, 105 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:05.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:04 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1551755185' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T10:36:05.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:04 smithi148 ceph-mon[33568]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T10:36:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:04 smithi195 ceph-mon[33773]: pgmap v299: 129 pgs: 4 creating+peering, 20 unknown, 105 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:04 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1551755185' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T10:36:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:04 smithi195 ceph-mon[33773]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T10:36:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 -- 172.21.15.112:0/2593644752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 msgr2=0x7f3654109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 --2- 172.21.15.112:0/2593644752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f3654109920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36500099c0 tx=0x7f3650009cd0 comp rx=0 tx=0).stop 2024-05-10T10:36:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 -- 172.21.15.112:0/2593644752 shutdown_connections 2024-05-10T10:36:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 --2- 172.21.15.112:0/2593644752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654103f20 0x7f3654110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 --2- 172.21.15.112:0/2593644752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f3654109920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 --2- 172.21.15.112:0/2593644752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3654103560 0x7f36541039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 -- 172.21.15.112:0/2593644752 >> 172.21.15.112:0/2593644752 conn(0x7f365407a660 msgr2=0x7f365407aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 -- 172.21.15.112:0/2593644752 shutdown_connections 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 -- 172.21.15.112:0/2593644752 wait complete. 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 Processor -- start 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.913+0000 7f365c5de700 1 -- start start 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365c5de700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654103560 0x7f365419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365c5de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3654103f20 0x7f365419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365c5de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f36541996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365c5de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36541134d0 con 0x7f3654109540 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365c5de700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3654113350 con 0x7f3654103560 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365c5de700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3654113650 con 0x7f3654103f20 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f36541996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f36541996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37852/0 (socket says 172.21.15.112:37852) 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 -- 172.21.15.112:0/3202265667 learned_addr learned my addr 172.21.15.112:0/3202265667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:06.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f3659b79700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3654103f20 0x7f365419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365a37a700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654103560 0x7f365419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 -- 172.21.15.112:0/3202265667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3654103f20 msgr2=0x7f365419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3654103f20 0x7f365419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 -- 172.21.15.112:0/3202265667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654103560 msgr2=0x7f365419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654103560 0x7f365419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 -- 172.21.15.112:0/3202265667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3654199fd0 con 0x7f3654109540 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365ab7b700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f36541996f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f364c007d70 tx=0x7f364c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f364b7fe700 1 -- 172.21.15.112:0/3202265667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c016020 con 0x7f3654109540 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f364b7fe700 1 -- 172.21.15.112:0/3202265667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c00ad40 con 0x7f3654109540 2024-05-10T10:36:06.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365419a2c0 con 0x7f3654109540 2024-05-10T10:36:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.914+0000 7f364b7fe700 1 -- 172.21.15.112:0/3202265667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0115e0 con 0x7f3654109540 2024-05-10T10:36:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.915+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36541a65a0 con 0x7f3654109540 2024-05-10T10:36:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.915+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365410b9d0 con 0x7f3654109540 2024-05-10T10:36:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.917+0000 7f364b7fe700 1 -- 172.21.15.112:0/3202265667 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f364c019020 con 0x7f3654109540 2024-05-10T10:36:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.918+0000 7f364b7fe700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36400b2880 0x7f36400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.918+0000 7f365a37a700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36400b2880 0x7f36400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.918+0000 7f364b7fe700 1 -- 172.21.15.112:0/3202265667 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f364c0e28d0 con 0x7f3654109540 2024-05-10T10:36:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.918+0000 7f365a37a700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36400b2880 0x7f36400b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f36500058e0 tx=0x7f365001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:06.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:06.922+0000 7f364b7fe700 1 -- 172.21.15.112:0/3202265667 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f364c020020 con 0x7f3654109540 2024-05-10T10:36:07.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.178+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f365419ac20 con 0x7f3654109540 2024-05-10T10:36:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:06 smithi148 ceph-mon[33568]: pgmap v301: 129 pgs: 4 creating+peering, 4 unknown, 121 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:07.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:06 smithi195 ceph-mon[33773]: pgmap v301: 129 pgs: 4 creating+peering, 4 unknown, 121 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:07.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:06 smithi112 ceph-mon[30100]: pgmap v301: 129 pgs: 4 creating+peering, 4 unknown, 121 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:07.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.990+0000 7f364b7fe700 1 -- 172.21.15.112:0/3202265667 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f364c055e20 con 0x7f3654109540 2024-05-10T10:36:07.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36400b2880 msgr2=0x7f36400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:07.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36400b2880 0x7f36400b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f36500058e0 tx=0x7f365001a040 comp rx=0 tx=0).stop 2024-05-10T10:36:07.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 msgr2=0x7f36541996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f36541996f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f364c007d70 tx=0x7f364c00a340 comp rx=0 tx=0).stop 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 shutdown_connections 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3654109540 0x7f36541996f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654103560 0x7f365419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36400b2880 0x7f36400b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 --2- 172.21.15.112:0/3202265667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3654103f20 0x7f365419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 >> 172.21.15.112:0/3202265667 conn(0x7f365407a660 msgr2=0x7f3654102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 shutdown_connections 2024-05-10T10:36:07.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:07.993+0000 7f365c5de700 1 -- 172.21.15.112:0/3202265667 wait complete. 2024-05-10T10:36:07.996 INFO:teuthology.orchestra.run.smithi112.stderr:added data pool 4 to fsmap 2024-05-10T10:36:08.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:07 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3202265667' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T10:36:08.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:07 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3202265667' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T10:36:08.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:07 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3202265667' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T10:36:08.816 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:36:09.064 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:09.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:08 smithi112 ceph-mon[30100]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:09.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:08 smithi112 ceph-mon[30100]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T10:36:09.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:08 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3202265667' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T10:36:09.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:08 smithi112 ceph-mon[30100]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:08 smithi148 ceph-mon[33568]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:08 smithi148 ceph-mon[33568]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T10:36:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:08 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3202265667' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T10:36:09.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:08 smithi148 ceph-mon[33568]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:08 smithi195 ceph-mon[33773]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:08 smithi195 ceph-mon[33773]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T10:36:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:08 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3202265667' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T10:36:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:08 smithi195 ceph-mon[33773]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:10.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:10 smithi148 ceph-mon[33568]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T10:36:10.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:10 smithi148 ceph-mon[33568]: Cluster is now healthy 2024-05-10T10:36:10.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:10 smithi112 ceph-mon[30100]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T10:36:10.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:10 smithi112 ceph-mon[30100]: Cluster is now healthy 2024-05-10T10:36:10.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:10 smithi195 ceph-mon[33773]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T10:36:10.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:10 smithi195 ceph-mon[33773]: Cluster is now healthy 2024-05-10T10:36:10.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3000877074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 msgr2=0x7ff7a4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3000877074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a4102510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000b0a0 tx=0x7ff7a000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3000877074 shutdown_connections 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3000877074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a410fcb0 0x7ff7a4112150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3000877074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a4102510 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3000877074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a4102ae0 0x7ff7a410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3000877074 >> 172.21.15.112:0/3000877074 conn(0x7ff7a4077610 msgr2=0x7ff7a4077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3000877074 shutdown_connections 2024-05-10T10:36:10.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.906+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3000877074 wait complete. 2024-05-10T10:36:10.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 Processor -- start 2024-05-10T10:36:10.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 -- start start 2024-05-10T10:36:10.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a41a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:10.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a4102ae0 0x7ff7a41a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:10.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410fcb0 0x7ff7a419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:10.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41157a0 con 0x7ff7a4102130 2024-05-10T10:36:10.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4115620 con 0x7ff7a4102ae0 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7ac3d6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4115920 con 0x7ff7a410fcb0 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7aa172700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a41a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7aa172700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a41a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37870/0 (socket says 172.21.15.112:37870) 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7aa172700 1 -- 172.21.15.112:0/3986262075 learned_addr learned my addr 172.21.15.112:0/3986262075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7a9971700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a4102ae0 0x7ff7a41a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7aa172700 1 -- 172.21.15.112:0/3986262075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410fcb0 msgr2=0x7ff7a419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.907+0000 7ff7aa172700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410fcb0 0x7ff7a419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff7aa172700 1 -- 172.21.15.112:0/3986262075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a4102ae0 msgr2=0x7ff7a41a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff7aa172700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a4102ae0 0x7ff7a41a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff7aa172700 1 -- 172.21.15.112:0/3986262075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a419c440 con 0x7ff7a4102130 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff7aa172700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a41a1a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000b070 tx=0x7ff7a0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff79b7fe700 1 -- 172.21.15.112:0/3986262075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0021020 con 0x7ff7a4102130 2024-05-10T10:36:10.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a419c6d0 con 0x7ff7a4102130 2024-05-10T10:36:10.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff79b7fe700 1 -- 172.21.15.112:0/3986262075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7a0007d30 con 0x7ff7a4102130 2024-05-10T10:36:10.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff79b7fe700 1 -- 172.21.15.112:0/3986262075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00052e0 con 0x7ff7a4102130 2024-05-10T10:36:10.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.908+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a89c0 con 0x7ff7a4102130 2024-05-10T10:36:10.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.909+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a419ca30 con 0x7ff7a4102130 2024-05-10T10:36:10.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.910+0000 7ff79b7fe700 1 -- 172.21.15.112:0/3986262075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff7a0024020 con 0x7ff7a4102130 2024-05-10T10:36:10.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.911+0000 7ff79b7fe700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:10.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.911+0000 7ff7a9971700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:10.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.911+0000 7ff79b7fe700 1 -- 172.21.15.112:0/3986262075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7ff7a000f020 con 0x7ff7a4102130 2024-05-10T10:36:10.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.911+0000 7ff7a9971700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff794006fd0 tx=0x7ff794008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:10.914+0000 7ff79b7fe700 1 -- 172.21.15.112:0/3986262075 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff7a00ab7a0 con 0x7ff7a4102130 2024-05-10T10:36:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.169+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7a406a1c0 con 0x7ff7a4102130 2024-05-10T10:36:11.171 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:11 smithi112 ceph-mon[30100]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.170+0000 7ff79b7fe700 1 -- 172.21.15.112:0/3986262075 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11818 (secure 0 0 0) 0x7ff7a406a1c0 con 0x7ff7a4102130 2024-05-10T10:36:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:11.172 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:36:07.179672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24469,"qdb_cluster":[24469]},"id":1}]} 2024-05-10T10:36:11.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 msgr2=0x7ff7900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff794006fd0 tx=0x7ff794008040 comp rx=0 tx=0).stop 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 msgr2=0x7ff7a41a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a41a1a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000b070 tx=0x7ff7a0004ed0 comp rx=0 tx=0).stop 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 shutdown_connections 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410fcb0 0x7ff7a419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4102130 0x7ff7a41a1a40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 --2- 172.21.15.112:0/3986262075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a4102ae0 0x7ff7a41a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 >> 172.21.15.112:0/3986262075 conn(0x7ff7a4077610 msgr2=0x7ff7a41103c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 shutdown_connections 2024-05-10T10:36:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:11.172+0000 7ff7ac3d6700 1 -- 172.21.15.112:0/3986262075 wait complete. 2024-05-10T10:36:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-05-10T10:36:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:11 smithi148 ceph-mon[33568]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:11 smithi195 ceph-mon[33773]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:11.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:36:12.166 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:12 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3986262075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:36:12.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:12 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3986262075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:36:12.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:12 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3986262075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:36:13.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:13 smithi195 ceph-mon[33773]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:13 smithi112 ceph-mon[30100]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:13.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:13 smithi148 ceph-mon[33568]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:14.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 -- 172.21.15.112:0/2964064127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc108410 msgr2=0x7f1bbc1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:14.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 --2- 172.21.15.112:0/2964064127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc108410 0x7f1bbc1087f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009990 tx=0x7f1bb8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 -- 172.21.15.112:0/2964064127 shutdown_connections 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 --2- 172.21.15.112:0/2964064127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc102d60 0x7f1bbc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 --2- 172.21.15.112:0/2964064127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc108410 0x7f1bbc1087f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 --2- 172.21.15.112:0/2964064127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bbc1023a0 0x7f1bbc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 -- 172.21.15.112:0/2964064127 >> 172.21.15.112:0/2964064127 conn(0x7f1bbc079350 msgr2=0x7f1bbc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 -- 172.21.15.112:0/2964064127 shutdown_connections 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.121+0000 7f1bc3791700 1 -- 172.21.15.112:0/2964064127 wait complete. 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 Processor -- start 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 -- start start 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc1023a0 0x7f1bbc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bbc102d60 0x7f1bbc19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc108410 0x7f1bbc198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc112460 con 0x7f1bbc1023a0 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1122e0 con 0x7f1bbc108410 2024-05-10T10:36:14.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc3791700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1125e0 con 0x7f1bbc102d60 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc1023a0 0x7f1bbc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc1023a0 0x7f1bbc19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37878/0 (socket says 172.21.15.112:37878) 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 -- 172.21.15.112:0/3005054690 learned_addr learned my addr 172.21.15.112:0/3005054690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc1d2e700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc108410 0x7f1bbc198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc0d2c700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bbc102d60 0x7f1bbc19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 -- 172.21.15.112:0/3005054690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bbc102d60 msgr2=0x7f1bbc19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bbc102d60 0x7f1bbc19ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 -- 172.21.15.112:0/3005054690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc108410 msgr2=0x7f1bbc198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc108410 0x7f1bbc198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc152d700 1 -- 172.21.15.112:0/3005054690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc198fe0 con 0x7f1bbc1023a0 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.122+0000 7f1bc1d2e700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc108410 0x7f1bbc198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.123+0000 7f1bc152d700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc1023a0 0x7f1bbc19e6c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000c00 tx=0x7f1bb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.123+0000 7f1bb27fc700 1 -- 172.21.15.112:0/3005054690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8027070 con 0x7f1bbc1023a0 2024-05-10T10:36:14.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.123+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc199270 con 0x7f1bbc1023a0 2024-05-10T10:36:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.123+0000 7f1bb27fc700 1 -- 172.21.15.112:0/3005054690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bb800fdb0 con 0x7f1bbc1023a0 2024-05-10T10:36:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.123+0000 7f1bb27fc700 1 -- 172.21.15.112:0/3005054690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80173f0 con 0x7f1bbc1023a0 2024-05-10T10:36:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.123+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc06b180 con 0x7f1bbc1023a0 2024-05-10T10:36:14.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.124+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc103ac0 con 0x7f1bbc1023a0 2024-05-10T10:36:14.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.125+0000 7f1bb27fc700 1 -- 172.21.15.112:0/3005054690 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1bb802a070 con 0x7f1bbc1023a0 2024-05-10T10:36:14.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.126+0000 7f1bb27fc700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ba80b2880 0x7f1ba80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:14.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.126+0000 7f1bc0d2c700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ba80b2880 0x7f1ba80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:14.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.126+0000 7f1bb27fc700 1 -- 172.21.15.112:0/3005054690 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f1bb80e3a00 con 0x7f1bbc1023a0 2024-05-10T10:36:14.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.126+0000 7f1bc0d2c700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ba80b2880 0x7f1ba80b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006fd0 tx=0x7f1bac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:14.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.129+0000 7f1bb27fc700 1 -- 172.21.15.112:0/3005054690 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1bb8031070 con 0x7f1bbc1023a0 2024-05-10T10:36:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.333+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bbc199db0 con 0x7f1bbc1023a0 2024-05-10T10:36:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.334+0000 7f1bb27fc700 1 -- 172.21.15.112:0/3005054690 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24164 (secure 0 0 0) 0x7f1bb8013070 con 0x7f1bbc1023a0 2024-05-10T10:36:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:14.336 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:36:07.179539+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:36:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ba80b2880 msgr2=0x7f1ba80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ba80b2880 0x7f1ba80b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006fd0 tx=0x7f1bac008040 comp rx=0 tx=0).stop 2024-05-10T10:36:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc1023a0 msgr2=0x7f1bbc19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc1023a0 0x7f1bbc19e6c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000c00 tx=0x7f1bb80047b0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 shutdown_connections 2024-05-10T10:36:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bbc108410 0x7f1bbc198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bbc1023a0 0x7f1bbc19e6c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ba80b2880 0x7f1ba80b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 --2- 172.21.15.112:0/3005054690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bbc102d60 0x7f1bbc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 >> 172.21.15.112:0/3005054690 conn(0x7f1bbc079350 msgr2=0x7f1bbc101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 shutdown_connections 2024-05-10T10:36:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:14.337+0000 7f1bc3791700 1 -- 172.21.15.112:0/3005054690 wait complete. 2024-05-10T10:36:15.011 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:36:15.251 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:15 smithi195 ceph-mon[33773]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:15 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3005054690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:15.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:15 smithi112 ceph-mon[30100]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:15.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:15 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3005054690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:15.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:15 smithi148 ceph-mon[33568]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:15.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:15 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3005054690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 -- 172.21.15.112:0/3482806515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 msgr2=0x7f0e641099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 --2- 172.21.15.112:0/3482806515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641099b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0e540099c0 tx=0x7f0e54009cd0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 -- 172.21.15.112:0/3482806515 shutdown_connections 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 --2- 172.21.15.112:0/3482806515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e64103ef0 0x7f0e641103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 --2- 172.21.15.112:0/3482806515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641099b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 --2- 172.21.15.112:0/3482806515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e64103530 0x7f0e641039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 -- 172.21.15.112:0/3482806515 >> 172.21.15.112:0/3482806515 conn(0x7f0e6407a690 msgr2=0x7f0e6407aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 -- 172.21.15.112:0/3482806515 shutdown_connections 2024-05-10T10:36:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.057+0000 7f0e6884a700 1 -- 172.21.15.112:0/3482806515 wait complete. 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 Processor -- start 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 -- start start 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e64103530 0x7f0e6419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e64103ef0 0x7f0e6419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641135c0 con 0x7f0e641095d0 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64113440 con 0x7f0e64103530 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6884a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64113740 con 0x7f0e64103ef0 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e62d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e62d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39026/0 (socket says 172.21.15.112:39026) 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e62d9d700 1 -- 172.21.15.112:0/2906581765 learned_addr learned my addr 172.21.15.112:0/2906581765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:17.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e6259c700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e64103530 0x7f0e6419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.058+0000 7f0e61d9b700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e64103ef0 0x7f0e6419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e62d9d700 1 -- 172.21.15.112:0/2906581765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e64103ef0 msgr2=0x7f0e6419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e62d9d700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e64103ef0 0x7f0e6419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e62d9d700 1 -- 172.21.15.112:0/2906581765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e64103530 msgr2=0x7f0e6419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e62d9d700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e64103530 0x7f0e6419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e62d9d700 1 -- 172.21.15.112:0/2906581765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6419a000 con 0x7f0e641095d0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e6259c700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e64103530 0x7f0e6419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e62d9d700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641998a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c007ee0 tx=0x7f0e5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e5b7fe700 1 -- 172.21.15.112:0/2906581765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c004bb0 con 0x7f0e641095d0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6419a2f0 con 0x7f0e641095d0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e5b7fe700 1 -- 172.21.15.112:0/2906581765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e5c004d50 con 0x7f0e641095d0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e5b7fe700 1 -- 172.21.15.112:0/2906581765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0058b0 con 0x7f0e641095d0 2024-05-10T10:36:17.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.059+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e641a65b0 con 0x7f0e641095d0 2024-05-10T10:36:17.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.060+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6410ba60 con 0x7f0e641095d0 2024-05-10T10:36:17.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.061+0000 7f0e5b7fe700 1 -- 172.21.15.112:0/2906581765 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0e5c01a020 con 0x7f0e641095d0 2024-05-10T10:36:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.062+0000 7f0e5b7fe700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0e500b2880 0x7f0e500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.062+0000 7f0e6259c700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0e500b2880 0x7f0e500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.062+0000 7f0e5b7fe700 1 -- 172.21.15.112:0/2906581765 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f0e5c0e21d0 con 0x7f0e641095d0 2024-05-10T10:36:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.063+0000 7f0e6259c700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0e500b2880 0x7f0e500b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0e6419a9e0 tx=0x7f0e5401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.066+0000 7f0e5b7fe700 1 -- 172.21.15.112:0/2906581765 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0e5c010070 con 0x7f0e641095d0 2024-05-10T10:36:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.271+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e6419add0 con 0x7f0e641095d0 2024-05-10T10:36:17.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:17 smithi112 ceph-mon[30100]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:17.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.273+0000 7f0e5b7fe700 1 -- 172.21.15.112:0/2906581765 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24164 (secure 0 0 0) 0x7f0e5c0a9c50 con 0x7f0e641095d0 2024-05-10T10:36:17.275 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:17.275 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:36:07.179539+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:36:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0e500b2880 msgr2=0x7f0e500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0e500b2880 0x7f0e500b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0e6419a9e0 tx=0x7f0e5401a040 comp rx=0 tx=0).stop 2024-05-10T10:36:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 msgr2=0x7f0e641998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641998a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c007ee0 tx=0x7f0e5c00d3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 shutdown_connections 2024-05-10T10:36:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e641095d0 0x7f0e641998a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e64103530 0x7f0e6419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0e500b2880 0x7f0e500b4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 --2- 172.21.15.112:0/2906581765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e64103ef0 0x7f0e6419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.275+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 >> 172.21.15.112:0/2906581765 conn(0x7f0e6407a690 msgr2=0x7f0e641022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.276+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 shutdown_connections 2024-05-10T10:36:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:17.276+0000 7f0e6884a700 1 -- 172.21.15.112:0/2906581765 wait complete. 2024-05-10T10:36:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:17 smithi195 ceph-mon[33773]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:17.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:17 smithi148 ceph-mon[33568]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:18.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:36:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:18.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:18 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2906581765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:18.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:18 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2906581765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:18.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:18 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2906581765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:19 smithi195 ceph-mon[33773]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:19.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:19 smithi112 ceph-mon[30100]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:19.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:19 smithi148 ceph-mon[33568]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.242+0000 7f207d469700 1 -- 172.21.15.112:0/1828251702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f207806bb70 msgr2=0x7f207806bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.242+0000 7f207d469700 1 --2- 172.21.15.112:0/1828251702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f207806bb70 0x7f207806bff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f206c009990 tx=0x7f206c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 -- 172.21.15.112:0/1828251702 shutdown_connections 2024-05-10T10:36:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 --2- 172.21.15.112:0/1828251702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207806c530 0x7f2078110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 --2- 172.21.15.112:0/1828251702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2078109520 0x7f2078109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 --2- 172.21.15.112:0/1828251702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f207806bb70 0x7f207806bff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 -- 172.21.15.112:0/1828251702 >> 172.21.15.112:0/1828251702 conn(0x7f2078079350 msgr2=0x7f2078079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:20.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 -- 172.21.15.112:0/1828251702 shutdown_connections 2024-05-10T10:36:20.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 -- 172.21.15.112:0/1828251702 wait complete. 2024-05-10T10:36:20.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 Processor -- start 2024-05-10T10:36:20.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.243+0000 7f207d469700 1 -- start start 2024-05-10T10:36:20.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207806bb70 0x7f207819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f207806c530 0x7f207819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2078109520 0x7f2078199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2078113500 con 0x7f2078109520 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2078113380 con 0x7f207806bb70 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2078113680 con 0x7f207806c530 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2078109520 0x7f2078199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2078109520 0x7f2078199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39040/0 (socket says 172.21.15.112:39040) 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f2076ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207806bb70 0x7f207819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 -- 172.21.15.112:0/812165664 learned_addr learned my addr 172.21.15.112:0/812165664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20767fc700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f207806c530 0x7f207819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 -- 172.21.15.112:0/812165664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f207806c530 msgr2=0x7f207819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:20.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f207806c530 0x7f207819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 -- 172.21.15.112:0/812165664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207806bb70 msgr2=0x7f207819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207806bb70 0x7f207819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 -- 172.21.15.112:0/812165664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207819a0a0 con 0x7f2078109520 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f20777fe700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2078109520 0x7f2078199940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f207000ada0 tx=0x7f207000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f205ffff700 1 -- 172.21.15.112:0/812165664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070017020 con 0x7f2078109520 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207819a390 con 0x7f2078109520 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f205ffff700 1 -- 172.21.15.112:0/812165664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f207000cd60 con 0x7f2078109520 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f205ffff700 1 -- 172.21.15.112:0/812165664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070005280 con 0x7f2078109520 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.244+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781a6600 con 0x7f2078109520 2024-05-10T10:36:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.245+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207810ba00 con 0x7f2078109520 2024-05-10T10:36:20.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.247+0000 7f205ffff700 1 -- 172.21.15.112:0/812165664 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f207000c8d0 con 0x7f2078109520 2024-05-10T10:36:20.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.248+0000 7f205ffff700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20600b2880 0x7f20600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:20.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.248+0000 7f2076ffd700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20600b2880 0x7f20600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:20.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.248+0000 7f205ffff700 1 -- 172.21.15.112:0/812165664 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f20700e21d0 con 0x7f2078109520 2024-05-10T10:36:20.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.248+0000 7f2076ffd700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20600b2880 0x7f20600b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2078100f40 tx=0x7f206800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.252+0000 7f205ffff700 1 -- 172.21.15.112:0/812165664 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2070021020 con 0x7f2078109520 2024-05-10T10:36:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.504+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f207810b3e0 con 0x7f2078109520 2024-05-10T10:36:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.504+0000 7f205ffff700 1 -- 172.21.15.112:0/812165664 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11818 (secure 0 0 0) 0x7f20700a9d40 con 0x7f2078109520 2024-05-10T10:36:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:20.506 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:36:07.179672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24469,"qdb_cluster":[24469]},"id":1}]} 2024-05-10T10:36:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20600b2880 msgr2=0x7f20600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20600b2880 0x7f20600b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2078100f40 tx=0x7f206800b3c0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2078109520 msgr2=0x7f2078199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2078109520 0x7f2078199940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f207000ada0 tx=0x7f207000c330 comp rx=0 tx=0).stop 2024-05-10T10:36:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 shutdown_connections 2024-05-10T10:36:20.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2078109520 0x7f2078199940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207806bb70 0x7f207819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20600b2880 0x7f20600b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 --2- 172.21.15.112:0/812165664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f207806c530 0x7f207819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 >> 172.21.15.112:0/812165664 conn(0x7f2078079350 msgr2=0x7f2078101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 shutdown_connections 2024-05-10T10:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:20.507+0000 7f207d469700 1 -- 172.21.15.112:0/812165664 wait complete. 2024-05-10T10:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-05-10T10:36:21.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:21 smithi112 ceph-mon[30100]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:21.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:21 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/812165664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:36:21.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:36:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:21 smithi195 ceph-mon[33773]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:21 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/812165664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:36:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:21.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:21 smithi148 ceph-mon[33568]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:21.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:21 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/812165664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:36:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 -- 172.21.15.112:0/4271102421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474102390 msgr2=0x7fd474102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 --2- 172.21.15.112:0/4271102421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474102390 0x7fd474102810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd468009990 tx=0x7fd468009ca0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 -- 172.21.15.112:0/4271102421 shutdown_connections 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 --2- 172.21.15.112:0/4271102421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474102d50 0x7fd47410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 --2- 172.21.15.112:0/4271102421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd474108400 0x7fd4741087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 --2- 172.21.15.112:0/4271102421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474102390 0x7fd474102810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 -- 172.21.15.112:0/4271102421 >> 172.21.15.112:0/4271102421 conn(0x7fd474079310 msgr2=0x7fd474079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 -- 172.21.15.112:0/4271102421 shutdown_connections 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 -- 172.21.15.112:0/4271102421 wait complete. 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.093+0000 7fd4793dc700 1 Processor -- start 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4793dc700 1 -- start start 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4793dc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474102390 0x7fd47419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4793dc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd474102d50 0x7fd47419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4793dc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474108400 0x7fd4741987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4793dc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd474112400 con 0x7fd474108400 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4793dc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd474112280 con 0x7fd474102d50 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4793dc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd474112580 con 0x7fd474102390 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474108400 0x7fd4741987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474108400 0x7fd4741987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39052/0 (socket says 172.21.15.112:39052) 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 -- 172.21.15.112:0/2745471589 learned_addr learned my addr 172.21.15.112:0/2745471589 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4727fc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd474102d50 0x7fd47419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd472ffd700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474102390 0x7fd47419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 -- 172.21.15.112:0/2745471589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474102390 msgr2=0x7fd47419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474102390 0x7fd47419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 -- 172.21.15.112:0/2745471589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd474102d50 msgr2=0x7fd47419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd474102d50 0x7fd47419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4737fe700 1 -- 172.21.15.112:0/2745471589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd474198fc0 con 0x7fd474108400 2024-05-10T10:36:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd472ffd700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474102390 0x7fd47419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.094+0000 7fd4727fc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd474102d50 0x7fd47419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.095+0000 7fd4737fe700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474108400 0x7fd4741987e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd46c007ee0 tx=0x7fd46c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.095+0000 7fd45bfff700 1 -- 172.21.15.112:0/2745471589 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00dbd0 con 0x7fd474108400 2024-05-10T10:36:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.095+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4741992b0 con 0x7fd474108400 2024-05-10T10:36:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.095+0000 7fd45bfff700 1 -- 172.21.15.112:0/2745471589 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd46c00dd70 con 0x7fd474108400 2024-05-10T10:36:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.095+0000 7fd45bfff700 1 -- 172.21.15.112:0/2745471589 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0058d0 con 0x7fd474108400 2024-05-10T10:36:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.095+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47406ba10 con 0x7fd474108400 2024-05-10T10:36:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.096+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd474103ab0 con 0x7fd474108400 2024-05-10T10:36:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.097+0000 7fd45bfff700 1 -- 172.21.15.112:0/2745471589 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd46c01a020 con 0x7fd474108400 2024-05-10T10:36:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.098+0000 7fd45bfff700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd45c0b2880 0x7fd45c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.098+0000 7fd472ffd700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd45c0b2880 0x7fd45c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.098+0000 7fd45bfff700 1 -- 172.21.15.112:0/2745471589 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd46c0e1ca0 con 0x7fd474108400 2024-05-10T10:36:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.098+0000 7fd472ffd700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd45c0b2880 0x7fd45c0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd464005e90 tx=0x7fd464005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.102+0000 7fd45bfff700 1 -- 172.21.15.112:0/2745471589 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd46c010070 con 0x7fd474108400 2024-05-10T10:36:23.103 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:23 smithi112 ceph-mon[30100]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:23.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.311+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd47406c4e0 con 0x7fd474108400 2024-05-10T10:36:23.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.312+0000 7fd45bfff700 1 -- 172.21.15.112:0/2745471589 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24164 (secure 0 0 0) 0x7fd46c0a9780 con 0x7fd474108400 2024-05-10T10:36:23.314 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:23.314 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:36:07.179539+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.314+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd45c0b2880 msgr2=0x7fd45c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.314+0000 7fd4793dc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd45c0b2880 0x7fd45c0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd464005e90 tx=0x7fd464005d20 comp rx=0 tx=0).stop 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.314+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474108400 msgr2=0x7fd4741987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.314+0000 7fd4793dc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474108400 0x7fd4741987e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd46c007ee0 tx=0x7fd46c00d3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 shutdown_connections 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474108400 0x7fd4741987e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd45c0b2880 0x7fd45c0b4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474102390 0x7fd47419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 --2- 172.21.15.112:0/2745471589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd474102d50 0x7fd47419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 >> 172.21.15.112:0/2745471589 conn(0x7fd474079310 msgr2=0x7fd4741010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:23.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 shutdown_connections 2024-05-10T10:36:23.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:23.315+0000 7fd4793dc700 1 -- 172.21.15.112:0/2745471589 wait complete. 2024-05-10T10:36:23.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:23 smithi148 ceph-mon[33568]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:23.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:23 smithi195 ceph-mon[33773]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:23.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:35:52.903233+0000', 'modified': '2024-05-10T10:36:07.179672+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.195:6839/2238321568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 2238321568}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 2238321568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24469, 'qdb_cluster': [24469]} 2024-05-10T10:36:23.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:36:23.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T10:36:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:24.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:24 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2745471589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:24.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:24 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2745471589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:24.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:24 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2745471589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:25.298 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:25 smithi112 ceph-mon[30100]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:25.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:25 smithi148 ceph-mon[33568]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:25.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:25 smithi195 ceph-mon[33773]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 -- 172.21.15.112:0/1113591848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c107ba0 msgr2=0x7f883c107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 --2- 172.21.15.112:0/1113591848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c107ba0 0x7f883c107f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f882c0099c0 tx=0x7f882c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 -- 172.21.15.112:0/1113591848 shutdown_connections 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 --2- 172.21.15.112:0/1113591848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f883c102890 0x7f883c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 --2- 172.21.15.112:0/1113591848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c107ba0 0x7f883c107f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 --2- 172.21.15.112:0/1113591848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883c101ed0 0x7f883c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 -- 172.21.15.112:0/1113591848 >> 172.21.15.112:0/1113591848 conn(0x7f883c077610 msgr2=0x7f883c077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 -- 172.21.15.112:0/1113591848 shutdown_connections 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.844+0000 7f884292e700 1 -- 172.21.15.112:0/1113591848 wait complete. 2024-05-10T10:36:25.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 Processor -- start 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 -- start start 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f883c101ed0 0x7f883c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c102890 0x7f883c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883c107ba0 0x7f883c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f883c111bb0 con 0x7f883c102890 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f883c111a30 con 0x7f883c101ed0 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f884292e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f883c111d30 con 0x7f883c107ba0 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f883b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c102890 0x7f883c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f883b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c102890 0x7f883c19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34052/0 (socket says 172.21.15.112:34052) 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f883b7fe700 1 -- 172.21.15.112:0/1696821157 learned_addr learned my addr 172.21.15.112:0/1696821157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.845+0000 7f883bfff700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f883c101ed0 0x7f883c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f8840ecb700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883c107ba0 0x7f883c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f883b7fe700 1 -- 172.21.15.112:0/1696821157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883c107ba0 msgr2=0x7f883c1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f883b7fe700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883c107ba0 0x7f883c1a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f883b7fe700 1 -- 172.21.15.112:0/1696821157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f883c101ed0 msgr2=0x7f883c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f883b7fe700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f883c101ed0 0x7f883c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f883b7fe700 1 -- 172.21.15.112:0/1696821157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c1a5f50 con 0x7f883c102890 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f883bfff700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f883c101ed0 0x7f883c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f883b7fe700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c102890 0x7f883c19aed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f883000c970 tx=0x7f883000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f88397fa700 1 -- 172.21.15.112:0/1696821157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88300078d0 con 0x7f883c102890 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f88397fa700 1 -- 172.21.15.112:0/1696821157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8830018480 con 0x7f883c102890 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c1a6180 con 0x7f883c102890 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f88397fa700 1 -- 172.21.15.112:0/1696821157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883000f6b0 con 0x7f883c102890 2024-05-10T10:36:25.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.846+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1a6690 con 0x7f883c102890 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.849+0000 7f88397fa700 1 -- 172.21.15.112:0/1696821157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8830018930 con 0x7f883c102890 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.849+0000 7f88397fa700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88240b2900 0x7f88240b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.849+0000 7f883bfff700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88240b2900 0x7f88240b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.849+0000 7f88397fa700 1 -- 172.21.15.112:0/1696821157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f88300e24c0 con 0x7f883c102890 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f883bfff700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88240b2900 0x7f88240b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f882c000c00 tx=0x7f882c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 Processor -- start 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 -- start start 2024-05-10T10:36:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0434860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e04386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0386810 con 0x55e8e0438c30 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0325770 con 0x55e8e0434480 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8e03827d0 con 0x55e8e0434da0 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f8840ecb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f8840ecb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34066/0 (socket says 172.21.15.112:34066) 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.850+0000 7f8840ecb700 1 -- 172.21.15.112:0/2708976150 learned_addr learned my addr 172.21.15.112:0/2708976150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f883bfff700 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0434860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f883b7fe700 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e04386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f8840ecb700 1 -- 172.21.15.112:0/2708976150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 msgr2=0x55e8e04386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f8840ecb700 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e04386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f8840ecb700 1 -- 172.21.15.112:0/2708976150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 msgr2=0x55e8e0434860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f8840ecb700 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0434860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f8840ecb700 1 -- 172.21.15.112:0/2708976150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e02d3880 con 0x55e8e0438c30 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f883bfff700 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0434860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f8840ecb700 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04390b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f883400a580 tx=0x7f88340078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2321d3c2775f1e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f88227fc700 1 -- 172.21.15.112:0/2708976150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834003a00 con 0x55e8e0438c30 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f88227fc700 1 -- 172.21.15.112:0/2708976150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8834003ba0 con 0x55e8e0438c30 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.851+0000 7f88227fc700 1 -- 172.21.15.112:0/2708976150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834014600 con 0x55e8e0438c30 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2708976150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 msgr2=0x55e8e04390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04390b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f883400a580 tx=0x7f88340078d0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2708976150 shutdown_connections 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04390b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0434860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2708976150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e04386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2708976150 >> 172.21.15.112:0/2708976150 conn(0x55e8e03600b0 msgr2=0x55e8e042ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2708976150 shutdown_connections 2024-05-10T10:36:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2708976150 wait complete. 2024-05-10T10:36:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 -- start start 2024-05-10T10:36:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.852+0000 7f884d1c1b80 1 Processor -- start 2024-05-10T10:36:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e043ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434da0 0x55e8e043d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 0x55e8e043d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0440de0 con 0x55e8e0434da0 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8e02ecbe0 con 0x55e8e0434480 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0440f60 con 0x55e8e0438c30 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f884d1c1b80 10 client.0 _ll_register_callbacks cb 0x55e8e031e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434da0 0x55e8e043d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434da0 0x55e8e043d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34076/0 (socket says 172.21.15.112:34076) 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 -- 172.21.15.112:0/655692970 learned_addr learned my addr 172.21.15.112:0/655692970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f8840ecb700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 0x55e8e043d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883bfff700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e043ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 -- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 msgr2=0x55e8e043d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 0x55e8e043d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 -- 172.21.15.112:0/655692970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 msgr2=0x55e8e043ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e043ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 -- 172.21.15.112:0/655692970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e0322640 con 0x55e8e0434da0 2024-05-10T10:36:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883bfff700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e043ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f883b7fe700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434da0 0x55e8e043d320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88300dfd80 tx=0x7f88300a9e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e62e6e744ca097 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f881b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.853+0000 7f881b7fe700 1 -- 172.21.15.112:0/655692970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88300a9440 con 0x55e8e0434da0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8834003ba0 con 0x55e8e0434da0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f881b7fe700 1 -- 172.21.15.112:0/655692970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f88300b00b0 con 0x55e8e0434da0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f881b7fe700 1 -- 172.21.15.112:0/655692970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f88300aa030 con 0x55e8e0434da0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 10 client.14706 fetch_fsmap learned FSMap version 17 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e8e044b580 con 0x55e8e0434da0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f881b7fe700 1 -- 172.21.15.112:0/655692970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8834003ba0 con 0x55e8e0434da0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f881b7fe700 1 -- 172.21.15.112:0/655692970 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f88300a8b90 con 0x55e8e0434da0 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 10 client.14706 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T10:36:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 10 client.14706 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 20 client.14706 populate_metadata read hostname 'smithi112' 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e044dc30 0x55e8e044f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 4 client.14706 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.854+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e0400600 con 0x55e8e044dc30 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.855+0000 7f883bfff700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e044dc30 0x55e8e044f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.855+0000 7f883bfff700 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e044dc30 0x55e8e044f0a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146957a5ce4d9e93 server_cookie=3832143d5ed60f38 in_seq=0 out_seq=0 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.855+0000 7f881b7fe700 10 client.14706 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f881b7fe700 1 -- 172.21.15.112:0/655692970 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8e0400600 con 0x55e8e044dc30 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f881b7fe700 10 client.14706 handle_command_reply: tid=0 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 client.14706 shutdown 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434da0 msgr2=0x55e8e043d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434da0 0x55e8e043d320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88300dfd80 tx=0x7f88300a9e70 comp rx=0 tx=0).stop 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 shutdown_connections 2024-05-10T10:36:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e044dc30 0x55e8e044f0a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 0x55e8e043d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e043ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/655692970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434da0 0x55e8e043d320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f881b7fe700 0 client.14706 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 >> 172.21.15.112:0/655692970 conn(0x55e8e03600b0 msgr2=0x55e8e0439030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 shutdown_connections 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 1 -- 172.21.15.112:0/655692970 wait complete. 2024-05-10T10:36:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.856+0000 7f884d1c1b80 20 client.14706 trim_cache size 0 max 16384 2024-05-10T10:36:25.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 Processor -- start 2024-05-10T10:36:25.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 -- start start 2024-05-10T10:36:25.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434480 0x55e8e04365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434da0 0x55e8e0436ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 0x55e8e0428e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8e044d3e0 con 0x55e8e0434480 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8e044dab0 con 0x55e8e0434da0 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.880+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8e044e180 con 0x55e8e0438c30 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434480 0x55e8e04365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434480 0x55e8e04365a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34080/0 (socket says 172.21.15.112:34080) 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 -- 172.21.15.112:0/2795626513 learned_addr learned my addr 172.21.15.112:0/2795626513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883b7fe700 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434da0 0x55e8e0436ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 -- 172.21.15.112:0/2795626513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 msgr2=0x55e8e0428e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 0x55e8e0428e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 -- 172.21.15.112:0/2795626513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434da0 msgr2=0x55e8e0436ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434da0 0x55e8e0436ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 -- 172.21.15.112:0/2795626513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e044d8d0 con 0x55e8e0434480 2024-05-10T10:36:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f883bfff700 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434480 0x55e8e04365a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f882c0099c0 tx=0x7f882c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19659ea4d78ea666 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f8821ffb700 1 -- 172.21.15.112:0/2795626513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c004bf0 con 0x55e8e0434480 2024-05-10T10:36:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f8821ffb700 1 -- 172.21.15.112:0/2795626513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f882c033e50 con 0x55e8e0434480 2024-05-10T10:36:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.881+0000 7f8821ffb700 1 -- 172.21.15.112:0/2795626513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c00f6c0 con 0x55e8e0434480 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2795626513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434480 msgr2=0x55e8e04365a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434480 0x55e8e04365a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f882c0099c0 tx=0x7f882c01f000 comp rx=0 tx=0).stop 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2795626513 shutdown_connections 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0438c30 0x55e8e0428e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0434480 0x55e8e04365a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/2795626513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434da0 0x55e8e0436ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2795626513 >> 172.21.15.112:0/2795626513 conn(0x55e8e03600b0 msgr2=0x55e8e042eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2795626513 shutdown_connections 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 -- 172.21.15.112:0/2795626513 wait complete. 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 -- start start 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.882+0000 7f884d1c1b80 1 Processor -- start 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0458520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e0458a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8e044ef20 con 0x55e8e0438c30 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8e044e850 con 0x55e8e0434480 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8e044f5f0 con 0x55e8e0434da0 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 10 client.0 _ll_register_callbacks cb 0x55e8e031e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34084/0 (socket says 172.21.15.112:34084) 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 -- 172.21.15.112:0/3528208234 learned_addr learned my addr 172.21.15.112:0/3528208234 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f883b7fe700 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e0458a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f883bfff700 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0458520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 -- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 msgr2=0x55e8e0458a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e0458a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 -- 172.21.15.112:0/3528208234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 msgr2=0x55e8e0458520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0458520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 -- 172.21.15.112:0/3528208234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e0453c70 con 0x55e8e0438c30 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f8840ecb700 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04534d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8834007fd0 tx=0x7f8834007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812efeda64018911 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f881b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f881b7fe700 1 -- 172.21.15.112:0/3528208234 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834007410 con 0x55e8e0438c30 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f881b7fe700 1 -- 172.21.15.112:0/3528208234 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f8834004a90 con 0x55e8e0438c30 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.883+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882c033e50 con 0x55e8e0438c30 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f881b7fe700 1 -- 172.21.15.112:0/3528208234 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8834004c90 con 0x55e8e0438c30 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 10 client.14712 fetch_fsmap learned FSMap version 17 2024-05-10T10:36:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e8e04b9b40 con 0x55e8e0438c30 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f881b7fe700 1 -- 172.21.15.112:0/3528208234 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f882c033e50 con 0x55e8e0438c30 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f881b7fe700 1 -- 172.21.15.112:0/3528208234 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f8834028070 con 0x55e8e0438c30 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 10 client.14712 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 10 client.14712 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 20 client.14712 populate_metadata read hostname 'smithi112' 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e04be560 0x55e8e04bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 4 client.14712 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "status"}) -- 0x55e8e03941f0 con 0x55e8e04be560 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.884+0000 7f883bfff700 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e04be560 0x55e8e04bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f881b7fe700 10 client.14712 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f883bfff700 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e04be560 0x55e8e04bd270 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ed7724aea03025 server_cookie=9b30d9a05b7e74a0 in_seq=0 out_seq=0 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f881b7fe700 1 -- 172.21.15.112:0/3528208234 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e8e03941f0 con 0x55e8e04be560 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f881b7fe700 10 client.14712 handle_command_reply: tid=0 2024-05-10T10:36:25.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 client.14712 shutdown 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 msgr2=0x55e8e04534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04534d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8834007fd0 tx=0x7f8834007b30 comp rx=0 tx=0).stop 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 shutdown_connections 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e8e04be560 0x55e8e04bd270 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8e0438c30 0x55e8e04534d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e0434480 0x55e8e0458520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 --2- 172.21.15.112:0/3528208234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8e0434da0 0x55e8e0458a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 >> 172.21.15.112:0/3528208234 conn(0x55e8e03600b0 msgr2=0x55e8e0428e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f881b7fe700 0 client.14712 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 shutdown_connections 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 1 -- 172.21.15.112:0/3528208234 wait complete. 2024-05-10T10:36:25.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.885+0000 7f884d1c1b80 20 client.14712 trim_cache size 0 max 16384 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24469, 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 32.955257744999997, 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 17, 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 86, 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 61.323747281999999 2024-05-10T10:36:25.892 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:36:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88240b2900 msgr2=0x7f88240b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88240b2900 0x7f88240b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f882c000c00 tx=0x7f882c01a040 comp rx=0 tx=0).stop 2024-05-10T10:36:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c102890 msgr2=0x7f883c19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c102890 0x7f883c19aed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f883000c970 tx=0x7f883000cc80 comp rx=0 tx=0).stop 2024-05-10T10:36:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 shutdown_connections 2024-05-10T10:36:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88240b2900 0x7f88240b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883c107ba0 0x7f883c1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f883c101ed0 0x7f883c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 --2- 172.21.15.112:0/1696821157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f883c102890 0x7f883c19aed0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 >> 172.21.15.112:0/1696821157 conn(0x7f883c077610 msgr2=0x7f883c106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 shutdown_connections 2024-05-10T10:36:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:25.886+0000 7f884292e700 1 -- 172.21.15.112:0/1696821157 wait complete. 2024-05-10T10:36:26.592 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T10:36:26.592 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:26.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T10:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:27.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:27 smithi112 ceph-mon[30100]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:27.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:27 smithi148 ceph-mon[33568]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:27.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:27 smithi195 ceph-mon[33773]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:28.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 -- 172.21.15.112:0/1311820501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 msgr2=0x7f5114101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:28.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1311820501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f5114101e70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51040099c0 tx=0x7f5104009cd0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 -- 172.21.15.112:0/1311820501 shutdown_connections 2024-05-10T10:36:28.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1311820501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51141023b0 0x7f5114110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1311820501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5114109570 0x7f5114109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1311820501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f5114101e70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 -- 172.21.15.112:0/1311820501 >> 172.21.15.112:0/1311820501 conn(0x7f51140789a0 msgr2=0x7f51140ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:28.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 -- 172.21.15.112:0/1311820501 shutdown_connections 2024-05-10T10:36:28.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.520+0000 7f511bc8f700 1 -- 172.21.15.112:0/1311820501 wait complete. 2024-05-10T10:36:28.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 Processor -- start 2024-05-10T10:36:28.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 -- start start 2024-05-10T10:36:28.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f511419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:28.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51141023b0 0x7f511419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:28.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5114109570 0x7f51141998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5114113600 con 0x7f51141019f0 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5114113480 con 0x7f5114109570 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511bc8f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5114113780 con 0x7f51141023b0 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f511419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f511419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34106/0 (socket says 172.21.15.112:34106) 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 -- 172.21.15.112:0/1640895886 learned_addr learned my addr 172.21.15.112:0/1640895886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511922a700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51141023b0 0x7f511419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511a22c700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5114109570 0x7f51141998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 -- 172.21.15.112:0/1640895886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51141023b0 msgr2=0x7f511419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51141023b0 0x7f511419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 -- 172.21.15.112:0/1640895886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5114109570 msgr2=0x7f51141998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5114109570 0x7f51141998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 -- 172.21.15.112:0/1640895886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511419a040 con 0x7f51141019f0 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f511a22c700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5114109570 0x7f51141998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f5119a2b700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f511419f720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f511000c970 tx=0x7f511000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.521+0000 7f510affd700 1 -- 172.21.15.112:0/1640895886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110007aa0 con 0x7f51141019f0 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.522+0000 7f510affd700 1 -- 172.21.15.112:0/1640895886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5110004cc0 con 0x7f51141019f0 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.522+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511419a330 con 0x7f51141019f0 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.522+0000 7f510affd700 1 -- 172.21.15.112:0/1640895886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110005450 con 0x7f51141019f0 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.522+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51141a65f0 con 0x7f51141019f0 2024-05-10T10:36:28.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.522+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f511406a190 con 0x7f51141019f0 2024-05-10T10:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.525+0000 7f510affd700 1 -- 172.21.15.112:0/1640895886 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f5110007c40 con 0x7f51141019f0 2024-05-10T10:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.525+0000 7f510affd700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51000b2880 0x7f51000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.525+0000 7f511922a700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51000b2880 0x7f51000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.526+0000 7f510affd700 1 -- 172.21.15.112:0/1640895886 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f51100e2490 con 0x7f51141019f0 2024-05-10T10:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.526+0000 7f511922a700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51000b2880 0x7f51000b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5104000c00 tx=0x7f5104005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:28.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.529+0000 7f510affd700 1 -- 172.21.15.112:0/1640895886 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f51100aa090 con 0x7f51141019f0 2024-05-10T10:36:28.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.777+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f511402e510 con 0x7f51141019f0 2024-05-10T10:36:28.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.777+0000 7f510affd700 1 -- 172.21.15.112:0/1640895886 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f51100a9990 con 0x7f51141019f0 2024-05-10T10:36:28.779 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51000b2880 msgr2=0x7f51000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51000b2880 0x7f51000b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5104000c00 tx=0x7f5104005fb0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 msgr2=0x7f511419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f511419f720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f511000c970 tx=0x7f511000cc80 comp rx=0 tx=0).stop 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 shutdown_connections 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5114109570 0x7f51141998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51141019f0 0x7f511419f720 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51000b2880 0x7f51000b4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 --2- 172.21.15.112:0/1640895886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51141023b0 0x7f511419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 >> 172.21.15.112:0/1640895886 conn(0x7f51140789a0 msgr2=0x7f5114100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 shutdown_connections 2024-05-10T10:36:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:28.780+0000 7f511bc8f700 1 -- 172.21.15.112:0/1640895886 wait complete. 2024-05-10T10:36:29.391 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:29 smithi112 ceph-mon[30100]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:29.391 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:29 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1640895886' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:36:29.421 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T10:36:29.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:29.422 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:36:29.439 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42248sec preferred_lft 42248sec 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-05-10T10:36:29.440 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:36:29.441 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T10:36:29.441 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:36:29.441 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:36:29.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-05-10T10:36:29.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-05-10T10:36:29.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-05-10T10:36:29.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T10:36:29.442 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:36:29.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:36:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:29 smithi148 ceph-mon[33568]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:29 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1640895886' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T10:36:29.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:29 smithi195 ceph-mon[33773]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:29.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:29 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1640895886' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T10:36:29.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:36:29.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:29.676 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:36:29.701 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-05-10T10:36:29.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:29.702 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-05-10T10:36:29.773 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-05-10T10:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T10:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T10:36:29.774 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:36:29.774 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:36:29.774 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:36:29.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T10:36:29.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T10:36:29.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T10:36:29.775 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:36:29.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:36:30.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:36:30.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:30.070 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-05-10T10:36:30.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:30.086 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-05-10T10:36:30.142 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:36:30.142 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:36:30.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:36:30.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-10T10:36:30.142 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:36:30.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:36:30.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:36:30.261 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-10T10:36:30.261 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:36:30.261 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:36:30.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-10T10:36:30.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T10:36:30.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-10T10:36:30.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-10T10:36:30.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-10T10:36:30.262 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:36:30.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:36:30.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:36:30.385 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:36:30.385 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:36:30.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-05-10T10:36:30.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T10:36:30.385 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:36:30.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:36:30.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:36:30.463 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-10T10:36:30.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:36:30.463 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:30.479 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T10:36:30.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:36:30.479 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:30.534 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-05-10T10:36:30.599 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T10:36:30.655 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-05-10T10:36:30.655 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-05-10T10:36:30.655 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-05-10T10:36:30.656 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/20486a8a595afbe58cac729b72405afc9f7f9278c279f11b397cb223eac6519a/merged 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/93590821b74976f66af12e86aa7c88893f9fcfb6f6f9500aa1082194399e78a8/merged 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1bfd503ad3c3ca647e986ba54284ed75603499362df07adb930882232816fbfb/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/5a44526cfeab6ba58717c3c0494d13d2dff9698e9f9cd7173f8a68c4d50a57cd/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b4b77948b3d51909ef8195ab86f2ff80b1c6f5d50102df42ebb36b7d8a193811/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/d4dbc60c8323acf7497abc36913b1353445ac0d7f864b1484a3300219dde14bd/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/3332e641574b5261511cb2cf44d0383568df36bf4996a0ede1f607f923339ea6/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b8898964e652a6eebbec96c2fe5cd0c9a7a2faab9969fbcaae625fcfeeaa105d/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/e84f7eccad36855a7c9894cc735bd11599c694daeca9bbc56be55a850e08eb54/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1c026c59a648379c30b280101fc69842cd4eadb169d77e0f5e552b4e874c4e01/merged 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:30.659 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-05-10T10:36:30.711 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T10:36:30.711 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-10T10:36:30.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-05-10T10:36:30.777 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-05-10T10:36:30.833 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-05-10T10:36:30.834 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-05-10T10:36:30.837 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-05-10T10:36:30.837 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T10:36:30.837 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T10:36:30.837 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T10:36:30.837 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/20486a8a595afbe58cac729b72405afc9f7f9278c279f11b397cb223eac6519a/merged 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/93590821b74976f66af12e86aa7c88893f9fcfb6f6f9500aa1082194399e78a8/merged 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1bfd503ad3c3ca647e986ba54284ed75603499362df07adb930882232816fbfb/merged 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/5a44526cfeab6ba58717c3c0494d13d2dff9698e9f9cd7173f8a68c4d50a57cd/merged 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b4b77948b3d51909ef8195ab86f2ff80b1c6f5d50102df42ebb36b7d8a193811/merged 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/d4dbc60c8323acf7497abc36913b1353445ac0d7f864b1484a3300219dde14bd/merged 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/3332e641574b5261511cb2cf44d0383568df36bf4996a0ede1f607f923339ea6/merged 2024-05-10T10:36:30.838 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b8898964e652a6eebbec96c2fe5cd0c9a7a2faab9969fbcaae625fcfeeaa105d/merged 2024-05-10T10:36:30.839 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/e84f7eccad36855a7c9894cc735bd11599c694daeca9bbc56be55a850e08eb54/merged 2024-05-10T10:36:30.839 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1c026c59a648379c30b280101fc69842cd4eadb169d77e0f5e552b4e874c4e01/merged 2024-05-10T10:36:30.839 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-05-10T10:36:30.839 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:36:30.839 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:36:30.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:30.839 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-05-10T10:36:30.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:2024-05-10T10:36:30.867+0000 7fb030d4e480 -1 init, newargv = 0x564982272280 newargc=15 2024-05-10T10:36:30.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[73671]: starting ceph client 2024-05-10T10:36:30.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[73671]: starting fuse 2024-05-10T10:36:30.884 INFO:teuthology.orchestra.run.smithi112.stdout:114 2024-05-10T10:36:30.884 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-10T10:36:30.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-05-10T10:36:30.884 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-05-10T10:36:30.884 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-05-10T10:36:30.884 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-05-10T10:36:30.885 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.admin")) 2024-05-10T10:36:30.886 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-05-10T10:36:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.admin.73671.asok 2024-05-10T10:36:31.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:36:31.014 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73671.asok 2024-05-10T10:36:31.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:31.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73671.asok status 2024-05-10T10:36:31.150 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "d39fa08867836ac070c47cb66c782fa559fd480b", 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "admin", 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "73671", 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-05-10T10:36:31.151 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 14724, 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 14724 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 976705876 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-05-10T10:36:31.152 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 976705876 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.14724 192.168.0.1:0/976705876", 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/976705876", 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 17, 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 86, 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2024-05-10T10:36:31.153 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:36:31.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:31.159 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-05-10T10:36:31.159 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:31.175 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-05-10T10:36:31.175 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:31.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:31.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:31.237 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:31 smithi112 ceph-mon[30100]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:31.241 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:36:31.351 INFO:teuthology.orchestra.run.smithi112.stdout:# file: . 2024-05-10T10:36:31.352 INFO:teuthology.orchestra.run.smithi112.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-10T10:36:31.352 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:36:31.358 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:36:31.358 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:36:31.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T10:36:31.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:31 smithi148 ceph-mon[33568]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:31.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:31 smithi195 ceph-mon[33773]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:31.658 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:33.357 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:33 smithi112 ceph-mon[30100]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:33.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 -- 172.21.15.112:0/2709054215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102d50 msgr2=0x7f88d410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:33.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 --2- 172.21.15.112:0/2709054215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102d50 0x7f88d410f240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b0a0 tx=0x7f88cc00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 -- 172.21.15.112:0/2709054215 shutdown_connections 2024-05-10T10:36:33.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 --2- 172.21.15.112:0/2709054215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102d50 0x7f88d410f240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 --2- 172.21.15.112:0/2709054215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88d4108400 0x7f88d41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 --2- 172.21.15.112:0/2709054215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d4102390 0x7f88d4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 -- 172.21.15.112:0/2709054215 >> 172.21.15.112:0/2709054215 conn(0x7f88d4079310 msgr2=0x7f88d4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:33.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 -- 172.21.15.112:0/2709054215 shutdown_connections 2024-05-10T10:36:33.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.359+0000 7f88d980f700 1 -- 172.21.15.112:0/2709054215 wait complete. 2024-05-10T10:36:33.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.360+0000 7f88d980f700 1 Processor -- start 2024-05-10T10:36:33.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.360+0000 7f88d980f700 1 -- start start 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.360+0000 7f88d980f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102390 0x7f88d419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.360+0000 7f88d980f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88d4102d50 0x7f88d419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d980f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d4108400 0x7f88d4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d980f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41123f0 con 0x7f88d4102390 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d980f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4112270 con 0x7f88d4108400 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d980f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4112570 con 0x7f88d4102d50 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102390 0x7f88d419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102390 0x7f88d419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34124/0 (socket says 172.21.15.112:34124) 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 -- 172.21.15.112:0/1507687830 learned_addr learned my addr 172.21.15.112:0/1507687830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:33.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d27fc700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88d4102d50 0x7f88d419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d37fe700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d4108400 0x7f88d4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 -- 172.21.15.112:0/1507687830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88d4102d50 msgr2=0x7f88d419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88d4102d50 0x7f88d419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 -- 172.21.15.112:0/1507687830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d4108400 msgr2=0x7f88d4199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d4108400 0x7f88d4199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 -- 172.21.15.112:0/1507687830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d4199fc0 con 0x7f88d4102390 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d37fe700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d4108400 0x7f88d4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d27fc700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88d4102d50 0x7f88d419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d2ffd700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102390 0x7f88d419f6a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f88c800e9c0 tx=0x7f88c800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d880d700 1 -- 172.21.15.112:0/1507687830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c800cbd0 con 0x7f88d4102390 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d880d700 1 -- 172.21.15.112:0/1507687830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88c800cd70 con 0x7f88d4102390 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d419a250 con 0x7f88d4102390 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d880d700 1 -- 172.21.15.112:0/1507687830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8005610 con 0x7f88d4102390 2024-05-10T10:36:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.361+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d41a6570 con 0x7f88d4102390 2024-05-10T10:36:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.362+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d406a190 con 0x7f88d4102390 2024-05-10T10:36:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.364+0000 7f88d880d700 1 -- 172.21.15.112:0/1507687830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f88c8004750 con 0x7f88d4102390 2024-05-10T10:36:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.364+0000 7f88d880d700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88bc0b2880 0x7f88bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:33.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.365+0000 7f88d880d700 1 -- 172.21.15.112:0/1507687830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f88c80e2a30 con 0x7f88d4102390 2024-05-10T10:36:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.365+0000 7f88d27fc700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88bc0b2880 0x7f88bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.365+0000 7f88d27fc700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88bc0b2880 0x7f88bc0b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005950 tx=0x7f88c40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:33.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.368+0000 7f88d880d700 1 -- 172.21.15.112:0/1507687830 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f88c8054d70 con 0x7f88d4102390 2024-05-10T10:36:33.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:33 smithi148 ceph-mon[33568]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:33 smithi195 ceph-mon[33773]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:33.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.586+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88d41a6740 con 0x7f88d4102390 2024-05-10T10:36:33.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.586+0000 7f88d880d700 1 -- 172.21.15.112:0/1507687830 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+602 (secure 0 0 0) 0x7f88c80aa510 con 0x7f88d4102390 2024-05-10T10:36:33.587 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3420175550 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2112593785 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1230086877 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3235282798 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2711978475 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:33.588 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3592023612 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88bc0b2880 msgr2=0x7f88bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88bc0b2880 0x7f88bc0b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005950 tx=0x7f88c40058e0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102390 msgr2=0x7f88d419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102390 0x7f88d419f6a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f88c800e9c0 tx=0x7f88c800ed80 comp rx=0 tx=0).stop 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 shutdown_connections 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d4108400 0x7f88d4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d4102390 0x7f88d419f6a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88bc0b2880 0x7f88bc0b4d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 --2- 172.21.15.112:0/1507687830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88d4102d50 0x7f88d419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.588+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 >> 172.21.15.112:0/1507687830 conn(0x7f88d4079310 msgr2=0x7f88d41016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.589+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 shutdown_connections 2024-05-10T10:36:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:33.589+0000 7f88d980f700 1 -- 172.21.15.112:0/1507687830 wait complete. 2024-05-10T10:36:33.591 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-05-10T10:36:34.263 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-10T10:36:34.263 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-10T10:36:34.263 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-05-10T10:36:34.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:34 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1507687830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:36:34.290 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:36:34.290 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:36:34.308 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T10:36:34.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-10T10:36:34.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-05-10T10:36:34.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:34 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1507687830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-05-10T10:36:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-10T10:36:34.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-10T10:36:34.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-05-10T10:36:34.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:34 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1507687830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:36:34.589 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-10T10:36:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-05-10T10:36:35.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-10T10:36:35.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-10T10:36:35.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-05-10T10:36:35.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:35 smithi112 ceph-mon[30100]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T10:36:35.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-05-10T10:36:35.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:35 smithi148 ceph-mon[33568]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-05-10T10:36:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-10T10:36:35.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-10T10:36:35.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-05-10T10:36:35.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:35 smithi195 ceph-mon[33773]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T10:36:36.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:36 smithi112 ceph-mon[30100]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 -- 172.21.15.112:0/2111725552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc103ef0 msgr2=0x7f60fc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 --2- 172.21.15.112:0/2111725552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc103ef0 0x7f60fc1103d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60f400b0a0 tx=0x7f60f400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 -- 172.21.15.112:0/2111725552 shutdown_connections 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 --2- 172.21.15.112:0/2111725552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc103ef0 0x7f60fc1103d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 --2- 172.21.15.112:0/2111725552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60fc108460 0x7f60fc108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 --2- 172.21.15.112:0/2111725552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60fc103550 0x7f60fc1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 -- 172.21.15.112:0/2111725552 >> 172.21.15.112:0/2111725552 conn(0x7f60fc07a600 msgr2=0x7f60fc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 -- 172.21.15.112:0/2111725552 shutdown_connections 2024-05-10T10:36:36.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.320+0000 7f6103e59700 1 -- 172.21.15.112:0/2111725552 wait complete. 2024-05-10T10:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 Processor -- start 2024-05-10T10:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 -- start start 2024-05-10T10:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60fc103550 0x7f60fc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60fc103ef0 0x7f60fc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc108460 0x7f60fc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc113550 con 0x7f60fc108460 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1133d0 con 0x7f60fc103550 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6103e59700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1136d0 con 0x7f60fc103ef0 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f61023f6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc108460 0x7f60fc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f61023f6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc108460 0x7f60fc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49966/0 (socket says 172.21.15.112:49966) 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f61023f6700 1 -- 172.21.15.112:0/3854273273 learned_addr learned my addr 172.21.15.112:0/3854273273 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.321+0000 7f6101bf5700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60fc103550 0x7f60fc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f61013f4700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60fc103ef0 0x7f60fc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f61023f6700 1 -- 172.21.15.112:0/3854273273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60fc103ef0 msgr2=0x7f60fc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f61023f6700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60fc103ef0 0x7f60fc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f61023f6700 1 -- 172.21.15.112:0/3854273273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60fc103550 msgr2=0x7f60fc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f61023f6700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60fc103550 0x7f60fc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f61023f6700 1 -- 172.21.15.112:0/3854273273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc199ff0 con 0x7f60fc108460 2024-05-10T10:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f61023f6700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc108460 0x7f60fc199890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f60f400b830 tx=0x7f60f4004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f60f2ffd700 1 -- 172.21.15.112:0/3854273273 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4021070 con 0x7f60fc108460 2024-05-10T10:36:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc19a280 con 0x7f60fc108460 2024-05-10T10:36:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f60f2ffd700 1 -- 172.21.15.112:0/3854273273 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60f4007b80 con 0x7f60fc108460 2024-05-10T10:36:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a65a0 con 0x7f60fc108460 2024-05-10T10:36:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.322+0000 7f60f2ffd700 1 -- 172.21.15.112:0/3854273273 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f401de10 con 0x7f60fc108460 2024-05-10T10:36:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.323+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc06a190 con 0x7f60fc108460 2024-05-10T10:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.324+0000 7f60f2ffd700 1 -- 172.21.15.112:0/3854273273 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f60f4024070 con 0x7f60fc108460 2024-05-10T10:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.325+0000 7f60f2ffd700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60e80b2880 0x7f60e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.325+0000 7f6101bf5700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60e80b2880 0x7f60e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.325+0000 7f60f2ffd700 1 -- 172.21.15.112:0/3854273273 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f60f40e3870 con 0x7f60fc108460 2024-05-10T10:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.326+0000 7f6101bf5700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60e80b2880 0x7f60e80b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60fc19a9d0 tx=0x7f60f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.330+0000 7f60f2ffd700 1 -- 172.21.15.112:0/3854273273 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f60f402b020 con 0x7f60fc108460 2024-05-10T10:36:36.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:36 smithi148 ceph-mon[33568]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T10:36:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.547+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60fc02e510 con 0x7f60fc108460 2024-05-10T10:36:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.547+0000 7f60f2ffd700 1 -- 172.21.15.112:0/3854273273 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+602 (secure 0 0 0) 0x7f60f40ab2c0 con 0x7f60fc108460 2024-05-10T10:36:36.548 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3420175550 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:36.548 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:36.548 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2112593785 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:36.549 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:36.549 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1230086877 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:36.549 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:36.549 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3235282798 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:36.549 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2711978475 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:36.549 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:36:36.549 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3592023612 2024-05-11T10:28:33.178665+0000 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.549+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60e80b2880 msgr2=0x7f60e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.549+0000 7f6103e59700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60e80b2880 0x7f60e80b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60fc19a9d0 tx=0x7f60f800b410 comp rx=0 tx=0).stop 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.549+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc108460 msgr2=0x7f60fc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.549+0000 7f6103e59700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc108460 0x7f60fc199890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f60f400b830 tx=0x7f60f4004ce0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.549+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 shutdown_connections 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.550+0000 7f6103e59700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60fc108460 0x7f60fc199890 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.550+0000 7f6103e59700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60fc103550 0x7f60fc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.550+0000 7f6103e59700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60e80b2880 0x7f60e80b4d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.550+0000 7f6103e59700 1 --2- 172.21.15.112:0/3854273273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60fc103ef0 0x7f60fc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.550+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 >> 172.21.15.112:0/3854273273 conn(0x7f60fc07a600 msgr2=0x7f60fc101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.550+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 shutdown_connections 2024-05-10T10:36:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:36.550+0000 7f6103e59700 1 -- 172.21.15.112:0/3854273273 wait complete. 2024-05-10T10:36:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-05-10T10:36:36.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:36 smithi195 ceph-mon[33773]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T10:36:37.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:37 smithi112 ceph-mon[30100]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T10:36:37.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:37 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3854273273' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:36:37.276 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-05-10T10:36:37.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:37.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:36:37.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[73671]: fuse finished with error 0 and tester_r 0 2024-05-10T10:36:37.305 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:36:37.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:37 smithi148 ceph-mon[33568]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T10:36:37.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:37 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3854273273' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:36:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:37 smithi195 ceph-mon[33773]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T10:36:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:37 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3854273273' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:36:39.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:39 smithi148 ceph-mon[33568]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T10:36:39.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:39 smithi195 ceph-mon[33773]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T10:36:39.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:39 smithi112 ceph-mon[30100]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T10:36:41.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:41 smithi148 ceph-mon[33568]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-05-10T10:36:41.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:41 smithi195 ceph-mon[33773]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-05-10T10:36:41.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:41 smithi112 ceph-mon[30100]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-05-10T10:36:43.407 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:36:43.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:43.407 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-10T10:36:43.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:36:43.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:43 smithi112 ceph-mon[30100]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-05-10T10:36:43.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:43 smithi148 ceph-mon[33568]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-05-10T10:36:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:43 smithi195 ceph-mon[33773]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-05-10T10:36:43.751 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:44.468 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:36:44.485 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:36:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:36:45.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:45 smithi112 ceph-mon[30100]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s wr, 0 op/s; 403 B/s, 0 objects/s recovering 2024-05-10T10:36:45.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:45 smithi148 ceph-mon[33568]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s wr, 0 op/s; 403 B/s, 0 objects/s recovering 2024-05-10T10:36:45.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:45 smithi195 ceph-mon[33773]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s wr, 0 op/s; 403 B/s, 0 objects/s recovering 2024-05-10T10:36:45.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 -- 172.21.15.112:0/279378147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458102890 msgr2=0x7fb458110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/279378147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458102890 0x7fb458110390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb45000b0a0 tx=0x7fb45000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 -- 172.21.15.112:0/279378147 shutdown_connections 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/279378147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458102890 0x7fb458110390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/279378147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb458109680 0x7fb458109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/279378147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb458101ed0 0x7fb458102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 -- 172.21.15.112:0/279378147 >> 172.21.15.112:0/279378147 conn(0x7fb458077610 msgr2=0x7fb458077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 -- 172.21.15.112:0/279378147 shutdown_connections 2024-05-10T10:36:45.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.729+0000 7fb45f2a8700 1 -- 172.21.15.112:0/279378147 wait complete. 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 Processor -- start 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 -- start start 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458101ed0 0x7fb45819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb458102890 0x7fb45819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb458109680 0x7fb4581998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4581135a0 con 0x7fb458101ed0 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb458113420 con 0x7fb458109680 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45f2a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb458113720 con 0x7fb458102890 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458101ed0 0x7fb45819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458101ed0 0x7fb45819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55634/0 (socket says 172.21.15.112:55634) 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 -- 172.21.15.112:0/2576361579 learned_addr learned my addr 172.21.15.112:0/2576361579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45c843700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb458102890 0x7fb45819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d845700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb458109680 0x7fb4581998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 -- 172.21.15.112:0/2576361579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb458102890 msgr2=0x7fb45819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb458102890 0x7fb45819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 -- 172.21.15.112:0/2576361579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb458109680 msgr2=0x7fb4581998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb458109680 0x7fb4581998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.730+0000 7fb45d044700 1 -- 172.21.15.112:0/2576361579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45819a040 con 0x7fb458101ed0 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.731+0000 7fb45d044700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458101ed0 0x7fb45819f720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9c0 tx=0x7fb45400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.731+0000 7fb44e7fc700 1 -- 172.21.15.112:0/2576361579 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45400cbd0 con 0x7fb458101ed0 2024-05-10T10:36:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.731+0000 7fb44e7fc700 1 -- 172.21.15.112:0/2576361579 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb45400ee60 con 0x7fb458101ed0 2024-05-10T10:36:45.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.731+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45819a330 con 0x7fb458101ed0 2024-05-10T10:36:45.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.731+0000 7fb44e7fc700 1 -- 172.21.15.112:0/2576361579 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4540054b0 con 0x7fb458101ed0 2024-05-10T10:36:45.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.731+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4581a65f0 con 0x7fb458101ed0 2024-05-10T10:36:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.732+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45806a190 con 0x7fb458101ed0 2024-05-10T10:36:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.733+0000 7fb44e7fc700 1 -- 172.21.15.112:0/2576361579 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb454004750 con 0x7fb458101ed0 2024-05-10T10:36:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.734+0000 7fb44e7fc700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb4440b2900 0x7fb4440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.734+0000 7fb44e7fc700 1 -- 172.21.15.112:0/2576361579 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb4540e2a00 con 0x7fb458101ed0 2024-05-10T10:36:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.734+0000 7fb45c843700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb4440b2900 0x7fb4440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.734+0000 7fb45c843700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb4440b2900 0x7fb4440b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb458077cc0 tx=0x7fb448009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.737+0000 7fb44e7fc700 1 -- 172.21.15.112:0/2576361579 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb454054d70 con 0x7fb458101ed0 2024-05-10T10:36:45.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.967+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb45819ae40 con 0x7fb458101ed0 2024-05-10T10:36:45.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.969+0000 7fb44e7fc700 1 -- 172.21.15.112:0/2576361579 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24636 (secure 0 0 0) 0x7fb45819ae40 con 0x7fb458101ed0 2024-05-10T10:36:45.971 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:45.971 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":88,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:36:35.100943+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:36:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb4440b2900 msgr2=0x7fb4440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:45.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb4440b2900 0x7fb4440b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb458077cc0 tx=0x7fb448009190 comp rx=0 tx=0).stop 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458101ed0 msgr2=0x7fb45819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458101ed0 0x7fb45819f720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9c0 tx=0x7fb45400ecd0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 shutdown_connections 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb458109680 0x7fb4581998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb458101ed0 0x7fb45819f720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb4440b2900 0x7fb4440b4dc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 --2- 172.21.15.112:0/2576361579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb458102890 0x7fb45819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 >> 172.21.15.112:0/2576361579 conn(0x7fb458077610 msgr2=0x7fb458100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 shutdown_connections 2024-05-10T10:36:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:45.971+0000 7fb45f2a8700 1 -- 172.21.15.112:0/2576361579 wait complete. 2024-05-10T10:36:46.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:46 smithi148 ceph-mon[33568]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s wr, 0 op/s; 363 B/s, 0 objects/s recovering 2024-05-10T10:36:46.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:46 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2576361579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:46.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:46 smithi195 ceph-mon[33773]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s wr, 0 op/s; 363 B/s, 0 objects/s recovering 2024-05-10T10:36:46.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:46 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2576361579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:46 smithi112 ceph-mon[30100]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s wr, 0 op/s; 363 B/s, 0 objects/s recovering 2024-05-10T10:36:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:46 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2576361579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:47.133 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:36:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:48.680 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:48 smithi112 ceph-mon[30100]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 308 B/s, 0 objects/s recovering 2024-05-10T10:36:48.681 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:48.681 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:48.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:48 smithi148 ceph-mon[33568]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 308 B/s, 0 objects/s recovering 2024-05-10T10:36:48.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:48.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:48.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:48 smithi195 ceph-mon[33773]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 308 B/s, 0 objects/s recovering 2024-05-10T10:36:48.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:48.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:49.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.244+0000 7fb964147700 1 -- 172.21.15.112:0/1569257770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c102fe0 msgr2=0x7fb95c106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:49.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.244+0000 7fb964147700 1 --2- 172.21.15.112:0/1569257770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c102fe0 0x7fb95c106fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0099c0 tx=0x7fb94c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.244+0000 7fb964147700 1 -- 172.21.15.112:0/1569257770 shutdown_connections 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.244+0000 7fb964147700 1 --2- 172.21.15.112:0/1569257770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95c1076c0 0x7fb95c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.244+0000 7fb964147700 1 --2- 172.21.15.112:0/1569257770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c102630 0x7fb95c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.244+0000 7fb964147700 1 --2- 172.21.15.112:0/1569257770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c102fe0 0x7fb95c106fe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.244+0000 7fb964147700 1 -- 172.21.15.112:0/1569257770 >> 172.21.15.112:0/1569257770 conn(0x7fb95c079370 msgr2=0x7fb95c079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.245+0000 7fb964147700 1 -- 172.21.15.112:0/1569257770 shutdown_connections 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.245+0000 7fb964147700 1 -- 172.21.15.112:0/1569257770 wait complete. 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.245+0000 7fb964147700 1 Processor -- start 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.245+0000 7fb964147700 1 -- start start 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb964147700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95c102630 0x7fb95c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb964147700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c102fe0 0x7fb95c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:49.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb964147700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c1076c0 0x7fb95c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb964147700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1123e0 con 0x7fb95c1076c0 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb964147700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c112260 con 0x7fb95c102630 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb964147700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c112560 con 0x7fb95c102fe0 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c1076c0 0x7fb95c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c1076c0 0x7fb95c104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55650/0 (socket says 172.21.15.112:55650) 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 -- 172.21.15.112:0/3476071862 learned_addr learned my addr 172.21.15.112:0/3476071862 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9616e2700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c102fe0 0x7fb95c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb961ee3700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95c102630 0x7fb95c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 -- 172.21.15.112:0/3476071862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c102fe0 msgr2=0x7fb95c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:49.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c102fe0 0x7fb95c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 -- 172.21.15.112:0/3476071862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95c102630 msgr2=0x7fb95c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95c102630 0x7fb95c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 -- 172.21.15.112:0/3476071862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c1a8040 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.246+0000 7fb9626e4700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c1076c0 0x7fb95c104b00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb95400ada0 tx=0x7fb95400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.247+0000 7fb952ffd700 1 -- 172.21.15.112:0/3476071862 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954017020 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.247+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c1a8330 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.247+0000 7fb952ffd700 1 -- 172.21.15.112:0/3476071862 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb95400cd60 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.247+0000 7fb952ffd700 1 -- 172.21.15.112:0/3476071862 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954005280 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.247+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c1a87f0 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.248+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c06a190 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.249+0000 7fb952ffd700 1 -- 172.21.15.112:0/3476071862 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb95400c8d0 con 0x7fb95c1076c0 2024-05-10T10:36:49.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.250+0000 7fb952ffd700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9480b2880 0x7fb9480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.250+0000 7fb961ee3700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9480b2880 0x7fb9480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.250+0000 7fb952ffd700 1 -- 172.21.15.112:0/3476071862 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb9540e2220 con 0x7fb95c1076c0 2024-05-10T10:36:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.251+0000 7fb961ee3700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9480b2880 0x7fb9480b4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb958005950 tx=0x7fb95800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:49.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.254+0000 7fb952ffd700 1 -- 172.21.15.112:0/3476071862 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb954021020 con 0x7fb95c1076c0 2024-05-10T10:36:49.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.459+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb95c02e6d0 con 0x7fb95c1076c0 2024-05-10T10:36:49.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.460+0000 7fb952ffd700 1 -- 172.21.15.112:0/3476071862 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24636 (secure 0 0 0) 0x7fb9540a9c70 con 0x7fb95c1076c0 2024-05-10T10:36:49.462 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:36:49.462 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":88,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:36:35.100943+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:36:49.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.462+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9480b2880 msgr2=0x7fb9480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:49.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.462+0000 7fb964147700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9480b2880 0x7fb9480b4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb958005950 tx=0x7fb95800b410 comp rx=0 tx=0).stop 2024-05-10T10:36:49.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.462+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c1076c0 msgr2=0x7fb95c104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:49.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.462+0000 7fb964147700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c1076c0 0x7fb95c104b00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb95400ada0 tx=0x7fb95400c330 comp rx=0 tx=0).stop 2024-05-10T10:36:49.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 shutdown_connections 2024-05-10T10:36:49.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb95c1076c0 0x7fb95c104b00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95c102630 0x7fb95c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9480b2880 0x7fb9480b4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 --2- 172.21.15.112:0/3476071862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb95c102fe0 0x7fb95c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:49.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 >> 172.21.15.112:0/3476071862 conn(0x7fb95c079370 msgr2=0x7fb95c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:49.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 shutdown_connections 2024-05-10T10:36:49.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:49.463+0000 7fb964147700 1 -- 172.21.15.112:0/3476071862 wait complete. 2024-05-10T10:36:49.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:36:49.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:36:49.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:49.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:36:49.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:36:49.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:49.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:36:49.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:36:49.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:36:50.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:36:50.557 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:50.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:50 smithi112 ceph-mon[30100]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 308 B/s, 0 objects/s recovering 2024-05-10T10:36:50.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:50 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3476071862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:50.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:50 smithi148 ceph-mon[33568]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 308 B/s, 0 objects/s recovering 2024-05-10T10:36:50.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:50 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3476071862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:50.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:50 smithi195 ceph-mon[33773]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 308 B/s, 0 objects/s recovering 2024-05-10T10:36:50.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:50 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3476071862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:36:52.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 -- 172.21.15.112:0/3799739185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8101a90 msgr2=0x7f7bd8101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:52.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 --2- 172.21.15.112:0/3799739185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8101a90 0x7f7bd8101f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009990 tx=0x7f7bd4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:36:52.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 -- 172.21.15.112:0/3799739185 shutdown_connections 2024-05-10T10:36:52.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 --2- 172.21.15.112:0/3799739185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd8102450 0x7f7bd8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:52.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 --2- 172.21.15.112:0/3799739185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bd8109520 0x7f7bd8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:52.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 --2- 172.21.15.112:0/3799739185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8101a90 0x7f7bd8101f10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:52.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 -- 172.21.15.112:0/3799739185 >> 172.21.15.112:0/3799739185 conn(0x7f7bd8078a10 msgr2=0x7f7bd80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:52.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 -- 172.21.15.112:0/3799739185 shutdown_connections 2024-05-10T10:36:52.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.267+0000 7f7be01b9700 1 -- 172.21.15.112:0/3799739185 wait complete. 2024-05-10T10:36:52.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 Processor -- start 2024-05-10T10:36:52.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 -- start start 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bd8101a90 0x7f7bd819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd8102450 0x7f7bd819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8109520 0x7f7bd81998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8113500 con 0x7f7bd8109520 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8113380 con 0x7f7bd8102450 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7be01b9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8113680 con 0x7f7bd8101a90 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7bde756700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8109520 0x7f7bd81998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7bde756700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8109520 0x7f7bd81998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55660/0 (socket says 172.21.15.112:55660) 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.268+0000 7f7bdd754700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd8102450 0x7f7bd819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bde756700 1 -- 172.21.15.112:0/1776740843 learned_addr learned my addr 172.21.15.112:0/1776740843 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bddf55700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bd8101a90 0x7f7bd819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bde756700 1 -- 172.21.15.112:0/1776740843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bd8101a90 msgr2=0x7f7bd819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bde756700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bd8101a90 0x7f7bd819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bde756700 1 -- 172.21.15.112:0/1776740843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd8102450 msgr2=0x7f7bd819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bde756700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd8102450 0x7f7bd819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bde756700 1 -- 172.21.15.112:0/1776740843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd819a020 con 0x7f7bd8109520 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bde756700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8109520 0x7f7bd81998c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0007ee0 tx=0x7f7bd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bcaffd700 1 -- 172.21.15.112:0/1776740843 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0004bb0 con 0x7f7bd8109520 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bcaffd700 1 -- 172.21.15.112:0/1776740843 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd0004d50 con 0x7f7bd8109520 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd819a310 con 0x7f7bd8109520 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7bcaffd700 1 -- 172.21.15.112:0/1776740843 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00058b0 con 0x7f7bd8109520 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.269+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd81a65d0 con 0x7f7bd8109520 2024-05-10T10:36:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.270+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd810ba00 con 0x7f7bd8109520 2024-05-10T10:36:52.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.271+0000 7f7bcaffd700 1 -- 172.21.15.112:0/1776740843 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7bd001a020 con 0x7f7bd8109520 2024-05-10T10:36:52.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.272+0000 7f7bcaffd700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7bc40b2880 0x7f7bc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:52.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.272+0000 7f7bddf55700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7bc40b2880 0x7f7bc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:52.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.272+0000 7f7bcaffd700 1 -- 172.21.15.112:0/1776740843 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f7bd00e2270 con 0x7f7bd8109520 2024-05-10T10:36:52.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.273+0000 7f7bddf55700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7bc40b2880 0x7f7bc40b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bd819aa00 tx=0x7f7bcc00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:52.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.276+0000 7f7bcaffd700 1 -- 172.21.15.112:0/1776740843 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7bd0010070 con 0x7f7bd8109520 2024-05-10T10:36:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:52.524+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7bd819adf0 con 0x7f7bd8109520 2024-05-10T10:36:52.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:52 smithi112 ceph-mon[30100]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:52.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:52 smithi148 ceph-mon[33568]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:52.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:52 smithi195 ceph-mon[33773]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:53.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.345+0000 7f7bcaffd700 1 -- 172.21.15.112:0/1776740843 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f7bd00a9c30 con 0x7f7bd8109520 2024-05-10T10:36:53.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7bc40b2880 msgr2=0x7f7bc40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:53.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7bc40b2880 0x7f7bc40b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bd819aa00 tx=0x7f7bcc00b3c0 comp rx=0 tx=0).stop 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8109520 msgr2=0x7f7bd81998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8109520 0x7f7bd81998c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0007ee0 tx=0x7f7bd000d3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 shutdown_connections 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bd8109520 0x7f7bd81998c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7bc40b2880 0x7f7bc40b4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bd8101a90 0x7f7bd819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 --2- 172.21.15.112:0/1776740843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd8102450 0x7f7bd819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 >> 172.21.15.112:0/1776740843 conn(0x7f7bd8078a10 msgr2=0x7f7bd8100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 shutdown_connections 2024-05-10T10:36:53.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:53.348+0000 7f7be01b9700 1 -- 172.21.15.112:0/1776740843 wait complete. 2024-05-10T10:36:53.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:53 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1776740843' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T10:36:53.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:53 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1776740843' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T10:36:53.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:53 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1776740843' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T10:36:54.110 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T10:36:54.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-10T10:36:54.359 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:54.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:54 smithi112 ceph-mon[30100]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:54.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:54 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1776740843' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T10:36:54.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:54 smithi112 ceph-mon[30100]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:54 smithi148 ceph-mon[33568]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:54 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1776740843' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T10:36:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:54 smithi148 ceph-mon[33568]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:54.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:54 smithi195 ceph-mon[33773]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:54.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:54 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1776740843' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T10:36:54.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:54 smithi195 ceph-mon[33773]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:56.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 -- 172.21.15.112:0/2285236159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06c5d0 msgr2=0x7ff45c110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:56.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 --2- 172.21.15.112:0/2285236159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06c5d0 0x7ff45c110380 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff454009990 tx=0x7ff454009ca0 comp rx=0 tx=0).stop 2024-05-10T10:36:56.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 -- 172.21.15.112:0/2285236159 shutdown_connections 2024-05-10T10:36:56.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 --2- 172.21.15.112:0/2285236159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06c5d0 0x7ff45c110380 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 --2- 172.21.15.112:0/2285236159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff45c109540 0x7ff45c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 --2- 172.21.15.112:0/2285236159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff45c06bc10 0x7ff45c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 -- 172.21.15.112:0/2285236159 >> 172.21.15.112:0/2285236159 conn(0x7ff45c078410 msgr2=0x7ff45c078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 -- 172.21.15.112:0/2285236159 shutdown_connections 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.126+0000 7ff4646b8700 1 -- 172.21.15.112:0/2285236159 wait complete. 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 Processor -- start 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 -- start start 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06bc10 0x7ff45c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff45c06c5d0 0x7ff45c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff45c109540 0x7ff45c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1134f0 con 0x7ff45c06bc10 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c113370 con 0x7ff45c109540 2024-05-10T10:36:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff4646b8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c113670 con 0x7ff45c06c5d0 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06bc10 0x7ff45c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06bc10 0x7ff45c19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50098/0 (socket says 172.21.15.112:50098) 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 -- 172.21.15.112:0/1362493928 learned_addr learned my addr 172.21.15.112:0/1362493928 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 -- 172.21.15.112:0/1362493928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff45c06c5d0 msgr2=0x7ff45c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff45c06c5d0 0x7ff45c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462c55700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff45c109540 0x7ff45c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 -- 172.21.15.112:0/1362493928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff45c109540 msgr2=0x7ff45c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff45c109540 0x7ff45c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462454700 1 -- 172.21.15.112:0/1362493928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c199fd0 con 0x7ff45c06bc10 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.127+0000 7ff462c55700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff45c109540 0x7ff45c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.128+0000 7ff462454700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06bc10 0x7ff45c19f6b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff458007f10 tx=0x7ff45800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.128+0000 7ff4537fe700 1 -- 172.21.15.112:0/1362493928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458003a00 con 0x7ff45c06bc10 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.128+0000 7ff4537fe700 1 -- 172.21.15.112:0/1362493928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff458003ba0 con 0x7ff45c06bc10 2024-05-10T10:36:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.128+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c19a2c0 con 0x7ff45c06bc10 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.128+0000 7ff4537fe700 1 -- 172.21.15.112:0/1362493928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458015660 con 0x7ff45c06bc10 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.128+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a6580 con 0x7ff45c06bc10 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.129+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c104c00 con 0x7ff45c06bc10 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.130+0000 7ff4537fe700 1 -- 172.21.15.112:0/1362493928 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff458004000 con 0x7ff45c06bc10 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.131+0000 7ff4537fe700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff4480b2900 0x7ff4480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.131+0000 7ff461c53700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff4480b2900 0x7ff4480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.131+0000 7ff4537fe700 1 -- 172.21.15.112:0/1362493928 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7ff4580e2900 con 0x7ff45c06bc10 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.131+0000 7ff461c53700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff4480b2900 0x7ff4480b4dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005950 tx=0x7ff44c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:56.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.134+0000 7ff4537fe700 1 -- 172.21.15.112:0/1362493928 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff458023070 con 0x7ff45c06bc10 2024-05-10T10:36:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:56.384+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff45c06a190 con 0x7ff45c06bc10 2024-05-10T10:36:56.385 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:56 smithi112 ceph-mon[30100]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:56.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:56 smithi148 ceph-mon[33568]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:56.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:56 smithi195 ceph-mon[33773]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.357+0000 7ff4537fe700 1 -- 172.21.15.112:0/1362493928 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ff458010070 con 0x7ff45c06bc10 2024-05-10T10:36:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff4480b2900 msgr2=0x7ff4480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff4480b2900 0x7ff4480b4dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005950 tx=0x7ff44c0058e0 comp rx=0 tx=0).stop 2024-05-10T10:36:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06bc10 msgr2=0x7ff45c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06bc10 0x7ff45c19f6b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff458007f10 tx=0x7ff45800d3b0 comp rx=0 tx=0).stop 2024-05-10T10:36:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 shutdown_connections 2024-05-10T10:36:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff45c109540 0x7ff45c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:57.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff45c06bc10 0x7ff45c19f6b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:57.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff4480b2900 0x7ff4480b4dc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:57.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 --2- 172.21.15.112:0/1362493928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff45c06c5d0 0x7ff45c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:57.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 >> 172.21.15.112:0/1362493928 conn(0x7ff45c078410 msgr2=0x7ff45c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:57.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 shutdown_connections 2024-05-10T10:36:57.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:57.361+0000 7ff4646b8700 1 -- 172.21.15.112:0/1362493928 wait complete. 2024-05-10T10:36:57.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:57 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1362493928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T10:36:57.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:57 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1362493928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T10:36:57.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:57 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1362493928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T10:36:58.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-10T10:36:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:36:58.706 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:58 smithi112 ceph-mon[30100]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:58.706 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:58 smithi112 ceph-mon[30100]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T10:36:58.706 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:58 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1362493928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T10:36:58.706 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:58 smithi112 ceph-mon[30100]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T10:36:58.706 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:58 smithi112 ceph-mon[30100]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:58.706 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:58 smithi112 ceph-mon[30100]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-10T10:36:58.706 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:58 smithi112 ceph-mon[30100]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-10T10:36:58.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:58 smithi148 ceph-mon[33568]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:58.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:58 smithi148 ceph-mon[33568]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T10:36:58.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:58 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1362493928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T10:36:58.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:58 smithi148 ceph-mon[33568]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T10:36:58.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:58 smithi148 ceph-mon[33568]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:58.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:58 smithi148 ceph-mon[33568]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-10T10:36:58.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:58 smithi148 ceph-mon[33568]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-10T10:36:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:58 smithi195 ceph-mon[33773]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:36:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:58 smithi195 ceph-mon[33773]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T10:36:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:58 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1362493928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T10:36:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:58 smithi195 ceph-mon[33773]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T10:36:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:58 smithi195 ceph-mon[33773]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T10:36:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:58 smithi195 ceph-mon[33773]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-10T10:36:58.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:58 smithi195 ceph-mon[33773]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-10T10:36:59.649 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:59 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] up:active 2024-05-10T10:36:59.649 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:59 smithi112 ceph-mon[30100]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T10:36:59.649 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:59 smithi112 ceph-mon[30100]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-10T10:36:59.649 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:59 smithi112 ceph-mon[30100]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-10T10:36:59.649 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:36:59 smithi112 ceph-mon[30100]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-10T10:36:59.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:59 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] up:active 2024-05-10T10:36:59.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:59 smithi148 ceph-mon[33568]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T10:36:59.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:59 smithi148 ceph-mon[33568]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-10T10:36:59.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:59 smithi148 ceph-mon[33568]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-10T10:36:59.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:36:59 smithi148 ceph-mon[33568]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-10T10:36:59.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:59 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] up:active 2024-05-10T10:36:59.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:59 smithi195 ceph-mon[33773]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T10:36:59.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:59 smithi195 ceph-mon[33773]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-10T10:36:59.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:59 smithi195 ceph-mon[33773]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-10T10:36:59.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:36:59 smithi195 ceph-mon[33773]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-10T10:36:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 -- 172.21.15.112:0/1565565805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841076e0 msgr2=0x7fc98410f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 --2- 172.21.15.112:0/1565565805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841076e0 0x7fc98410f2b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00b0d0 tx=0x7fc97c00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 -- 172.21.15.112:0/1565565805 shutdown_connections 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 --2- 172.21.15.112:0/1565565805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841076e0 0x7fc98410f2b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 --2- 172.21.15.112:0/1565565805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc984102690 0x7fc984102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 --2- 172.21.15.112:0/1565565805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc984103040 0x7fc984107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 -- 172.21.15.112:0/1565565805 >> 172.21.15.112:0/1565565805 conn(0x7fc9840783f0 msgr2=0x7fc984078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 -- 172.21.15.112:0/1565565805 shutdown_connections 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 -- 172.21.15.112:0/1565565805 wait complete. 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 Processor -- start 2024-05-10T10:36:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.902+0000 7fc98a768700 1 -- start start 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc98a768700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102690 0x7fc98419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc98a768700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc984103040 0x7fc98419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc98a768700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9841076e0 0x7fc9841998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc983fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102690 0x7fc98419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc98a768700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc984112450 con 0x7fc984102690 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc983fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102690 0x7fc98419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50124/0 (socket says 172.21.15.112:50124) 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc98a768700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841122d0 con 0x7fc984103040 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc983fff700 1 -- 172.21.15.112:0/1463901758 learned_addr learned my addr 172.21.15.112:0/1463901758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:36:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841125d0 con 0x7fc9841076e0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc988d05700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9841076e0 0x7fc9841998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc9837fe700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc984103040 0x7fc98419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc988d05700 1 -- 172.21.15.112:0/1463901758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc984103040 msgr2=0x7fc98419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc988d05700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc984103040 0x7fc98419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc988d05700 1 -- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102690 msgr2=0x7fc98419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc988d05700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102690 0x7fc98419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc988d05700 1 -- 172.21.15.112:0/1463901758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98419a020 con 0x7fc9841076e0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.903+0000 7fc983fff700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102690 0x7fc98419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.904+0000 7fc988d05700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9841076e0 0x7fc9841998c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00b0d0 tx=0x7fc97c007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.904+0000 7fc9817fa700 1 -- 172.21.15.112:0/1463901758 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c01f070 con 0x7fc9841076e0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.904+0000 7fc9817fa700 1 -- 172.21.15.112:0/1463901758 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc97c004c50 con 0x7fc9841076e0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.904+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98419a2b0 con 0x7fc9841076e0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.904+0000 7fc9817fa700 1 -- 172.21.15.112:0/1463901758 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c01b430 con 0x7fc9841076e0 2024-05-10T10:36:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.904+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a65d0 con 0x7fc9841076e0 2024-05-10T10:36:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.906+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc984103ae0 con 0x7fc9841076e0 2024-05-10T10:36:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.906+0000 7fc9817fa700 1 -- 172.21.15.112:0/1463901758 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc97c022070 con 0x7fc9841076e0 2024-05-10T10:36:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.907+0000 7fc9817fa700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc96c0b2880 0x7fc96c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.907+0000 7fc983fff700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc96c0b2880 0x7fc96c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.907+0000 7fc9817fa700 1 -- 172.21.15.112:0/1463901758 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fc97c00f020 con 0x7fc9841076e0 2024-05-10T10:36:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.907+0000 7fc983fff700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc96c0b2880 0x7fc96c0b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc98419aa00 tx=0x7fc974009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:59.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:36:59.913+0000 7fc9817fa700 1 -- 172.21.15.112:0/1463901758 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc97c029020 con 0x7fc9841076e0 2024-05-10T10:37:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.162+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc98419adf0 con 0x7fc9841076e0 2024-05-10T10:37:00.316 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:36:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: 2024-05-10T10:36:59.869+0000 7f2b9a8c7700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-10T10:37:00.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.390+0000 7fc9817fa700 1 -- 172.21.15.112:0/1463901758 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7fc97c0aaf80 con 0x7fc9841076e0 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc96c0b2880 msgr2=0x7fc96c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc96c0b2880 0x7fc96c0b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc98419aa00 tx=0x7fc974009260 comp rx=0 tx=0).stop 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9841076e0 msgr2=0x7fc9841998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9841076e0 0x7fc9841998c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00b0d0 tx=0x7fc97c007a00 comp rx=0 tx=0).stop 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 shutdown_connections 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc96c0b2880 0x7fc96c0b4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9841076e0 0x7fc9841998c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102690 0x7fc98419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 --2- 172.21.15.112:0/1463901758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc984103040 0x7fc98419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 >> 172.21.15.112:0/1463901758 conn(0x7fc9840783f0 msgr2=0x7fc98406bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 shutdown_connections 2024-05-10T10:37:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:00.392+0000 7fc98a768700 1 -- 172.21.15.112:0/1463901758 wait complete. 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] up:active 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1463901758' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T10:37:00.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:00 smithi112 ceph-mon[30100]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T10:37:00.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-05-10T10:37:00.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] up:active 2024-05-10T10:37:00.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T10:37:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-10T10:37:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-05-10T10:37:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-10T10:37:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1463901758' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T10:37:00.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:00 smithi148 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] up:active 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1463901758' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T10:37:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:00 smithi195 ceph-mon[33773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T10:37:01.127 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-10T10:37:01.370 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:01.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T10:37:01.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] up:active 2024-05-10T10:37:01.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T10:37:01.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T10:37:01.653 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: Cluster is now healthy 2024-05-10T10:37:01.653 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-05-10T10:37:01.653 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-10T10:37:01.653 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:01 smithi112 ceph-mon[30100]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-10T10:37:01.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T10:37:01.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] up:active 2024-05-10T10:37:01.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T10:37:01.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T10:37:01.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: Cluster is now healthy 2024-05-10T10:37:01.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-05-10T10:37:01.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-10T10:37:01.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:01 smithi148 ceph-mon[33568]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] up:active 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: Cluster is now healthy 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-10T10:37:01.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:01 smithi195 ceph-mon[33773]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-10T10:37:02.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:02 smithi112 ceph-mon[30100]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-10T10:37:02.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:02 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] up:active 2024-05-10T10:37:02.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:02 smithi112 ceph-mon[30100]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-10T10:37:02.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:02 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:37:02.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:02 smithi148 ceph-mon[33568]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-10T10:37:02.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:02 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] up:active 2024-05-10T10:37:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:02 smithi148 ceph-mon[33568]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-10T10:37:02.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:02 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:37:02.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:02 smithi195 ceph-mon[33773]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-10T10:37:02.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:02 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] up:active 2024-05-10T10:37:02.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:02 smithi195 ceph-mon[33773]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-10T10:37:02.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:02 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:37:03.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 -- 172.21.15.112:0/737843849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c102d60 msgr2=0x7fa64c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 --2- 172.21.15.112:0/737843849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c102d60 0x7fa64c10f250 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa64400b0a0 tx=0x7fa64400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 -- 172.21.15.112:0/737843849 shutdown_connections 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 --2- 172.21.15.112:0/737843849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c102d60 0x7fa64c10f250 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 --2- 172.21.15.112:0/737843849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa64c108410 0x7fa64c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 --2- 172.21.15.112:0/737843849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa64c1023a0 0x7fa64c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 -- 172.21.15.112:0/737843849 >> 172.21.15.112:0/737843849 conn(0x7fa64c079350 msgr2=0x7fa64c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 -- 172.21.15.112:0/737843849 shutdown_connections 2024-05-10T10:37:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.096+0000 7fa6512bf700 1 -- 172.21.15.112:0/737843849 wait complete. 2024-05-10T10:37:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 Processor -- start 2024-05-10T10:37:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 -- start start 2024-05-10T10:37:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa64c1023a0 0x7fa64c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa64c102d60 0x7fa64c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c108410 0x7fa64c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c112460 con 0x7fa64c108410 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1122e0 con 0x7fa64c102d60 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa6512bf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1125e0 con 0x7fa64c1023a0 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa64b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c108410 0x7fa64c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa64b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c108410 0x7fa64c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50142/0 (socket says 172.21.15.112:50142) 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa64b7fe700 1 -- 172.21.15.112:0/2022091879 learned_addr learned my addr 172.21.15.112:0/2022091879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.097+0000 7fa64affd700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa64c1023a0 0x7fa64c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa643fff700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa64c102d60 0x7fa64c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa64b7fe700 1 -- 172.21.15.112:0/2022091879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa64c1023a0 msgr2=0x7fa64c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa64b7fe700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa64c1023a0 0x7fa64c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa64b7fe700 1 -- 172.21.15.112:0/2022091879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa64c102d60 msgr2=0x7fa64c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa64b7fe700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa64c102d60 0x7fa64c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa64b7fe700 1 -- 172.21.15.112:0/2022091879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c199fe0 con 0x7fa64c108410 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa643fff700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa64c102d60 0x7fa64c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa64b7fe700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c108410 0x7fa64c199800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa6440062a0 tx=0x7fa644004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa648ff9700 1 -- 172.21.15.112:0/2022091879 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644021070 con 0x7fa64c108410 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c19a270 con 0x7fa64c108410 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa648ff9700 1 -- 172.21.15.112:0/2022091879 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa644007d00 con 0x7fa64c108410 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1a64e0 con 0x7fa64c108410 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.098+0000 7fa648ff9700 1 -- 172.21.15.112:0/2022091879 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644005280 con 0x7fa64c108410 2024-05-10T10:37:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.099+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c103ac0 con 0x7fa64c108410 2024-05-10T10:37:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.100+0000 7fa648ff9700 1 -- 172.21.15.112:0/2022091879 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa644024070 con 0x7fa64c108410 2024-05-10T10:37:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.101+0000 7fa648ff9700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6380b2900 0x7fa6380b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.101+0000 7fa64affd700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6380b2900 0x7fa6380b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.101+0000 7fa648ff9700 1 -- 172.21.15.112:0/2022091879 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fa64400f020 con 0x7fa64c108410 2024-05-10T10:37:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.102+0000 7fa64affd700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6380b2900 0x7fa6380b4dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005ea0 tx=0x7fa63c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.105+0000 7fa648ff9700 1 -- 172.21.15.112:0/2022091879 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa644113d70 con 0x7fa64c108410 2024-05-10T10:37:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.362+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa64c1a6710 con 0x7fa64c108410 2024-05-10T10:37:03.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.403+0000 7fa648ff9700 1 -- 172.21.15.112:0/2022091879 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fa64402b020 con 0x7fa64c108410 2024-05-10T10:37:03.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6380b2900 msgr2=0x7fa6380b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6380b2900 0x7fa6380b4dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005ea0 tx=0x7fa63c005e30 comp rx=0 tx=0).stop 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c108410 msgr2=0x7fa64c199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c108410 0x7fa64c199800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa6440062a0 tx=0x7fa644004ed0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 shutdown_connections 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa64c108410 0x7fa64c199800 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa6380b2900 0x7fa6380b4dc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa64c1023a0 0x7fa64c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 --2- 172.21.15.112:0/2022091879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa64c102d60 0x7fa64c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 >> 172.21.15.112:0/2022091879 conn(0x7fa64c079350 msgr2=0x7fa64c101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 shutdown_connections 2024-05-10T10:37:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:03.406+0000 7fa6512bf700 1 -- 172.21.15.112:0/2022091879 wait complete. 2024-05-10T10:37:03.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:03 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2022091879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T10:37:03.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:03 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2022091879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T10:37:03.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:03 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2022091879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T10:37:04.155 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-10T10:37:04.155 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-10T10:37:04.402 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:04.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:04 smithi112 ceph-mon[30100]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-10T10:37:04.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:04 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2022091879' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T10:37:04.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:04 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:37:04.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:04 smithi148 ceph-mon[33568]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-10T10:37:04.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:04 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2022091879' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T10:37:04.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:04 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:37:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:04 smithi195 ceph-mon[33773]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-10T10:37:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:04 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2022091879' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T10:37:04.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:04 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.131+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4066761854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c102600 msgr2=0x7f0d0c108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.131+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4066761854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c102600 0x7f0d0c108160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d080099c0 tx=0x7f0d08009cd0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.131+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4066761854 shutdown_connections 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.131+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4066761854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c108890 0x7f0d0c110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.131+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4066761854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c101c50 0x7f0d0c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.131+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4066761854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c102600 0x7f0d0c108160 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.131+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4066761854 >> 172.21.15.112:0/4066761854 conn(0x7f0d0c0789a0 msgr2=0x7f0d0c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4066761854 shutdown_connections 2024-05-10T10:37:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4066761854 wait complete. 2024-05-10T10:37:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 Processor -- start 2024-05-10T10:37:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 -- start start 2024-05-10T10:37:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c101c50 0x7f0d0c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c102600 0x7f0d0c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c108890 0x7f0d0c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c113580 con 0x7f0d0c101c50 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c113400 con 0x7f0d0c102600 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d13ab4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c113700 con 0x7f0d0c108890 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.132+0000 7f0d11850700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c101c50 0x7f0d0c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c101c50 0x7f0d0c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39938/0 (socket says 172.21.15.112:39938) 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 -- 172.21.15.112:0/4064017288 learned_addr learned my addr 172.21.15.112:0/4064017288 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d1104f700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c102600 0x7f0d0c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d12051700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c108890 0x7f0d0c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 -- 172.21.15.112:0/4064017288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c108890 msgr2=0x7f0d0c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c108890 0x7f0d0c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 -- 172.21.15.112:0/4064017288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c102600 msgr2=0x7f0d0c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c102600 0x7f0d0c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 -- 172.21.15.112:0/4064017288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c19a060 con 0x7f0d0c101c50 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d12051700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c108890 0x7f0d0c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d11850700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c101c50 0x7f0d0c19f740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000b530 tx=0x7f0d0000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0cfeffd700 1 -- 172.21.15.112:0/4064017288 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00015020 con 0x7f0d0c101c50 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0cfeffd700 1 -- 172.21.15.112:0/4064017288 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d00011cc0 con 0x7f0d0c101c50 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0cfeffd700 1 -- 172.21.15.112:0/4064017288 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00010dc0 con 0x7f0d0c101c50 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c19a350 con 0x7f0d0c101c50 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.133+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1a6610 con 0x7f0d0c101c50 2024-05-10T10:37:06.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.134+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d0c10bad0 con 0x7f0d0c101c50 2024-05-10T10:37:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.136+0000 7f0cfeffd700 1 -- 172.21.15.112:0/4064017288 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0d00018020 con 0x7f0d0c101c50 2024-05-10T10:37:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.136+0000 7f0cfeffd700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0cf80b2880 0x7f0cf80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.136+0000 7f0d1104f700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0cf80b2880 0x7f0cf80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.136+0000 7f0cfeffd700 1 -- 172.21.15.112:0/4064017288 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0d000e2330 con 0x7f0d0c101c50 2024-05-10T10:37:06.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.137+0000 7f0d1104f700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0cf80b2880 0x7f0cf80b4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d08000c00 tx=0x7f0d08005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:06.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.140+0000 7f0cfeffd700 1 -- 172.21.15.112:0/4064017288 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0d0001f020 con 0x7f0d0c101c50 2024-05-10T10:37:06.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.371+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0d0c0638e0 con 0x7f0cf80b2880 2024-05-10T10:37:06.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.400+0000 7f0cfeffd700 1 -- 172.21.15.112:0/4064017288 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d0c0638e0 con 0x7f0cf80b2880 2024-05-10T10:37:06.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0cf80b2880 msgr2=0x7f0cf80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0cf80b2880 0x7f0cf80b4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d08000c00 tx=0x7f0d08005fb0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c101c50 msgr2=0x7f0d0c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c101c50 0x7f0d0c19f740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000b530 tx=0x7f0d0000b840 comp rx=0 tx=0).stop 2024-05-10T10:37:06.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 shutdown_connections 2024-05-10T10:37:06.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0cf80b2880 0x7f0cf80b4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d0c108890 0x7f0d0c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c101c50 0x7f0d0c19f740 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 --2- 172.21.15.112:0/4064017288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c102600 0x7f0d0c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 >> 172.21.15.112:0/4064017288 conn(0x7f0d0c0789a0 msgr2=0x7f0d0c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:06.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 shutdown_connections 2024-05-10T10:37:06.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:06.402+0000 7f0d13ab4700 1 -- 172.21.15.112:0/4064017288 wait complete. 2024-05-10T10:37:06.773 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:06 smithi112 ceph-mon[30100]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-10T10:37:06.773 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:37:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:37:06.376+0000 7f3dc0371700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T10:37:06.773 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:37:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:37:06.376+0000 7f3dc0371700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T10:37:06.773 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:37:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:37:06.376+0000 7f3dc0371700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T10:37:06.773 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:37:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:37:06.376+0000 7f3dc0371700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T10:37:06.774 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:37:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:37:06.376+0000 7f3dc0371700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T10:37:06.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:06 smithi148 ceph-mon[33568]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-10T10:37:06.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:06 smithi195 ceph-mon[33773]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-10T10:37:07.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-10T10:37:07.365 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:07.753 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:07 smithi112 ceph-mon[30100]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:07.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:07 smithi148 ceph-mon[33568]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:07.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:07 smithi195 ceph-mon[33773]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:08.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:08 smithi148 ceph-mon[33568]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-10T10:37:08.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:08 smithi148 ceph-mon[33568]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-10T10:37:08.797 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:08 smithi112 ceph-mon[30100]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-10T10:37:08.797 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:08 smithi112 ceph-mon[30100]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-10T10:37:08.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:08 smithi195 ceph-mon[33773]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-10T10:37:08.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:08 smithi195 ceph-mon[33773]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 -- 172.21.15.112:0/262975731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 msgr2=0x7f3998109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 --2- 172.21.15.112:0/262975731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998109a60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f398c009990 tx=0x7f398c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 -- 172.21.15.112:0/262975731 shutdown_connections 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 --2- 172.21.15.112:0/262975731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3998102890 0x7f3998110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 --2- 172.21.15.112:0/262975731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998109a60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 --2- 172.21.15.112:0/262975731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3998101ed0 0x7f3998102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 -- 172.21.15.112:0/262975731 >> 172.21.15.112:0/262975731 conn(0x7f3998077610 msgr2=0x7f3998077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 -- 172.21.15.112:0/262975731 shutdown_connections 2024-05-10T10:37:09.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 -- 172.21.15.112:0/262975731 wait complete. 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 Processor -- start 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.245+0000 7f399f12b700 1 -- start start 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399f12b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3998101ed0 0x7f399819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399f12b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3998102890 0x7f399819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399f12b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399f12b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3998113570 con 0x7f3998109680 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399f12b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39981133f0 con 0x7f3998102890 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399f12b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39981136f0 con 0x7f3998101ed0 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39972/0 (socket says 172.21.15.112:39972) 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 -- 172.21.15.112:0/421173236 learned_addr learned my addr 172.21.15.112:0/421173236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399cec7700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3998101ed0 0x7f399819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f3997fff700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3998102890 0x7f399819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 -- 172.21.15.112:0/421173236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3998101ed0 msgr2=0x7f399819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3998101ed0 0x7f399819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 -- 172.21.15.112:0/421173236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3998102890 msgr2=0x7f399819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3998102890 0x7f399819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 -- 172.21.15.112:0/421173236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399819a0b0 con 0x7f3998109680 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.246+0000 7f399d6c8700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998199950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3990007ee0 tx=0x7f399000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.247+0000 7f3995ffb700 1 -- 172.21.15.112:0/421173236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399000dbd0 con 0x7f3998109680 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.247+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399819a3a0 con 0x7f3998109680 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.247+0000 7f3995ffb700 1 -- 172.21.15.112:0/421173236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f399000dd70 con 0x7f3998109680 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.247+0000 7f3995ffb700 1 -- 172.21.15.112:0/421173236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900058d0 con 0x7f3998109680 2024-05-10T10:37:09.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.247+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39981a6660 con 0x7f3998109680 2024-05-10T10:37:09.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.248+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399810bac0 con 0x7f3998109680 2024-05-10T10:37:09.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.249+0000 7f3995ffb700 1 -- 172.21.15.112:0/421173236 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f399001a020 con 0x7f3998109680 2024-05-10T10:37:09.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.249+0000 7f3995ffb700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f39880b2910 0x7f39880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.250+0000 7f399cec7700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f39880b2910 0x7f39880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.250+0000 7f3995ffb700 1 -- 172.21.15.112:0/421173236 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f39900e1ee0 con 0x7f3998109680 2024-05-10T10:37:09.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.250+0000 7f399cec7700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f39880b2910 0x7f39880b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f398c00b5c0 tx=0x7f398c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.254+0000 7f3995ffb700 1 -- 172.21.15.112:0/421173236 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3990010070 con 0x7f3998109680 2024-05-10T10:37:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.489+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f39980638e0 con 0x7f39880b2910 2024-05-10T10:37:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.517+0000 7f3995ffb700 1 -- 172.21.15.112:0/421173236 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39980638e0 con 0x7f39880b2910 2024-05-10T10:37:09.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f39880b2910 msgr2=0x7f39880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f39880b2910 0x7f39880b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f398c00b5c0 tx=0x7f398c01a040 comp rx=0 tx=0).stop 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 msgr2=0x7f3998199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998199950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3990007ee0 tx=0x7f399000d3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 shutdown_connections 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998109680 0x7f3998199950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f39880b2910 0x7f39880b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3998101ed0 0x7f399819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 --2- 172.21.15.112:0/421173236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3998102890 0x7f399819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 >> 172.21.15.112:0/421173236 conn(0x7f3998077610 msgr2=0x7f3998100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 shutdown_connections 2024-05-10T10:37:09.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:09.519+0000 7f399f12b700 1 -- 172.21.15.112:0/421173236 wait complete. 2024-05-10T10:37:10.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-10T10:37:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:10 smithi112 ceph-mon[30100]: pgmap v336: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-05-10T10:37:10.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:10 smithi112 ceph-mon[30100]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:10.598 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:10.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:10 smithi148 ceph-mon[33568]: pgmap v336: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-05-10T10:37:10.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:10 smithi148 ceph-mon[33568]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:10.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:10 smithi195 ceph-mon[33773]: pgmap v336: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-05-10T10:37:10.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:10 smithi195 ceph-mon[33773]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 -- 172.21.15.112:0/801435209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 msgr2=0x7f41b81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 --2- 172.21.15.112:0/801435209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b81103d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41b00099c0 tx=0x7f41b0009cd0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 -- 172.21.15.112:0/801435209 shutdown_connections 2024-05-10T10:37:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 --2- 172.21.15.112:0/801435209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b81103d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 --2- 172.21.15.112:0/801435209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8103750 0x7f41b8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 --2- 172.21.15.112:0/801435209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 0x7f41b8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 -- 172.21.15.112:0/801435209 >> 172.21.15.112:0/801435209 conn(0x7f41b807a600 msgr2=0x7f41b807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 -- 172.21.15.112:0/801435209 shutdown_connections 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.392+0000 7f41be049700 1 -- 172.21.15.112:0/801435209 wait complete. 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 Processor -- start 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 -- start start 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8103750 0x7f41b819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 0x7f41b819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81134f0 con 0x7f41b8108850 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113370 con 0x7f41b8104100 2024-05-10T10:37:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41be049700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113670 con 0x7f41b8103750 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 0x7f41b819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41b77fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8103750 0x7f41b819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 0x7f41b819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:39990/0 (socket says 172.21.15.112:39990) 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41b6ffd700 1 -- 172.21.15.112:0/3576575490 learned_addr learned my addr 172.21.15.112:0/3576575490 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.393+0000 7f41b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b8199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40004/0 (socket says 172.21.15.112:40004) 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b77fe700 1 -- 172.21.15.112:0/3576575490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 msgr2=0x7f41b819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b77fe700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 0x7f41b819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b77fe700 1 -- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 msgr2=0x7f41b8199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b77fe700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b8199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b77fe700 1 -- 172.21.15.112:0/3576575490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b819a050 con 0x7f41b8103750 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b7fff700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b6ffd700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 0x7f41b819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b77fe700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8103750 0x7f41b819f6a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41a8007f10 tx=0x7f41a800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b4ff9700 1 -- 172.21.15.112:0/3576575490 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8004db0 con 0x7f41b8103750 2024-05-10T10:37:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b819a340 con 0x7f41b8103750 2024-05-10T10:37:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b4ff9700 1 -- 172.21.15.112:0/3576575490 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41a8005020 con 0x7f41b8103750 2024-05-10T10:37:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41b4ff9700 1 -- 172.21.15.112:0/3576575490 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8015660 con 0x7f41b8103750 2024-05-10T10:37:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.394+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a6620 con 0x7f41b8103750 2024-05-10T10:37:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.395+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b810ba20 con 0x7f41b8103750 2024-05-10T10:37:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.397+0000 7f41b4ff9700 1 -- 172.21.15.112:0/3576575490 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f41a800dac0 con 0x7f41b8103750 2024-05-10T10:37:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.397+0000 7f41b4ff9700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41a40b2910 0x7f41a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.397+0000 7f41b6ffd700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41a40b2910 0x7f41a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.397+0000 7f41b4ff9700 1 -- 172.21.15.112:0/3576575490 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f41a80e2900 con 0x7f41b8103750 2024-05-10T10:37:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.398+0000 7f41b6ffd700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41a40b2910 0x7f41a40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41b819af30 tx=0x7f41a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.401+0000 7f41b4ff9700 1 -- 172.21.15.112:0/3576575490 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f41a8010070 con 0x7f41b8103750 2024-05-10T10:37:12.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:12 smithi112 ceph-mon[30100]: pgmap v337: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 22 op/s 2024-05-10T10:37:12.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.640+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f41b819aca0 con 0x7f41a40b2910 2024-05-10T10:37:12.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.646+0000 7f41b4ff9700 1 -- 172.21.15.112:0/3576575490 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f41b819aca0 con 0x7f41a40b2910 2024-05-10T10:37:12.647 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2 2024-05-10T10:37:12.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41a40b2910 msgr2=0x7f41a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41a40b2910 0x7f41a40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41b819af30 tx=0x7f41a0005f90 comp rx=0 tx=0).stop 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8103750 msgr2=0x7f41b819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8103750 0x7f41b819f6a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41a8007f10 tx=0x7f41a800d3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 shutdown_connections 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8108850 0x7f41b8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41a40b2910 0x7f41a40b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8103750 0x7f41b819f6a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 --2- 172.21.15.112:0/3576575490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b8104100 0x7f41b819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 >> 172.21.15.112:0/3576575490 conn(0x7f41b807a600 msgr2=0x7f41b8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:12.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 shutdown_connections 2024-05-10T10:37:12.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:12.648+0000 7f41be049700 1 -- 172.21.15.112:0/3576575490 wait complete. 2024-05-10T10:37:12.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:12 smithi148 ceph-mon[33568]: pgmap v337: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 22 op/s 2024-05-10T10:37:12.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:12 smithi195 ceph-mon[33773]: pgmap v337: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 22 op/s 2024-05-10T10:37:13.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-10T10:37:13.717 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:13 smithi195 ceph-mon[33773]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:13.720 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:13.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:13 smithi148 ceph-mon[33568]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:13.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:13 smithi112 ceph-mon[30100]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:14.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:14 smithi148 ceph-mon[33568]: pgmap v338: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-05-10T10:37:14.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:14 smithi195 ceph-mon[33773]: pgmap v338: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-05-10T10:37:14.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:14 smithi112 ceph-mon[30100]: pgmap v338: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-05-10T10:37:15.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 -- 172.21.15.112:0/3075611575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7388109510 msgr2=0x7f73881098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3075611575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7388109510 0x7f73881098f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7378009990 tx=0x7f7378009ca0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 -- 172.21.15.112:0/3075611575 shutdown_connections 2024-05-10T10:37:15.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3075611575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f738806c590 0x7f7388110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3075611575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7388109510 0x7f73881098f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3075611575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738806bbd0 0x7f738806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 -- 172.21.15.112:0/3075611575 >> 172.21.15.112:0/3075611575 conn(0x7f7388079350 msgr2=0x7f7388079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 -- 172.21.15.112:0/3075611575 shutdown_connections 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.458+0000 7f738d3fd700 1 -- 172.21.15.112:0/3075611575 wait complete. 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 Processor -- start 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 -- start start 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738806bbd0 0x7f73881024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738806c590 0x7f73881029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7388109510 0x7f7388102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73881134d0 con 0x7f738806c590 2024-05-10T10:37:15.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7388113350 con 0x7f738806bbd0 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f738d3fd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7388113650 con 0x7f7388109510 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f73867fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738806c590 0x7f73881029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f73867fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738806c590 0x7f73881029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40016/0 (socket says 172.21.15.112:40016) 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f73867fc700 1 -- 172.21.15.112:0/3785992332 learned_addr learned my addr 172.21.15.112:0/3785992332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f7386ffd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738806bbd0 0x7f73881024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f73877fe700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7388109510 0x7f7388102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f73867fc700 1 -- 172.21.15.112:0/3785992332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7388109510 msgr2=0x7f7388102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.459+0000 7f73867fc700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7388109510 0x7f7388102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f73867fc700 1 -- 172.21.15.112:0/3785992332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738806bbd0 msgr2=0x7f73881024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f73867fc700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738806bbd0 0x7f73881024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f73867fc700 1 -- 172.21.15.112:0/3785992332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73881a7f50 con 0x7f738806c590 2024-05-10T10:37:15.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f73867fc700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738806c590 0x7f73881029e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f737c00c9b0 tx=0x7f737c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:15.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f736ffff700 1 -- 172.21.15.112:0/3785992332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c007b00 con 0x7f738806c590 2024-05-10T10:37:15.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73881a8240 con 0x7f738806c590 2024-05-10T10:37:15.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f736ffff700 1 -- 172.21.15.112:0/3785992332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f737c007ca0 con 0x7f738806c590 2024-05-10T10:37:15.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f736ffff700 1 -- 172.21.15.112:0/3785992332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c018660 con 0x7f738806c590 2024-05-10T10:37:15.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.460+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73881a8780 con 0x7f738806c590 2024-05-10T10:37:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.461+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738810b9f0 con 0x7f738806c590 2024-05-10T10:37:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.462+0000 7f736ffff700 1 -- 172.21.15.112:0/3785992332 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f737c00f450 con 0x7f738806c590 2024-05-10T10:37:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.463+0000 7f736ffff700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73700b2910 0x7f73700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.463+0000 7f7386ffd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73700b2910 0x7f73700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.463+0000 7f736ffff700 1 -- 172.21.15.112:0/3785992332 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f737c0e26e0 con 0x7f738806c590 2024-05-10T10:37:15.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.463+0000 7f7386ffd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73700b2910 0x7f73700b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f7378005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:15.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.468+0000 7f736ffff700 1 -- 172.21.15.112:0/3785992332 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f737c014030 con 0x7f738806c590 2024-05-10T10:37:15.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.702+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f73880638e0 con 0x7f73700b2910 2024-05-10T10:37:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.730+0000 7f736ffff700 1 -- 172.21.15.112:0/3785992332 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73880638e0 con 0x7f73700b2910 2024-05-10T10:37:15.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73700b2910 msgr2=0x7f73700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73700b2910 0x7f73700b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f7378005960 comp rx=0 tx=0).stop 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738806c590 msgr2=0x7f73881029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738806c590 0x7f73881029e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f737c00c9b0 tx=0x7f737c00ccc0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 shutdown_connections 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f73700b2910 0x7f73700b4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7388109510 0x7f7388102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f738806bbd0 0x7f73881024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 --2- 172.21.15.112:0/3785992332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738806c590 0x7f73881029e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 >> 172.21.15.112:0/3785992332 conn(0x7f7388079350 msgr2=0x7f7388101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 shutdown_connections 2024-05-10T10:37:15.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:15.733+0000 7f738d3fd700 1 -- 172.21.15.112:0/3785992332 wait complete. 2024-05-10T10:37:16.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-10T10:37:16.678 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:16.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:16 smithi148 ceph-mon[33568]: pgmap v339: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 16 op/s 2024-05-10T10:37:16.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:16 smithi148 ceph-mon[33568]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:16.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:16 smithi112 ceph-mon[30100]: pgmap v339: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 16 op/s 2024-05-10T10:37:16.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:16 smithi112 ceph-mon[30100]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:16.815 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:16 smithi195 ceph-mon[33773]: pgmap v339: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 16 op/s 2024-05-10T10:37:16.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:16 smithi195 ceph-mon[33773]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:18.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.407+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1184477406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 msgr2=0x7f5f38107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.407+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1184477406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f38107020 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f280099c0 tx=0x7f5f28009cd0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.407+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1184477406 shutdown_connections 2024-05-10T10:37:18.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.407+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1184477406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f38107700 0x7f5f3810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.407+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1184477406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f38102620 0x7f5f38102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.407+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1184477406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f38107020 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.407+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1184477406 >> 172.21.15.112:0/1184477406 conn(0x7f5f38079440 msgr2=0x7f5f38079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:18.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.408+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1184477406 shutdown_connections 2024-05-10T10:37:18.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.408+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1184477406 wait complete. 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.408+0000 7f5f3fa36700 1 Processor -- start 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3fa36700 1 -- start start 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3fa36700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f38102620 0x7f5f3819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3fa36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f3819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3fa36700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f38107700 0x7f5f381988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3fa36700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381124a0 con 0x7f5f38102fd0 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3fa36700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38112320 con 0x7f5f38107700 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3fa36700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38112620 con 0x7f5f38102620 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f3819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f3819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54710/0 (socket says 172.21.15.112:54710) 2024-05-10T10:37:18.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 -- 172.21.15.112:0/1415808810 learned_addr learned my addr 172.21.15.112:0/1415808810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3d7d2700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f38102620 0x7f5f3819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3dfd3700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f38107700 0x7f5f381988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 -- 172.21.15.112:0/1415808810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f38102620 msgr2=0x7f5f3819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f38102620 0x7f5f3819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 -- 172.21.15.112:0/1415808810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f38107700 msgr2=0x7f5f381988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f38107700 0x7f5f381988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 -- 172.21.15.112:0/1415808810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f38199030 con 0x7f5f38102fd0 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3dfd3700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f38107700 0x7f5f381988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:18.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.409+0000 7f5f3cfd1700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f3819ec50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800b580 tx=0x7f5f2800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.410+0000 7f5f2e7fc700 1 -- 172.21.15.112:0/1415808810 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28027070 con 0x7f5f38102fd0 2024-05-10T10:37:18.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.410+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f381992c0 con 0x7f5f38102fd0 2024-05-10T10:37:18.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.410+0000 7f5f2e7fc700 1 -- 172.21.15.112:0/1415808810 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f280056d0 con 0x7f5f38102fd0 2024-05-10T10:37:18.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.410+0000 7f5f2e7fc700 1 -- 172.21.15.112:0/1415808810 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f280173f0 con 0x7f5f38102fd0 2024-05-10T10:37:18.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.410+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3806bbd0 con 0x7f5f38102fd0 2024-05-10T10:37:18.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.411+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3806a190 con 0x7f5f38102fd0 2024-05-10T10:37:18.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.412+0000 7f5f2e7fc700 1 -- 172.21.15.112:0/1415808810 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5f2802a070 con 0x7f5f38102fd0 2024-05-10T10:37:18.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.413+0000 7f5f2e7fc700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f240b2910 0x7f5f240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.413+0000 7f5f3d7d2700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f240b2910 0x7f5f240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.413+0000 7f5f2e7fc700 1 -- 172.21.15.112:0/1415808810 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f5f280e3720 con 0x7f5f38102fd0 2024-05-10T10:37:18.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.414+0000 7f5f3d7d2700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f240b2910 0x7f5f240b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005950 tx=0x7f5f3400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.417+0000 7f5f2e7fc700 1 -- 172.21.15.112:0/1415808810 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5f28013070 con 0x7f5f38102fd0 2024-05-10T10:37:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.656+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5f3806bef0 con 0x7f5f240b2910 2024-05-10T10:37:18.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.662+0000 7f5f2e7fc700 1 -- 172.21.15.112:0/1415808810 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5f3806bef0 con 0x7f5f240b2910 2024-05-10T10:37:18.664 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_1/2e57a5b2-d01f-4129-9da8-33ee352bf2ef 2024-05-10T10:37:18.664 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:18 smithi112 ceph-mon[30100]: pgmap v340: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f240b2910 msgr2=0x7f5f240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f240b2910 0x7f5f240b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005950 tx=0x7f5f3400a300 comp rx=0 tx=0).stop 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 msgr2=0x7f5f3819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f3819ec50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800b580 tx=0x7f5f2800f690 comp rx=0 tx=0).stop 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 shutdown_connections 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f38107700 0x7f5f381988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f240b2910 0x7f5f240b4dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f38102620 0x7f5f3819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 --2- 172.21.15.112:0/1415808810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f38102fd0 0x7f5f3819ec50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 >> 172.21.15.112:0/1415808810 conn(0x7f5f38079440 msgr2=0x7f5f38100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:18.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 shutdown_connections 2024-05-10T10:37:18.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:18.664+0000 7f5f3fa36700 1 -- 172.21.15.112:0/1415808810 wait complete. 2024-05-10T10:37:18.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:18 smithi148 ceph-mon[33568]: pgmap v340: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-10T10:37:18.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:18 smithi195 ceph-mon[33773]: pgmap v340: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-10T10:37:19.521 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:37:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:19.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:19 smithi148 ceph-mon[33568]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:19.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:19 smithi195 ceph-mon[33773]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:19.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:19 smithi112 ceph-mon[30100]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:37:20.708 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:20 smithi112 ceph-mon[30100]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-10T10:37:20.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:20 smithi148 ceph-mon[33568]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-10T10:37:20.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:20 smithi195 ceph-mon[33773]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-10T10:37:21.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 -- 172.21.15.112:0/3225917758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 msgr2=0x7f0b4c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 --2- 172.21.15.112:0/3225917758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c102a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0099c0 tx=0x7f0b3c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 -- 172.21.15.112:0/3225917758 shutdown_connections 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 --2- 172.21.15.112:0/3225917758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c1076c0 0x7f0b4c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 --2- 172.21.15.112:0/3225917758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c102a10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 --2- 172.21.15.112:0/3225917758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c102fe0 0x7f0b4c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 -- 172.21.15.112:0/3225917758 >> 172.21.15.112:0/3225917758 conn(0x7f0b4c079370 msgr2=0x7f0b4c079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 -- 172.21.15.112:0/3225917758 shutdown_connections 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.679+0000 7f0b5431c700 1 -- 172.21.15.112:0/3225917758 wait complete. 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 Processor -- start 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 -- start start 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c102fe0 0x7f0b4c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c1076c0 0x7f0b4c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c112400 con 0x7f0b4c102630 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c112280 con 0x7f0b4c1076c0 2024-05-10T10:37:21.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b5431c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c112580 con 0x7f0b4c102fe0 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.680+0000 7f0b520b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54734/0 (socket says 172.21.15.112:54734) 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 -- 172.21.15.112:0/4001678273 learned_addr learned my addr 172.21.15.112:0/4001678273 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b528b9700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c1076c0 0x7f0b4c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 -- 172.21.15.112:0/4001678273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c102fe0 msgr2=0x7f0b4c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c102fe0 0x7f0b4c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 -- 172.21.15.112:0/4001678273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c1076c0 msgr2=0x7f0b4c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c1076c0 0x7f0b4c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 -- 172.21.15.112:0/4001678273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c19a090 con 0x7f0b4c102630 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b528b9700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c1076c0 0x7f0b4c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b520b8700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c19f720 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c006010 tx=0x7f0b3c004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b437fe700 1 -- 172.21.15.112:0/4001678273 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c026070 con 0x7f0b4c102630 2024-05-10T10:37:21.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4c19a320 con 0x7f0b4c102630 2024-05-10T10:37:21.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b437fe700 1 -- 172.21.15.112:0/4001678273 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b3c00f460 con 0x7f0b4c102630 2024-05-10T10:37:21.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b437fe700 1 -- 172.21.15.112:0/4001678273 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c009e60 con 0x7f0b4c102630 2024-05-10T10:37:21.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.681+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4c103fb0 con 0x7f0b4c102630 2024-05-10T10:37:21.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.682+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4c103ac0 con 0x7f0b4c102630 2024-05-10T10:37:21.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.684+0000 7f0b437fe700 1 -- 172.21.15.112:0/4001678273 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0b3c029070 con 0x7f0b4c102630 2024-05-10T10:37:21.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.684+0000 7f0b437fe700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0b380b6d20 0x7f0b380b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.684+0000 7f0b518b7700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0b380b6d20 0x7f0b380b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.685+0000 7f0b437fe700 1 -- 172.21.15.112:0/4001678273 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0b3c0e32a0 con 0x7f0b4c102630 2024-05-10T10:37:21.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.685+0000 7f0b518b7700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0b380b6d20 0x7f0b380b91e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005fd0 tx=0x7f0b48005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.688+0000 7f0b437fe700 1 -- 172.21.15.112:0/4001678273 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0b3c030080 con 0x7f0b4c102630 2024-05-10T10:37:21.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.939+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b4c19ae10 con 0x7f0b4c102630 2024-05-10T10:37:21.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.940+0000 7f0b437fe700 1 -- 172.21.15.112:0/4001678273 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7f0b3c013070 con 0x7f0b4c102630 2024-05-10T10:37:21.941 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:37:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:37:03.364184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24428,24445,24446,24451,24469]},"id":1}]} 2024-05-10T10:37:21.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0b380b6d20 msgr2=0x7f0b380b91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0b380b6d20 0x7f0b380b91e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005fd0 tx=0x7f0b48005d00 comp rx=0 tx=0).stop 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 msgr2=0x7f0b4c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c19f720 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c006010 tx=0x7f0b3c004690 comp rx=0 tx=0).stop 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 shutdown_connections 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c1076c0 0x7f0b4c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b4c102630 0x7f0b4c19f720 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0b380b6d20 0x7f0b380b91e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 --2- 172.21.15.112:0/4001678273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b4c102fe0 0x7f0b4c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 >> 172.21.15.112:0/4001678273 conn(0x7f0b4c079370 msgr2=0x7f0b4c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 shutdown_connections 2024-05-10T10:37:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:21.942+0000 7f0b5431c700 1 -- 172.21.15.112:0/4001678273 wait complete. 2024-05-10T10:37:21.945 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-10T10:37:22.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:22 smithi112 ceph-mon[30100]: pgmap v342: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-05-10T10:37:22.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:22 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4001678273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:37:22.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:37:22.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:22 smithi148 ceph-mon[33568]: pgmap v342: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-05-10T10:37:22.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:22 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4001678273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:37:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:22 smithi195 ceph-mon[33773]: pgmap v342: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-05-10T10:37:22.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:22 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4001678273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:37:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:24.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:24 smithi112 ceph-mon[30100]: pgmap v343: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-10T10:37:24.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:24 smithi148 ceph-mon[33568]: pgmap v343: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-10T10:37:24.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:24 smithi195 ceph-mon[33773]: pgmap v343: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-10T10:37:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 -- 172.21.15.112:0/2015586576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164103750 msgr2=0x7f4164103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 --2- 172.21.15.112:0/2015586576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164103750 0x7f4164103b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41540099c0 tx=0x7f4154009cd0 comp rx=0 tx=0).stop 2024-05-10T10:37:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 -- 172.21.15.112:0/2015586576 shutdown_connections 2024-05-10T10:37:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 --2- 172.21.15.112:0/2015586576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164108850 0x7f41641103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 --2- 172.21.15.112:0/2015586576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164103750 0x7f4164103b30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 --2- 172.21.15.112:0/2015586576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4164104100 0x7f4164108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 -- 172.21.15.112:0/2015586576 >> 172.21.15.112:0/2015586576 conn(0x7f416407a600 msgr2=0x7f416407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 -- 172.21.15.112:0/2015586576 shutdown_connections 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.832+0000 7f4169844700 1 -- 172.21.15.112:0/2015586576 wait complete. 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 Processor -- start 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 -- start start 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164103750 0x7f416419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4164104100 0x7f416419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164108850 0x7f4164199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4164113550 con 0x7f4164108850 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41641133d0 con 0x7f4164104100 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4169844700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41641136d0 con 0x7f4164103750 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164108850 0x7f4164199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164108850 0x7f4164199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54754/0 (socket says 172.21.15.112:54754) 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 -- 172.21.15.112:0/4088052656 learned_addr learned my addr 172.21.15.112:0/4088052656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41627fc700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4164104100 0x7f416419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f4162ffd700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164103750 0x7f416419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 -- 172.21.15.112:0/4088052656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164103750 msgr2=0x7f416419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164103750 0x7f416419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 -- 172.21.15.112:0/4088052656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4164104100 msgr2=0x7f416419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4164104100 0x7f416419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.833+0000 7f41637fe700 1 -- 172.21.15.112:0/4088052656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416419a030 con 0x7f4164108850 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.834+0000 7f41637fe700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164108850 0x7f4164199850 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f415c00b970 tx=0x7f415c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.834+0000 7f4168842700 1 -- 172.21.15.112:0/4088052656 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c019070 con 0x7f4164108850 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.834+0000 7f4168842700 1 -- 172.21.15.112:0/4088052656 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f415c00cdc0 con 0x7f4164108850 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.834+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416419a320 con 0x7f4164108850 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.834+0000 7f4168842700 1 -- 172.21.15.112:0/4088052656 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c014e30 con 0x7f4164108850 2024-05-10T10:37:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.834+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41641a6530 con 0x7f4164108850 2024-05-10T10:37:24.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.835+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416410ba90 con 0x7f4164108850 2024-05-10T10:37:24.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.836+0000 7f4168842700 1 -- 172.21.15.112:0/4088052656 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f415c01c070 con 0x7f4164108850 2024-05-10T10:37:24.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.837+0000 7f4168842700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f414c0b2990 0x7f414c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:24.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.837+0000 7f4162ffd700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f414c0b2990 0x7f414c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:24.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.838+0000 7f4168842700 1 -- 172.21.15.112:0/4088052656 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f415c0e1d60 con 0x7f4164108850 2024-05-10T10:37:24.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.838+0000 7f4162ffd700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f414c0b2990 0x7f414c0b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f415401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:24.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:24.841+0000 7f4168842700 1 -- 172.21.15.112:0/4088052656 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f415c0a96d0 con 0x7f4164108850 2024-05-10T10:37:25.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.048+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f416410b470 con 0x7f4164108850 2024-05-10T10:37:25.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.049+0000 7f4168842700 1 -- 172.21.15.112:0/4088052656 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24636 (secure 0 0 0) 0x7f415c0a8fd0 con 0x7f4164108850 2024-05-10T10:37:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:37:25.051 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":88,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:36:35.100943+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:25.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.051+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f414c0b2990 msgr2=0x7f414c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:25.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.051+0000 7f4169844700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f414c0b2990 0x7f414c0b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f415401a040 comp rx=0 tx=0).stop 2024-05-10T10:37:25.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.051+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164108850 msgr2=0x7f4164199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:25.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.051+0000 7f4169844700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164108850 0x7f4164199850 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f415c00b970 tx=0x7f415c00bc80 comp rx=0 tx=0).stop 2024-05-10T10:37:25.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 shutdown_connections 2024-05-10T10:37:25.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4164108850 0x7f4164199850 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:25.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f414c0b2990 0x7f414c0b4e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164103750 0x7f416419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 --2- 172.21.15.112:0/4088052656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4164104100 0x7f416419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 >> 172.21.15.112:0/4088052656 conn(0x7f416407a600 msgr2=0x7f4164101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 shutdown_connections 2024-05-10T10:37:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:25.052+0000 7f4169844700 1 -- 172.21.15.112:0/4088052656 wait complete. 2024-05-10T10:37:25.704 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:25 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4088052656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:25.706 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T10:37:25.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T10:37:25.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:25 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4088052656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:25.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:25 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4088052656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:25.949 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:26.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:26 smithi148 ceph-mon[33568]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-05-10T10:37:26.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:26 smithi195 ceph-mon[33773]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-05-10T10:37:26.820 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:26 smithi112 ceph-mon[30100]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-05-10T10:37:27.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/1816518753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 msgr2=0x7fe2e4102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/1816518753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e4102810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009990 tx=0x7fe2d8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/1816518753 shutdown_connections 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/1816518753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2e4102d50 0x7fe2e410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/1816518753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2e4108400 0x7fe2e41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/1816518753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e4102810 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/1816518753 >> 172.21.15.112:0/1816518753 conn(0x7fe2e4079310 msgr2=0x7fe2e4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/1816518753 shutdown_connections 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.793+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/1816518753 wait complete. 2024-05-10T10:37:27.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 Processor -- start 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 -- start start 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2e4102d50 0x7fe2e419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2e4108400 0x7fe2e41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e41124d0 con 0x7fe2e4102390 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e4112350 con 0x7fe2e4102d50 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2ec1f2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e4112650 con 0x7fe2e4108400 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.794+0000 7fe2e9f8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34776/0 (socket says 172.21.15.112:34776) 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 -- 172.21.15.112:0/2410924784 learned_addr learned my addr 172.21.15.112:0/2410924784 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e978d700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2e4102d50 0x7fe2e419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2ea78f700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2e4108400 0x7fe2e41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 -- 172.21.15.112:0/2410924784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2e4108400 msgr2=0x7fe2e41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2e4108400 0x7fe2e41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 -- 172.21.15.112:0/2410924784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2e4102d50 msgr2=0x7fe2e419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2e4102d50 0x7fe2e419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 -- 172.21.15.112:0/2410924784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e4199010 con 0x7fe2e4102390 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2e9f8e700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e419e6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000b530 tx=0x7fe2e000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2d6ffd700 1 -- 172.21.15.112:0/2410924784 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0015020 con 0x7fe2e4102390 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2d6ffd700 1 -- 172.21.15.112:0/2410924784 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2e0011cc0 con 0x7fe2e4102390 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2d6ffd700 1 -- 172.21.15.112:0/2410924784 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0010dc0 con 0x7fe2e4102390 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e4199300 con 0x7fe2e4102390 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.795+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e406ba10 con 0x7fe2e4102390 2024-05-10T10:37:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.796+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e406a190 con 0x7fe2e4102390 2024-05-10T10:37:27.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.798+0000 7fe2d6ffd700 1 -- 172.21.15.112:0/2410924784 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe2e0018020 con 0x7fe2e4102390 2024-05-10T10:37:27.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.798+0000 7fe2d6ffd700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe2d00b2910 0x7fe2d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.798+0000 7fe2e978d700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe2d00b2910 0x7fe2d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.798+0000 7fe2d6ffd700 1 -- 172.21.15.112:0/2410924784 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fe2e00e2450 con 0x7fe2e4102390 2024-05-10T10:37:27.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.799+0000 7fe2e978d700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe2d00b2910 0x7fe2d00b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009750 tx=0x7fe2d800b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:27.802+0000 7fe2d6ffd700 1 -- 172.21.15.112:0/2410924784 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe2e001f020 con 0x7fe2e4102390 2024-05-10T10:37:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.006+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2e406bcd0 con 0x7fe2d00b2910 2024-05-10T10:37:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.016+0000 7fe2d6ffd700 1 -- 172.21.15.112:0/2410924784 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340399 (secure 0 0 0) 0x7fe2e406bcd0 con 0x7fe2d00b2910 2024-05-10T10:37:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe2d00b2910 msgr2=0x7fe2d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe2d00b2910 0x7fe2d00b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009750 tx=0x7fe2d800b560 comp rx=0 tx=0).stop 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 msgr2=0x7fe2e419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e419e6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000b530 tx=0x7fe2e000b840 comp rx=0 tx=0).stop 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 shutdown_connections 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe2d00b2910 0x7fe2d00b4dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2e4108400 0x7fe2e41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e4102390 0x7fe2e419e6f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 --2- 172.21.15.112:0/2410924784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2e4102d50 0x7fe2e419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 >> 172.21.15.112:0/2410924784 conn(0x7fe2e4079310 msgr2=0x7fe2e4101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 shutdown_connections 2024-05-10T10:37:28.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:28.019+0000 7fe2ec1f2700 1 -- 172.21.15.112:0/2410924784 wait complete. 2024-05-10T10:37:28.028 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-10T10:37:28.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:28 smithi148 ceph-mon[33568]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-10T10:37:28.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:28 smithi148 ceph-mon[33568]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:37:28.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:28 smithi112 ceph-mon[30100]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-10T10:37:28.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:28 smithi112 ceph-mon[30100]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:37:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:28 smithi195 ceph-mon[33773]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-10T10:37:28.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:28 smithi195 ceph-mon[33773]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:37:28.826 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-10T10:37:27.289069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654084,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":350,"num_read_kb":269,"num_write":365,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2955541,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367296,"kb_used_data":13984,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574528,"statfs":{"total":1151940427776,"available":1151564316672,"internally_reserved":0,"allocated":14319616,"data_stored":7966167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27579,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":113,"num_read_kb":78,"num_write":18,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003941"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105537+0000","last_change":"2024-05-10T10:35:50.924449+0000","last_active":"2024-05-10T10:36:35.105537+0000","last_peered":"2024-05-10T10:36:35.105537+0000","last_clean":"2024-05-10T10:36:35.105537+0000","last_became_active":"2024-05-10T10:35:50.924114+0000","last_became_peered":"2024-05-10T10:35:50.924114+0000","last_unstale":"2024-05-10T10:36:35.105537+0000","last_undegraded":"2024-05-10T10:36:35.105537+0000","last_fullsized":"2024-05-10T10:36:35.105537+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:50.636501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108327+0000","last_change":"2024-05-10T10:35:50.928715+0000","last_active":"2024-05-10T10:36:35.108327+0000","last_peered":"2024-05-10T10:36:35.108327+0000","last_clean":"2024-05-10T10:36:35.108327+0000","last_became_active":"2024-05-10T10:35:50.928404+0000","last_became_peered":"2024-05-10T10:35:50.928404+0000","last_unstale":"2024-05-10T10:36:35.108327+0000","last_undegraded":"2024-05-10T10:36:35.108327+0000","last_fullsized":"2024-05-10T10:36:35.108327+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:44.934022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107892+0000","last_change":"2024-05-10T10:35:50.925134+0000","last_active":"2024-05-10T10:36:35.107892+0000","last_peered":"2024-05-10T10:36:35.107892+0000","last_clean":"2024-05-10T10:36:35.107892+0000","last_became_active":"2024-05-10T10:35:50.924797+0000","last_became_peered":"2024-05-10T10:35:50.924797+0000","last_unstale":"2024-05-10T10:36:35.107892+0000","last_undegraded":"2024-05-10T10:36:35.107892+0000","last_fullsized":"2024-05-10T10:36:35.107892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:43:52.666519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142241+0000","last_change":"2024-05-10T10:35:50.930916+0000","last_active":"2024-05-10T10:36:35.142241+0000","last_peered":"2024-05-10T10:36:35.142241+0000","last_clean":"2024-05-10T10:36:35.142241+0000","last_became_active":"2024-05-10T10:35:50.930378+0000","last_became_peered":"2024-05-10T10:35:50.930378+0000","last_unstale":"2024-05-10T10:36:35.142241+0000","last_undegraded":"2024-05-10T10:36:35.142241+0000","last_fullsized":"2024-05-10T10:36:35.142241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:07.165595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108632+0000","last_change":"2024-05-10T10:35:50.925459+0000","last_active":"2024-05-10T10:36:35.108632+0000","last_peered":"2024-05-10T10:36:35.108632+0000","last_clean":"2024-05-10T10:36:35.108632+0000","last_became_active":"2024-05-10T10:35:50.925061+0000","last_became_peered":"2024-05-10T10:35:50.925061+0000","last_unstale":"2024-05-10T10:36:35.108632+0000","last_undegraded":"2024-05-10T10:36:35.108632+0000","last_fullsized":"2024-05-10T10:36:35.108632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:48:06.622369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107623+0000","last_change":"2024-05-10T10:35:50.932052+0000","last_active":"2024-05-10T10:36:35.107623+0000","last_peered":"2024-05-10T10:36:35.107623+0000","last_clean":"2024-05-10T10:36:35.107623+0000","last_became_active":"2024-05-10T10:35:50.931685+0000","last_became_peered":"2024-05-10T10:35:50.931685+0000","last_unstale":"2024-05-10T10:36:35.107623+0000","last_undegraded":"2024-05-10T10:36:35.107623+0000","last_fullsized":"2024-05-10T10:36:35.107623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:13:39.930917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141143+0000","last_change":"2024-05-10T10:35:50.930620+0000","last_active":"2024-05-10T10:36:35.141143+0000","last_peered":"2024-05-10T10:36:35.141143+0000","last_clean":"2024-05-10T10:36:35.141143+0000","last_became_active":"2024-05-10T10:35:50.929696+0000","last_became_peered":"2024-05-10T10:35:50.929696+0000","last_unstale":"2024-05-10T10:36:35.141143+0000","last_undegraded":"2024-05-10T10:36:35.141143+0000","last_fullsized":"2024-05-10T10:36:35.141143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:23.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107834+0000","last_change":"2024-05-10T10:35:50.937517+0000","last_active":"2024-05-10T10:36:35.107834+0000","last_peered":"2024-05-10T10:36:35.107834+0000","last_clean":"2024-05-10T10:36:35.107834+0000","last_became_active":"2024-05-10T10:35:50.937180+0000","last_became_peered":"2024-05-10T10:35:50.937180+0000","last_unstale":"2024-05-10T10:36:35.107834+0000","last_undegraded":"2024-05-10T10:36:35.107834+0000","last_fullsized":"2024-05-10T10:36:35.107834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.568509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110387+0000","last_change":"2024-05-10T10:35:50.937825+0000","last_active":"2024-05-10T10:36:35.110387+0000","last_peered":"2024-05-10T10:36:35.110387+0000","last_clean":"2024-05-10T10:36:35.110387+0000","last_became_active":"2024-05-10T10:35:50.935650+0000","last_became_peered":"2024-05-10T10:35:50.935650+0000","last_unstale":"2024-05-10T10:36:35.110387+0000","last_undegraded":"2024-05-10T10:36:35.110387+0000","last_fullsized":"2024-05-10T10:36:35.110387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:43:55.780266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107567+0000","last_change":"2024-05-10T10:35:50.933598+0000","last_active":"2024-05-10T10:36:35.107567+0000","last_peered":"2024-05-10T10:36:35.107567+0000","last_clean":"2024-05-10T10:36:35.107567+0000","last_became_active":"2024-05-10T10:35:50.932767+0000","last_became_peered":"2024-05-10T10:35:50.932767+0000","last_unstale":"2024-05-10T10:36:35.107567+0000","last_undegraded":"2024-05-10T10:36:35.107567+0000","last_fullsized":"2024-05-10T10:36:35.107567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:40.610376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141292+0000","last_change":"2024-05-10T10:35:50.934882+0000","last_active":"2024-05-10T10:36:35.141292+0000","last_peered":"2024-05-10T10:36:35.141292+0000","last_clean":"2024-05-10T10:36:35.141292+0000","last_became_active":"2024-05-10T10:35:50.934528+0000","last_became_peered":"2024-05-10T10:35:50.934528+0000","last_unstale":"2024-05-10T10:36:35.141292+0000","last_undegraded":"2024-05-10T10:36:35.141292+0000","last_fullsized":"2024-05-10T10:36:35.141292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:15.319765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111545+0000","last_change":"2024-05-10T10:36:35.110939+0000","last_active":"2024-05-10T10:36:35.111545+0000","last_peered":"2024-05-10T10:36:35.111545+0000","last_clean":"2024-05-10T10:36:35.111545+0000","last_became_active":"2024-05-10T10:36:35.110600+0000","last_became_peered":"2024-05-10T10:36:35.110600+0000","last_unstale":"2024-05-10T10:36:35.111545+0000","last_undegraded":"2024-05-10T10:36:35.111545+0000","last_fullsized":"2024-05-10T10:36:35.111545+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:45:58.547301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141104+0000","last_change":"2024-05-10T10:35:50.934161+0000","last_active":"2024-05-10T10:36:35.141104+0000","last_peered":"2024-05-10T10:36:35.141104+0000","last_clean":"2024-05-10T10:36:35.141104+0000","last_became_active":"2024-05-10T10:35:50.933831+0000","last_became_peered":"2024-05-10T10:35:50.933831+0000","last_unstale":"2024-05-10T10:36:35.141104+0000","last_undegraded":"2024-05-10T10:36:35.141104+0000","last_fullsized":"2024-05-10T10:36:35.141104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:09.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107892+0000","last_change":"2024-05-10T10:35:50.927404+0000","last_active":"2024-05-10T10:36:35.107892+0000","last_peered":"2024-05-10T10:36:35.107892+0000","last_clean":"2024-05-10T10:36:35.107892+0000","last_became_active":"2024-05-10T10:35:50.927094+0000","last_became_peered":"2024-05-10T10:35:50.927094+0000","last_unstale":"2024-05-10T10:36:35.107892+0000","last_undegraded":"2024-05-10T10:36:35.107892+0000","last_fullsized":"2024-05-10T10:36:35.107892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:25.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142063+0000","last_change":"2024-05-10T10:36:35.142063+0000","last_active":"2024-05-10T10:36:35.142063+0000","last_peered":"2024-05-10T10:36:35.142063+0000","last_clean":"2024-05-10T10:36:35.142063+0000","last_became_active":"2024-05-10T10:36:35.141685+0000","last_became_peered":"2024-05-10T10:36:35.141685+0000","last_unstale":"2024-05-10T10:36:35.142063+0000","last_undegraded":"2024-05-10T10:36:35.142063+0000","last_fullsized":"2024-05-10T10:36:35.142063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:47.284288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109527+0000","last_change":"2024-05-10T10:35:50.933172+0000","last_active":"2024-05-10T10:36:35.109527+0000","last_peered":"2024-05-10T10:36:35.109527+0000","last_clean":"2024-05-10T10:36:35.109527+0000","last_became_active":"2024-05-10T10:35:50.931864+0000","last_became_peered":"2024-05-10T10:35:50.931864+0000","last_unstale":"2024-05-10T10:36:35.109527+0000","last_undegraded":"2024-05-10T10:36:35.109527+0000","last_fullsized":"2024-05-10T10:36:35.109527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:50.620315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105374+0000","last_change":"2024-05-10T10:35:50.925952+0000","last_active":"2024-05-10T10:36:35.105374+0000","last_peered":"2024-05-10T10:36:35.105374+0000","last_clean":"2024-05-10T10:36:35.105374+0000","last_became_active":"2024-05-10T10:35:50.925610+0000","last_became_peered":"2024-05-10T10:35:50.925610+0000","last_unstale":"2024-05-10T10:36:35.105374+0000","last_undegraded":"2024-05-10T10:36:35.105374+0000","last_fullsized":"2024-05-10T10:36:35.105374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:38.287235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105152+0000","last_change":"2024-05-10T10:35:50.929379+0000","last_active":"2024-05-10T10:36:35.105152+0000","last_peered":"2024-05-10T10:36:35.105152+0000","last_clean":"2024-05-10T10:36:35.105152+0000","last_became_active":"2024-05-10T10:35:50.929082+0000","last_became_peered":"2024-05-10T10:35:50.929082+0000","last_unstale":"2024-05-10T10:36:35.105152+0000","last_undegraded":"2024-05-10T10:36:35.105152+0000","last_fullsized":"2024-05-10T10:36:35.105152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:54.097855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108118+0000","last_change":"2024-05-10T10:35:50.937754+0000","last_active":"2024-05-10T10:36:35.108118+0000","last_peered":"2024-05-10T10:36:35.108118+0000","last_clean":"2024-05-10T10:36:35.108118+0000","last_became_active":"2024-05-10T10:35:50.935662+0000","last_became_peered":"2024-05-10T10:35:50.935662+0000","last_unstale":"2024-05-10T10:36:35.108118+0000","last_undegraded":"2024-05-10T10:36:35.108118+0000","last_fullsized":"2024-05-10T10:36:35.108118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:52.692279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107234+0000","last_change":"2024-05-10T10:36:01.989463+0000","last_active":"2024-05-10T10:36:35.107234+0000","last_peered":"2024-05-10T10:36:35.107234+0000","last_clean":"2024-05-10T10:36:35.107234+0000","last_became_active":"2024-05-10T10:36:01.988949+0000","last_became_peered":"2024-05-10T10:36:01.988949+0000","last_unstale":"2024-05-10T10:36:35.107234+0000","last_undegraded":"2024-05-10T10:36:35.107234+0000","last_fullsized":"2024-05-10T10:36:35.107234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:47.529845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:21.434251+0000","last_change":"2024-05-10T10:35:47.909033+0000","last_active":"2024-05-10T10:37:21.434251+0000","last_peered":"2024-05-10T10:37:21.434251+0000","last_clean":"2024-05-10T10:37:21.434251+0000","last_became_active":"2024-05-10T10:35:47.908680+0000","last_became_peered":"2024-05-10T10:35:47.908680+0000","last_unstale":"2024-05-10T10:37:21.434251+0000","last_undegraded":"2024-05-10T10:37:21.434251+0000","last_fullsized":"2024-05-10T10:37:21.434251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:46.642113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108365+0000","last_change":"2024-05-10T10:35:50.939233+0000","last_active":"2024-05-10T10:36:35.108365+0000","last_peered":"2024-05-10T10:36:35.108365+0000","last_clean":"2024-05-10T10:36:35.108365+0000","last_became_active":"2024-05-10T10:35:50.938949+0000","last_became_peered":"2024-05-10T10:35:50.938949+0000","last_unstale":"2024-05-10T10:36:35.108365+0000","last_undegraded":"2024-05-10T10:36:35.108365+0000","last_fullsized":"2024-05-10T10:36:35.108365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:06.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141798+0000","last_change":"2024-05-10T10:36:01.990982+0000","last_active":"2024-05-10T10:36:35.141798+0000","last_peered":"2024-05-10T10:36:35.141798+0000","last_clean":"2024-05-10T10:36:35.141798+0000","last_became_active":"2024-05-10T10:36:01.990325+0000","last_became_peered":"2024-05-10T10:36:01.990325+0000","last_unstale":"2024-05-10T10:36:35.141798+0000","last_undegraded":"2024-05-10T10:36:35.141798+0000","last_fullsized":"2024-05-10T10:36:35.141798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:18.424130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108588+0000","last_change":"2024-05-10T10:35:47.911239+0000","last_active":"2024-05-10T10:36:35.108588+0000","last_peered":"2024-05-10T10:36:35.108588+0000","last_clean":"2024-05-10T10:36:35.108588+0000","last_became_active":"2024-05-10T10:35:47.910945+0000","last_became_peered":"2024-05-10T10:35:47.910945+0000","last_unstale":"2024-05-10T10:36:35.108588+0000","last_undegraded":"2024-05-10T10:36:35.108588+0000","last_fullsized":"2024-05-10T10:36:35.108588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:08:52.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108444+0000","last_change":"2024-05-10T10:35:50.923694+0000","last_active":"2024-05-10T10:36:35.108444+0000","last_peered":"2024-05-10T10:36:35.108444+0000","last_clean":"2024-05-10T10:36:35.108444+0000","last_became_active":"2024-05-10T10:35:50.923381+0000","last_became_peered":"2024-05-10T10:35:50.923381+0000","last_unstale":"2024-05-10T10:36:35.108444+0000","last_undegraded":"2024-05-10T10:36:35.108444+0000","last_fullsized":"2024-05-10T10:36:35.108444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:59.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142217+0000","last_change":"2024-05-10T10:36:01.989868+0000","last_active":"2024-05-10T10:36:35.142217+0000","last_peered":"2024-05-10T10:36:35.142217+0000","last_clean":"2024-05-10T10:36:35.142217+0000","last_became_active":"2024-05-10T10:36:01.989283+0000","last_became_peered":"2024-05-10T10:36:01.989283+0000","last_unstale":"2024-05-10T10:36:35.142217+0000","last_undegraded":"2024-05-10T10:36:35.142217+0000","last_fullsized":"2024-05-10T10:36:35.142217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:35.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:01.405807+0000","last_change":"2024-05-10T10:35:47.915555+0000","last_active":"2024-05-10T10:37:01.405807+0000","last_peered":"2024-05-10T10:37:01.405807+0000","last_clean":"2024-05-10T10:37:01.405807+0000","last_became_active":"2024-05-10T10:35:47.912852+0000","last_became_peered":"2024-05-10T10:35:47.912852+0000","last_unstale":"2024-05-10T10:37:01.405807+0000","last_undegraded":"2024-05-10T10:37:01.405807+0000","last_fullsized":"2024-05-10T10:37:01.405807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:56:45.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109741+0000","last_change":"2024-05-10T10:35:50.934010+0000","last_active":"2024-05-10T10:36:35.109741+0000","last_peered":"2024-05-10T10:36:35.109741+0000","last_clean":"2024-05-10T10:36:35.109741+0000","last_became_active":"2024-05-10T10:35:50.933653+0000","last_became_peered":"2024-05-10T10:35:50.933653+0000","last_unstale":"2024-05-10T10:36:35.109741+0000","last_undegraded":"2024-05-10T10:36:35.109741+0000","last_fullsized":"2024-05-10T10:36:35.109741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:34.862734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142118+0000","last_change":"2024-05-10T10:36:01.990524+0000","last_active":"2024-05-10T10:36:35.142118+0000","last_peered":"2024-05-10T10:36:35.142118+0000","last_clean":"2024-05-10T10:36:35.142118+0000","last_became_active":"2024-05-10T10:36:01.990011+0000","last_became_peered":"2024-05-10T10:36:01.990011+0000","last_unstale":"2024-05-10T10:36:35.142118+0000","last_undegraded":"2024-05-10T10:36:35.142118+0000","last_fullsized":"2024-05-10T10:36:35.142118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:06.123095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.403594+0000","last_change":"2024-05-10T10:35:47.910977+0000","last_active":"2024-05-10T10:37:00.403594+0000","last_peered":"2024-05-10T10:37:00.403594+0000","last_clean":"2024-05-10T10:37:00.403594+0000","last_became_active":"2024-05-10T10:35:47.910623+0000","last_became_peered":"2024-05-10T10:35:47.910623+0000","last_unstale":"2024-05-10T10:37:00.403594+0000","last_undegraded":"2024-05-10T10:37:00.403594+0000","last_fullsized":"2024-05-10T10:37:00.403594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:37.760666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.112228+0000","last_change":"2024-05-10T10:35:50.937238+0000","last_active":"2024-05-10T10:36:35.112228+0000","last_peered":"2024-05-10T10:36:35.112228+0000","last_clean":"2024-05-10T10:36:35.112228+0000","last_became_active":"2024-05-10T10:35:50.936298+0000","last_became_peered":"2024-05-10T10:35:50.936298+0000","last_unstale":"2024-05-10T10:36:35.112228+0000","last_undegraded":"2024-05-10T10:36:35.112228+0000","last_fullsized":"2024-05-10T10:36:35.112228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:54.724452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107861+0000","last_change":"2024-05-10T10:36:01.987769+0000","last_active":"2024-05-10T10:36:35.107861+0000","last_peered":"2024-05-10T10:36:35.107861+0000","last_clean":"2024-05-10T10:36:35.107861+0000","last_became_active":"2024-05-10T10:36:01.987367+0000","last_became_peered":"2024-05-10T10:36:01.987367+0000","last_unstale":"2024-05-10T10:36:35.107861+0000","last_undegraded":"2024-05-10T10:36:35.107861+0000","last_fullsized":"2024-05-10T10:36:35.107861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:36.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.438422+0000","last_change":"2024-05-10T10:35:47.915082+0000","last_active":"2024-05-10T10:37:23.438422+0000","last_peered":"2024-05-10T10:37:23.438422+0000","last_clean":"2024-05-10T10:37:23.438422+0000","last_became_active":"2024-05-10T10:35:47.914722+0000","last_became_peered":"2024-05-10T10:35:47.914722+0000","last_unstale":"2024-05-10T10:37:23.438422+0000","last_undegraded":"2024-05-10T10:37:23.438422+0000","last_fullsized":"2024-05-10T10:37:23.438422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:31.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108511+0000","last_change":"2024-05-10T10:35:50.926181+0000","last_active":"2024-05-10T10:36:35.108511+0000","last_peered":"2024-05-10T10:36:35.108511+0000","last_clean":"2024-05-10T10:36:35.108511+0000","last_became_active":"2024-05-10T10:35:50.925865+0000","last_became_peered":"2024-05-10T10:35:50.925865+0000","last_unstale":"2024-05-10T10:36:35.108511+0000","last_undegraded":"2024-05-10T10:36:35.108511+0000","last_fullsized":"2024-05-10T10:36:35.108511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:04.976908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142412+0000","last_change":"2024-05-10T10:36:35.142412+0000","last_active":"2024-05-10T10:36:35.142412+0000","last_peered":"2024-05-10T10:36:35.142412+0000","last_clean":"2024-05-10T10:36:35.142412+0000","last_became_active":"2024-05-10T10:36:35.141959+0000","last_became_peered":"2024-05-10T10:36:35.141959+0000","last_unstale":"2024-05-10T10:36:35.142412+0000","last_undegraded":"2024-05-10T10:36:35.142412+0000","last_fullsized":"2024-05-10T10:36:35.142412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:20:34.133261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:59.392578+0000","last_change":"2024-05-10T10:35:47.912826+0000","last_active":"2024-05-10T10:36:59.392578+0000","last_peered":"2024-05-10T10:36:59.392578+0000","last_clean":"2024-05-10T10:36:59.392578+0000","last_became_active":"2024-05-10T10:35:47.912468+0000","last_became_peered":"2024-05-10T10:35:47.912468+0000","last_unstale":"2024-05-10T10:36:59.392578+0000","last_undegraded":"2024-05-10T10:36:59.392578+0000","last_fullsized":"2024-05-10T10:36:59.392578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:56.482104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.114920+0000","last_change":"2024-05-10T10:36:35.114920+0000","last_active":"2024-05-10T10:36:35.114920+0000","last_peered":"2024-05-10T10:36:35.114920+0000","last_clean":"2024-05-10T10:36:35.114920+0000","last_became_active":"2024-05-10T10:36:35.114546+0000","last_became_peered":"2024-05-10T10:36:35.114546+0000","last_unstale":"2024-05-10T10:36:35.114920+0000","last_undegraded":"2024-05-10T10:36:35.114920+0000","last_fullsized":"2024-05-10T10:36:35.114920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:52.454111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105050+0000","last_change":"2024-05-10T10:36:01.990086+0000","last_active":"2024-05-10T10:36:35.105050+0000","last_peered":"2024-05-10T10:36:35.105050+0000","last_clean":"2024-05-10T10:36:35.105050+0000","last_became_active":"2024-05-10T10:36:01.989624+0000","last_became_peered":"2024-05-10T10:36:01.989624+0000","last_unstale":"2024-05-10T10:36:35.105050+0000","last_undegraded":"2024-05-10T10:36:35.105050+0000","last_fullsized":"2024-05-10T10:36:35.105050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:33.708069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:58.384130+0000","last_change":"2024-05-10T10:35:47.905182+0000","last_active":"2024-05-10T10:36:58.384130+0000","last_peered":"2024-05-10T10:36:58.384130+0000","last_clean":"2024-05-10T10:36:58.384130+0000","last_became_active":"2024-05-10T10:35:47.904855+0000","last_became_peered":"2024-05-10T10:35:47.904855+0000","last_unstale":"2024-05-10T10:36:58.384130+0000","last_undegraded":"2024-05-10T10:36:58.384130+0000","last_fullsized":"2024-05-10T10:36:58.384130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:17:44.844156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107749+0000","last_change":"2024-05-10T10:35:50.935478+0000","last_active":"2024-05-10T10:36:35.107749+0000","last_peered":"2024-05-10T10:36:35.107749+0000","last_clean":"2024-05-10T10:36:35.107749+0000","last_became_active":"2024-05-10T10:35:50.933487+0000","last_became_peered":"2024-05-10T10:35:50.933487+0000","last_unstale":"2024-05-10T10:36:35.107749+0000","last_undegraded":"2024-05-10T10:36:35.107749+0000","last_fullsized":"2024-05-10T10:36:35.107749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:02.433423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108100+0000","last_change":"2024-05-10T10:36:01.984285+0000","last_active":"2024-05-10T10:36:35.108100+0000","last_peered":"2024-05-10T10:36:35.108100+0000","last_clean":"2024-05-10T10:36:35.108100+0000","last_became_active":"2024-05-10T10:36:01.983900+0000","last_became_peered":"2024-05-10T10:36:01.983900+0000","last_unstale":"2024-05-10T10:36:35.108100+0000","last_undegraded":"2024-05-10T10:36:35.108100+0000","last_fullsized":"2024-05-10T10:36:35.108100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:53.406106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.403241+0000","last_change":"2024-05-10T10:35:47.911123+0000","last_active":"2024-05-10T10:37:00.403241+0000","last_peered":"2024-05-10T10:37:00.403241+0000","last_clean":"2024-05-10T10:37:00.403241+0000","last_became_active":"2024-05-10T10:35:47.910762+0000","last_became_peered":"2024-05-10T10:35:47.910762+0000","last_unstale":"2024-05-10T10:37:00.403241+0000","last_undegraded":"2024-05-10T10:37:00.403241+0000","last_fullsized":"2024-05-10T10:37:00.403241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:07.045586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104519+0000","last_change":"2024-05-10T10:35:50.928093+0000","last_active":"2024-05-10T10:36:35.104519+0000","last_peered":"2024-05-10T10:36:35.104519+0000","last_clean":"2024-05-10T10:36:35.104519+0000","last_became_active":"2024-05-10T10:35:50.926945+0000","last_became_peered":"2024-05-10T10:35:50.926945+0000","last_unstale":"2024-05-10T10:36:35.104519+0000","last_undegraded":"2024-05-10T10:36:35.104519+0000","last_fullsized":"2024-05-10T10:36:35.104519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:11.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.143043+0000","last_change":"2024-05-10T10:36:35.143043+0000","last_active":"2024-05-10T10:36:35.143043+0000","last_peered":"2024-05-10T10:36:35.143043+0000","last_clean":"2024-05-10T10:36:35.143043+0000","last_became_active":"2024-05-10T10:36:35.142739+0000","last_became_peered":"2024-05-10T10:36:35.142739+0000","last_unstale":"2024-05-10T10:36:35.143043+0000","last_undegraded":"2024-05-10T10:36:35.143043+0000","last_fullsized":"2024-05-10T10:36:35.143043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:10.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.437686+0000","last_change":"2024-05-10T10:35:47.908687+0000","last_active":"2024-05-10T10:37:23.437686+0000","last_peered":"2024-05-10T10:37:23.437686+0000","last_clean":"2024-05-10T10:37:23.437686+0000","last_became_active":"2024-05-10T10:35:47.908349+0000","last_became_peered":"2024-05-10T10:35:47.908349+0000","last_unstale":"2024-05-10T10:37:23.437686+0000","last_undegraded":"2024-05-10T10:37:23.437686+0000","last_fullsized":"2024-05-10T10:37:23.437686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:48.696104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110473+0000","last_change":"2024-05-10T10:35:50.933729+0000","last_active":"2024-05-10T10:36:35.110473+0000","last_peered":"2024-05-10T10:36:35.110473+0000","last_clean":"2024-05-10T10:36:35.110473+0000","last_became_active":"2024-05-10T10:35:50.933109+0000","last_became_peered":"2024-05-10T10:35:50.933109+0000","last_unstale":"2024-05-10T10:36:35.110473+0000","last_undegraded":"2024-05-10T10:36:35.110473+0000","last_fullsized":"2024-05-10T10:36:35.110473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:10:38.305592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104491+0000","last_change":"2024-05-10T10:36:01.991969+0000","last_active":"2024-05-10T10:36:35.104491+0000","last_peered":"2024-05-10T10:36:35.104491+0000","last_clean":"2024-05-10T10:36:35.104491+0000","last_became_active":"2024-05-10T10:36:01.990278+0000","last_became_peered":"2024-05-10T10:36:01.990278+0000","last_unstale":"2024-05-10T10:36:35.104491+0000","last_undegraded":"2024-05-10T10:36:35.104491+0000","last_fullsized":"2024-05-10T10:36:35.104491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:33.070520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.401808+0000","last_change":"2024-05-10T10:35:47.911682+0000","last_active":"2024-05-10T10:37:00.401808+0000","last_peered":"2024-05-10T10:37:00.401808+0000","last_clean":"2024-05-10T10:37:00.401808+0000","last_became_active":"2024-05-10T10:35:47.911328+0000","last_became_peered":"2024-05-10T10:35:47.911328+0000","last_unstale":"2024-05-10T10:37:00.401808+0000","last_undegraded":"2024-05-10T10:37:00.401808+0000","last_fullsized":"2024-05-10T10:37:00.401808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.361340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108539+0000","last_change":"2024-05-10T10:35:50.931541+0000","last_active":"2024-05-10T10:36:35.108539+0000","last_peered":"2024-05-10T10:36:35.108539+0000","last_clean":"2024-05-10T10:36:35.108539+0000","last_became_active":"2024-05-10T10:35:50.931244+0000","last_became_peered":"2024-05-10T10:35:50.931244+0000","last_unstale":"2024-05-10T10:36:35.108539+0000","last_undegraded":"2024-05-10T10:36:35.108539+0000","last_fullsized":"2024-05-10T10:36:35.108539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:38.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142480+0000","last_change":"2024-05-10T10:36:01.990390+0000","last_active":"2024-05-10T10:36:35.142480+0000","last_peered":"2024-05-10T10:36:35.142480+0000","last_clean":"2024-05-10T10:36:35.142480+0000","last_became_active":"2024-05-10T10:36:01.989434+0000","last_became_peered":"2024-05-10T10:36:01.989434+0000","last_unstale":"2024-05-10T10:36:35.142480+0000","last_undegraded":"2024-05-10T10:36:35.142480+0000","last_fullsized":"2024-05-10T10:36:35.142480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:53.129212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:59.392291+0000","last_change":"2024-05-10T10:35:47.911590+0000","last_active":"2024-05-10T10:36:59.392291+0000","last_peered":"2024-05-10T10:36:59.392291+0000","last_clean":"2024-05-10T10:36:59.392291+0000","last_became_active":"2024-05-10T10:35:47.910950+0000","last_became_peered":"2024-05-10T10:35:47.910950+0000","last_unstale":"2024-05-10T10:36:59.392291+0000","last_undegraded":"2024-05-10T10:36:59.392291+0000","last_fullsized":"2024-05-10T10:36:59.392291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:18.844774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110047+0000","last_change":"2024-05-10T10:35:50.938174+0000","last_active":"2024-05-10T10:36:35.110047+0000","last_peered":"2024-05-10T10:36:35.110047+0000","last_clean":"2024-05-10T10:36:35.110047+0000","last_became_active":"2024-05-10T10:35:50.937618+0000","last_became_peered":"2024-05-10T10:35:50.937618+0000","last_unstale":"2024-05-10T10:36:35.110047+0000","last_undegraded":"2024-05-10T10:36:35.110047+0000","last_fullsized":"2024-05-10T10:36:35.110047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:26:48.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104514+0000","last_change":"2024-05-10T10:36:01.979226+0000","last_active":"2024-05-10T10:36:35.104514+0000","last_peered":"2024-05-10T10:36:35.104514+0000","last_clean":"2024-05-10T10:36:35.104514+0000","last_became_active":"2024-05-10T10:36:01.978847+0000","last_became_peered":"2024-05-10T10:36:01.978847+0000","last_unstale":"2024-05-10T10:36:35.104514+0000","last_undegraded":"2024-05-10T10:36:35.104514+0000","last_fullsized":"2024-05-10T10:36:35.104514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:10.629813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":140,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.439285+0000","last_change":"2024-05-10T10:36:35.150814+0000","last_active":"2024-05-10T10:37:23.439285+0000","last_peered":"2024-05-10T10:37:23.439285+0000","last_clean":"2024-05-10T10:37:23.439285+0000","last_became_active":"2024-05-10T10:36:35.142980+0000","last_became_peered":"2024-05-10T10:36:35.142980+0000","last_unstale":"2024-05-10T10:37:23.439285+0000","last_undegraded":"2024-05-10T10:37:23.439285+0000","last_fullsized":"2024-05-10T10:37:23.439285+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:17.907275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56669,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":82,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109665+0000","last_change":"2024-05-10T10:35:50.925584+0000","last_active":"2024-05-10T10:36:35.109665+0000","last_peered":"2024-05-10T10:36:35.109665+0000","last_clean":"2024-05-10T10:36:35.109665+0000","last_became_active":"2024-05-10T10:35:50.925215+0000","last_became_peered":"2024-05-10T10:35:50.925215+0000","last_unstale":"2024-05-10T10:36:35.109665+0000","last_undegraded":"2024-05-10T10:36:35.109665+0000","last_fullsized":"2024-05-10T10:36:35.109665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:03.377110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.112541+0000","last_change":"2024-05-10T10:36:01.985090+0000","last_active":"2024-05-10T10:36:35.112541+0000","last_peered":"2024-05-10T10:36:35.112541+0000","last_clean":"2024-05-10T10:36:35.112541+0000","last_became_active":"2024-05-10T10:36:01.984059+0000","last_became_peered":"2024-05-10T10:36:01.984059+0000","last_unstale":"2024-05-10T10:36:35.112541+0000","last_undegraded":"2024-05-10T10:36:35.112541+0000","last_fullsized":"2024-05-10T10:36:35.112541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:41.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:01.404996+0000","last_change":"2024-05-10T10:35:47.914377+0000","last_active":"2024-05-10T10:37:01.404996+0000","last_peered":"2024-05-10T10:37:01.404996+0000","last_clean":"2024-05-10T10:37:01.404996+0000","last_became_active":"2024-05-10T10:35:47.913830+0000","last_became_peered":"2024-05-10T10:35:47.913830+0000","last_unstale":"2024-05-10T10:37:01.404996+0000","last_undegraded":"2024-05-10T10:37:01.404996+0000","last_fullsized":"2024-05-10T10:37:01.404996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:54:18.540952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.140857+0000","last_change":"2024-05-10T10:35:50.932721+0000","last_active":"2024-05-10T10:36:35.140857+0000","last_peered":"2024-05-10T10:36:35.140857+0000","last_clean":"2024-05-10T10:36:35.140857+0000","last_became_active":"2024-05-10T10:35:50.932410+0000","last_became_peered":"2024-05-10T10:35:50.932410+0000","last_unstale":"2024-05-10T10:36:35.140857+0000","last_undegraded":"2024-05-10T10:36:35.140857+0000","last_fullsized":"2024-05-10T10:36:35.140857+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:42.255534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.143581+0000","last_change":"2024-05-10T10:36:35.143581+0000","last_active":"2024-05-10T10:36:35.143581+0000","last_peered":"2024-05-10T10:36:35.143581+0000","last_clean":"2024-05-10T10:36:35.143581+0000","last_became_active":"2024-05-10T10:36:35.142957+0000","last_became_peered":"2024-05-10T10:36:35.142957+0000","last_unstale":"2024-05-10T10:36:35.143581+0000","last_undegraded":"2024-05-10T10:36:35.143581+0000","last_fullsized":"2024-05-10T10:36:35.143581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:14:39.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402941+0000","last_change":"2024-05-10T10:35:47.908874+0000","last_active":"2024-05-10T10:37:00.402941+0000","last_peered":"2024-05-10T10:37:00.402941+0000","last_clean":"2024-05-10T10:37:00.402941+0000","last_became_active":"2024-05-10T10:35:47.908478+0000","last_became_peered":"2024-05-10T10:35:47.908478+0000","last_unstale":"2024-05-10T10:37:00.402941+0000","last_undegraded":"2024-05-10T10:37:00.402941+0000","last_fullsized":"2024-05-10T10:37:00.402941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:13.147762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141379+0000","last_change":"2024-05-10T10:35:50.925168+0000","last_active":"2024-05-10T10:36:35.141379+0000","last_peered":"2024-05-10T10:36:35.141379+0000","last_clean":"2024-05-10T10:36:35.141379+0000","last_became_active":"2024-05-10T10:35:50.924509+0000","last_became_peered":"2024-05-10T10:35:50.924509+0000","last_unstale":"2024-05-10T10:36:35.141379+0000","last_undegraded":"2024-05-10T10:36:35.141379+0000","last_fullsized":"2024-05-10T10:36:35.141379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:08.458195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.144972+0000","last_change":"2024-05-10T10:36:35.144575+0000","last_active":"2024-05-10T10:36:35.144972+0000","last_peered":"2024-05-10T10:36:35.144972+0000","last_clean":"2024-05-10T10:36:35.144972+0000","last_became_active":"2024-05-10T10:36:35.144165+0000","last_became_peered":"2024-05-10T10:36:35.144165+0000","last_unstale":"2024-05-10T10:36:35.144972+0000","last_undegraded":"2024-05-10T10:36:35.144972+0000","last_fullsized":"2024-05-10T10:36:35.144972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:45.179163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.437814+0000","last_change":"2024-05-10T10:35:47.911208+0000","last_active":"2024-05-10T10:37:23.437814+0000","last_peered":"2024-05-10T10:37:23.437814+0000","last_clean":"2024-05-10T10:37:23.437814+0000","last_became_active":"2024-05-10T10:35:47.910820+0000","last_became_peered":"2024-05-10T10:35:47.910820+0000","last_unstale":"2024-05-10T10:37:23.437814+0000","last_undegraded":"2024-05-10T10:37:23.437814+0000","last_fullsized":"2024-05-10T10:37:23.437814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:04.804203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141550+0000","last_change":"2024-05-10T10:35:50.928195+0000","last_active":"2024-05-10T10:36:35.141550+0000","last_peered":"2024-05-10T10:36:35.141550+0000","last_clean":"2024-05-10T10:36:35.141550+0000","last_became_active":"2024-05-10T10:35:50.927832+0000","last_became_peered":"2024-05-10T10:35:50.927832+0000","last_unstale":"2024-05-10T10:36:35.141550+0000","last_undegraded":"2024-05-10T10:36:35.141550+0000","last_fullsized":"2024-05-10T10:36:35.141550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:03.071525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141227+0000","last_change":"2024-05-10T10:36:01.982483+0000","last_active":"2024-05-10T10:36:35.141227+0000","last_peered":"2024-05-10T10:36:35.141227+0000","last_clean":"2024-05-10T10:36:35.141227+0000","last_became_active":"2024-05-10T10:36:01.982117+0000","last_became_peered":"2024-05-10T10:36:01.982117+0000","last_unstale":"2024-05-10T10:36:35.141227+0000","last_undegraded":"2024-05-10T10:36:35.141227+0000","last_fullsized":"2024-05-10T10:36:35.141227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:08.145035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.401506+0000","last_change":"2024-05-10T10:35:47.912929+0000","last_active":"2024-05-10T10:37:00.401506+0000","last_peered":"2024-05-10T10:37:00.401506+0000","last_clean":"2024-05-10T10:37:00.401506+0000","last_became_active":"2024-05-10T10:35:47.912575+0000","last_became_peered":"2024-05-10T10:35:47.912575+0000","last_unstale":"2024-05-10T10:37:00.401506+0000","last_undegraded":"2024-05-10T10:37:00.401506+0000","last_fullsized":"2024-05-10T10:37:00.401506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:03.235579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111381+0000","last_change":"2024-05-10T10:35:50.935967+0000","last_active":"2024-05-10T10:36:35.111381+0000","last_peered":"2024-05-10T10:36:35.111381+0000","last_clean":"2024-05-10T10:36:35.111381+0000","last_became_active":"2024-05-10T10:35:50.935635+0000","last_became_peered":"2024-05-10T10:35:50.935635+0000","last_unstale":"2024-05-10T10:36:35.111381+0000","last_undegraded":"2024-05-10T10:36:35.111381+0000","last_fullsized":"2024-05-10T10:36:35.111381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:02:53.681522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111262+0000","last_change":"2024-05-10T10:36:01.979488+0000","last_active":"2024-05-10T10:36:35.111262+0000","last_peered":"2024-05-10T10:36:35.111262+0000","last_clean":"2024-05-10T10:36:35.111262+0000","last_became_active":"2024-05-10T10:36:01.978839+0000","last_became_peered":"2024-05-10T10:36:01.978839+0000","last_unstale":"2024-05-10T10:36:35.111262+0000","last_undegraded":"2024-05-10T10:36:35.111262+0000","last_fullsized":"2024-05-10T10:36:35.111262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:20.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.380137+0000","last_change":"2024-05-10T10:35:47.913235+0000","last_active":"2024-05-10T10:37:00.380137+0000","last_peered":"2024-05-10T10:37:00.380137+0000","last_clean":"2024-05-10T10:37:00.380137+0000","last_became_active":"2024-05-10T10:35:47.912788+0000","last_became_peered":"2024-05-10T10:35:47.912788+0000","last_unstale":"2024-05-10T10:37:00.380137+0000","last_undegraded":"2024-05-10T10:37:00.380137+0000","last_fullsized":"2024-05-10T10:37:00.380137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:56:10.275488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105111+0000","last_change":"2024-05-10T10:35:50.930269+0000","last_active":"2024-05-10T10:36:35.105111+0000","last_peered":"2024-05-10T10:36:35.105111+0000","last_clean":"2024-05-10T10:36:35.105111+0000","last_became_active":"2024-05-10T10:35:50.929918+0000","last_became_peered":"2024-05-10T10:35:50.929918+0000","last_unstale":"2024-05-10T10:36:35.105111+0000","last_undegraded":"2024-05-10T10:36:35.105111+0000","last_fullsized":"2024-05-10T10:36:35.105111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:31.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108511+0000","last_change":"2024-05-10T10:36:02.018506+0000","last_active":"2024-05-10T10:36:35.108511+0000","last_peered":"2024-05-10T10:36:35.108511+0000","last_clean":"2024-05-10T10:36:35.108511+0000","last_became_active":"2024-05-10T10:36:02.018004+0000","last_became_peered":"2024-05-10T10:36:02.018004+0000","last_unstale":"2024-05-10T10:36:35.108511+0000","last_undegraded":"2024-05-10T10:36:35.108511+0000","last_fullsized":"2024-05-10T10:36:35.108511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:37.404970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104862+0000","last_change":"2024-05-10T10:35:47.910654+0000","last_active":"2024-05-10T10:36:35.104862+0000","last_peered":"2024-05-10T10:36:35.104862+0000","last_clean":"2024-05-10T10:36:35.104862+0000","last_became_active":"2024-05-10T10:35:47.910338+0000","last_became_peered":"2024-05-10T10:35:47.910338+0000","last_unstale":"2024-05-10T10:36:35.104862+0000","last_undegraded":"2024-05-10T10:36:35.104862+0000","last_fullsized":"2024-05-10T10:36:35.104862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:07.127574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107328+0000","last_change":"2024-05-10T10:35:50.940950+0000","last_active":"2024-05-10T10:36:35.107328+0000","last_peered":"2024-05-10T10:36:35.107328+0000","last_clean":"2024-05-10T10:36:35.107328+0000","last_became_active":"2024-05-10T10:35:50.940562+0000","last_became_peered":"2024-05-10T10:35:50.940562+0000","last_unstale":"2024-05-10T10:36:35.107328+0000","last_undegraded":"2024-05-10T10:36:35.107328+0000","last_fullsized":"2024-05-10T10:36:35.107328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:24.825900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141056+0000","last_change":"2024-05-10T10:36:01.989277+0000","last_active":"2024-05-10T10:36:35.141056+0000","last_peered":"2024-05-10T10:36:35.141056+0000","last_clean":"2024-05-10T10:36:35.141056+0000","last_became_active":"2024-05-10T10:36:01.988866+0000","last_became_peered":"2024-05-10T10:36:01.988866+0000","last_unstale":"2024-05-10T10:36:35.141056+0000","last_undegraded":"2024-05-10T10:36:35.141056+0000","last_fullsized":"2024-05-10T10:36:35.141056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:08:29.740044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:24.438971+0000","last_change":"2024-05-10T10:35:47.908332+0000","last_active":"2024-05-10T10:37:24.438971+0000","last_peered":"2024-05-10T10:37:24.438971+0000","last_clean":"2024-05-10T10:37:24.438971+0000","last_became_active":"2024-05-10T10:35:47.907933+0000","last_became_peered":"2024-05-10T10:35:47.907933+0000","last_unstale":"2024-05-10T10:37:24.438971+0000","last_undegraded":"2024-05-10T10:37:24.438971+0000","last_fullsized":"2024-05-10T10:37:24.438971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:47:48.702338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107175+0000","last_change":"2024-05-10T10:35:50.934638+0000","last_active":"2024-05-10T10:36:35.107175+0000","last_peered":"2024-05-10T10:36:35.107175+0000","last_clean":"2024-05-10T10:36:35.107175+0000","last_became_active":"2024-05-10T10:35:50.934285+0000","last_became_peered":"2024-05-10T10:35:50.934285+0000","last_unstale":"2024-05-10T10:36:35.107175+0000","last_undegraded":"2024-05-10T10:36:35.107175+0000","last_fullsized":"2024-05-10T10:36:35.107175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:24:31.179005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141235+0000","last_change":"2024-05-10T10:36:02.018468+0000","last_active":"2024-05-10T10:36:35.141235+0000","last_peered":"2024-05-10T10:36:35.141235+0000","last_clean":"2024-05-10T10:36:35.141235+0000","last_became_active":"2024-05-10T10:36:02.018087+0000","last_became_peered":"2024-05-10T10:36:02.018087+0000","last_unstale":"2024-05-10T10:36:35.141235+0000","last_undegraded":"2024-05-10T10:36:35.141235+0000","last_fullsized":"2024-05-10T10:36:35.141235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:55.712012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.112756+0000","last_change":"2024-05-10T10:35:50.933660+0000","last_active":"2024-05-10T10:36:35.112756+0000","last_peered":"2024-05-10T10:36:35.112756+0000","last_clean":"2024-05-10T10:36:35.112756+0000","last_became_active":"2024-05-10T10:35:50.933230+0000","last_became_peered":"2024-05-10T10:35:50.933230+0000","last_unstale":"2024-05-10T10:36:35.112756+0000","last_undegraded":"2024-05-10T10:36:35.112756+0000","last_fullsized":"2024-05-10T10:36:35.112756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:34:54.272278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.437181+0000","last_change":"2024-05-10T10:35:47.910161+0000","last_active":"2024-05-10T10:37:23.437181+0000","last_peered":"2024-05-10T10:37:23.437181+0000","last_clean":"2024-05-10T10:37:23.437181+0000","last_became_active":"2024-05-10T10:35:47.909875+0000","last_became_peered":"2024-05-10T10:35:47.909875+0000","last_unstale":"2024-05-10T10:37:23.437181+0000","last_undegraded":"2024-05-10T10:37:23.437181+0000","last_fullsized":"2024-05-10T10:37:23.437181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:21.620921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141225+0000","last_change":"2024-05-10T10:36:01.986755+0000","last_active":"2024-05-10T10:36:35.141225+0000","last_peered":"2024-05-10T10:36:35.141225+0000","last_clean":"2024-05-10T10:36:35.141225+0000","last_became_active":"2024-05-10T10:36:01.986340+0000","last_became_peered":"2024-05-10T10:36:01.986340+0000","last_unstale":"2024-05-10T10:36:35.141225+0000","last_undegraded":"2024-05-10T10:36:35.141225+0000","last_fullsized":"2024-05-10T10:36:35.141225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:38.434323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.114648+0000","last_change":"2024-05-10T10:36:35.114218+0000","last_active":"2024-05-10T10:36:35.114648+0000","last_peered":"2024-05-10T10:36:35.114648+0000","last_clean":"2024-05-10T10:36:35.114648+0000","last_became_active":"2024-05-10T10:36:35.113876+0000","last_became_peered":"2024-05-10T10:36:35.113876+0000","last_unstale":"2024-05-10T10:36:35.114648+0000","last_undegraded":"2024-05-10T10:36:35.114648+0000","last_fullsized":"2024-05-10T10:36:35.114648+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:21.415678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:01.407362+0000","last_change":"2024-05-10T10:35:47.909161+0000","last_active":"2024-05-10T10:37:01.407362+0000","last_peered":"2024-05-10T10:37:01.407362+0000","last_clean":"2024-05-10T10:37:01.407362+0000","last_became_active":"2024-05-10T10:35:47.908546+0000","last_became_peered":"2024-05-10T10:35:47.908546+0000","last_unstale":"2024-05-10T10:37:01.407362+0000","last_undegraded":"2024-05-10T10:37:01.407362+0000","last_fullsized":"2024-05-10T10:37:01.407362+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:52.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141367+0000","last_change":"2024-05-10T10:34:20.980948+0000","last_active":"2024-05-10T10:36:35.141367+0000","last_peered":"2024-05-10T10:36:35.141367+0000","last_clean":"2024-05-10T10:36:35.141367+0000","last_became_active":"2024-05-10T10:34:20.975662+0000","last_became_peered":"2024-05-10T10:34:20.975662+0000","last_unstale":"2024-05-10T10:36:35.141367+0000","last_undegraded":"2024-05-10T10:36:35.141367+0000","last_fullsized":"2024-05-10T10:36:35.141367+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_clean_scrub_stamp":"2024-05-10T10:30:55.419211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:33.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111792+0000","last_change":"2024-05-10T10:36:02.018578+0000","last_active":"2024-05-10T10:36:35.111792+0000","last_peered":"2024-05-10T10:36:35.111792+0000","last_clean":"2024-05-10T10:36:35.111792+0000","last_became_active":"2024-05-10T10:36:02.018126+0000","last_became_peered":"2024-05-10T10:36:02.018126+0000","last_unstale":"2024-05-10T10:36:35.111792+0000","last_undegraded":"2024-05-10T10:36:35.111792+0000","last_fullsized":"2024-05-10T10:36:35.111792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:18:17.112432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.140858+0000","last_change":"2024-05-10T10:35:50.927472+0000","last_active":"2024-05-10T10:36:35.140858+0000","last_peered":"2024-05-10T10:36:35.140858+0000","last_clean":"2024-05-10T10:36:35.140858+0000","last_became_active":"2024-05-10T10:35:50.927030+0000","last_became_peered":"2024-05-10T10:35:50.927030+0000","last_unstale":"2024-05-10T10:36:35.140858+0000","last_undegraded":"2024-05-10T10:36:35.140858+0000","last_fullsized":"2024-05-10T10:36:35.140858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:15:03.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:58.384938+0000","last_change":"2024-05-10T10:35:47.912675+0000","last_active":"2024-05-10T10:36:58.384938+0000","last_peered":"2024-05-10T10:36:58.384938+0000","last_clean":"2024-05-10T10:36:58.384938+0000","last_became_active":"2024-05-10T10:35:47.912297+0000","last_became_peered":"2024-05-10T10:35:47.912297+0000","last_unstale":"2024-05-10T10:36:58.384938+0000","last_undegraded":"2024-05-10T10:36:58.384938+0000","last_fullsized":"2024-05-10T10:36:58.384938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:46.178221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104688+0000","last_change":"2024-05-10T10:36:01.981501+0000","last_active":"2024-05-10T10:36:35.104688+0000","last_peered":"2024-05-10T10:36:35.104688+0000","last_clean":"2024-05-10T10:36:35.104688+0000","last_became_active":"2024-05-10T10:36:01.981029+0000","last_became_peered":"2024-05-10T10:36:01.981029+0000","last_unstale":"2024-05-10T10:36:35.104688+0000","last_undegraded":"2024-05-10T10:36:35.104688+0000","last_fullsized":"2024-05-10T10:36:35.104688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:18.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402125+0000","last_change":"2024-05-10T10:35:47.910701+0000","last_active":"2024-05-10T10:37:00.402125+0000","last_peered":"2024-05-10T10:37:00.402125+0000","last_clean":"2024-05-10T10:37:00.402125+0000","last_became_active":"2024-05-10T10:35:47.910286+0000","last_became_peered":"2024-05-10T10:35:47.910286+0000","last_unstale":"2024-05-10T10:37:00.402125+0000","last_undegraded":"2024-05-10T10:37:00.402125+0000","last_fullsized":"2024-05-10T10:37:00.402125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:07.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141217+0000","last_change":"2024-05-10T10:35:50.926026+0000","last_active":"2024-05-10T10:36:35.141217+0000","last_peered":"2024-05-10T10:36:35.141217+0000","last_clean":"2024-05-10T10:36:35.141217+0000","last_became_active":"2024-05-10T10:35:50.925624+0000","last_became_peered":"2024-05-10T10:35:50.925624+0000","last_unstale":"2024-05-10T10:36:35.141217+0000","last_undegraded":"2024-05-10T10:36:35.141217+0000","last_fullsized":"2024-05-10T10:36:35.141217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:22:02.828681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107718+0000","last_change":"2024-05-10T10:36:02.018403+0000","last_active":"2024-05-10T10:36:35.107718+0000","last_peered":"2024-05-10T10:36:35.107718+0000","last_clean":"2024-05-10T10:36:35.107718+0000","last_became_active":"2024-05-10T10:36:02.017980+0000","last_became_peered":"2024-05-10T10:36:02.017980+0000","last_unstale":"2024-05-10T10:36:35.107718+0000","last_undegraded":"2024-05-10T10:36:35.107718+0000","last_fullsized":"2024-05-10T10:36:35.107718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:07.264730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.947914+0000","last_change":"2024-05-10T10:35:47.912737+0000","last_active":"2024-05-10T10:37:19.947914+0000","last_peered":"2024-05-10T10:37:19.947914+0000","last_clean":"2024-05-10T10:37:19.947914+0000","last_became_active":"2024-05-10T10:35:47.912300+0000","last_became_peered":"2024-05-10T10:35:47.912300+0000","last_unstale":"2024-05-10T10:37:19.947914+0000","last_undegraded":"2024-05-10T10:37:19.947914+0000","last_fullsized":"2024-05-10T10:37:19.947914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:31:57.606863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108518+0000","last_change":"2024-05-10T10:35:50.935817+0000","last_active":"2024-05-10T10:36:35.108518+0000","last_peered":"2024-05-10T10:36:35.108518+0000","last_clean":"2024-05-10T10:36:35.108518+0000","last_became_active":"2024-05-10T10:35:50.935515+0000","last_became_peered":"2024-05-10T10:35:50.935515+0000","last_unstale":"2024-05-10T10:36:35.108518+0000","last_undegraded":"2024-05-10T10:36:35.108518+0000","last_fullsized":"2024-05-10T10:36:35.108518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:49.042193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.106866+0000","last_change":"2024-05-10T10:36:01.986002+0000","last_active":"2024-05-10T10:36:35.106866+0000","last_peered":"2024-05-10T10:36:35.106866+0000","last_clean":"2024-05-10T10:36:35.106866+0000","last_became_active":"2024-05-10T10:36:01.985558+0000","last_became_peered":"2024-05-10T10:36:01.985558+0000","last_unstale":"2024-05-10T10:36:35.106866+0000","last_undegraded":"2024-05-10T10:36:35.106866+0000","last_fullsized":"2024-05-10T10:36:35.106866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:27.732316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402360+0000","last_change":"2024-05-10T10:35:47.909254+0000","last_active":"2024-05-10T10:37:00.402360+0000","last_peered":"2024-05-10T10:37:00.402360+0000","last_clean":"2024-05-10T10:37:00.402360+0000","last_became_active":"2024-05-10T10:35:47.908370+0000","last_became_peered":"2024-05-10T10:35:47.908370+0000","last_unstale":"2024-05-10T10:37:00.402360+0000","last_undegraded":"2024-05-10T10:37:00.402360+0000","last_fullsized":"2024-05-10T10:37:00.402360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:44.643592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108194+0000","last_change":"2024-05-10T10:35:50.933001+0000","last_active":"2024-05-10T10:36:35.108194+0000","last_peered":"2024-05-10T10:36:35.108194+0000","last_clean":"2024-05-10T10:36:35.108194+0000","last_became_active":"2024-05-10T10:35:50.932010+0000","last_became_peered":"2024-05-10T10:35:50.932010+0000","last_unstale":"2024-05-10T10:36:35.108194+0000","last_undegraded":"2024-05-10T10:36:35.108194+0000","last_fullsized":"2024-05-10T10:36:35.108194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:01.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142379+0000","last_change":"2024-05-10T10:36:02.018650+0000","last_active":"2024-05-10T10:36:35.142379+0000","last_peered":"2024-05-10T10:36:35.142379+0000","last_clean":"2024-05-10T10:36:35.142379+0000","last_became_active":"2024-05-10T10:36:02.018258+0000","last_became_peered":"2024-05-10T10:36:02.018258+0000","last_unstale":"2024-05-10T10:36:35.142379+0000","last_undegraded":"2024-05-10T10:36:35.142379+0000","last_fullsized":"2024-05-10T10:36:35.142379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:33.590752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:57.376072+0000","last_change":"2024-05-10T10:35:47.910419+0000","last_active":"2024-05-10T10:36:57.376072+0000","last_peered":"2024-05-10T10:36:57.376072+0000","last_clean":"2024-05-10T10:36:57.376072+0000","last_became_active":"2024-05-10T10:35:47.909930+0000","last_became_peered":"2024-05-10T10:35:47.909930+0000","last_unstale":"2024-05-10T10:36:57.376072+0000","last_undegraded":"2024-05-10T10:36:57.376072+0000","last_fullsized":"2024-05-10T10:36:57.376072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:32.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109956+0000","last_change":"2024-05-10T10:35:50.923207+0000","last_active":"2024-05-10T10:36:35.109956+0000","last_peered":"2024-05-10T10:36:35.109956+0000","last_clean":"2024-05-10T10:36:35.109956+0000","last_became_active":"2024-05-10T10:35:50.922866+0000","last_became_peered":"2024-05-10T10:35:50.922866+0000","last_unstale":"2024-05-10T10:36:35.109956+0000","last_undegraded":"2024-05-10T10:36:35.109956+0000","last_fullsized":"2024-05-10T10:36:35.109956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:48.096498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104984+0000","last_change":"2024-05-10T10:36:01.985399+0000","last_active":"2024-05-10T10:36:35.104984+0000","last_peered":"2024-05-10T10:36:35.104984+0000","last_clean":"2024-05-10T10:36:35.104984+0000","last_became_active":"2024-05-10T10:36:01.985016+0000","last_became_peered":"2024-05-10T10:36:01.985016+0000","last_unstale":"2024-05-10T10:36:35.104984+0000","last_undegraded":"2024-05-10T10:36:35.104984+0000","last_fullsized":"2024-05-10T10:36:35.104984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:12.151693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402150+0000","last_change":"2024-05-10T10:35:47.908594+0000","last_active":"2024-05-10T10:37:00.402150+0000","last_peered":"2024-05-10T10:37:00.402150+0000","last_clean":"2024-05-10T10:37:00.402150+0000","last_became_active":"2024-05-10T10:35:47.908278+0000","last_became_peered":"2024-05-10T10:35:47.908278+0000","last_unstale":"2024-05-10T10:37:00.402150+0000","last_undegraded":"2024-05-10T10:37:00.402150+0000","last_fullsized":"2024-05-10T10:37:00.402150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:24.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104603+0000","last_change":"2024-05-10T10:35:50.933089+0000","last_active":"2024-05-10T10:36:35.104603+0000","last_peered":"2024-05-10T10:36:35.104603+0000","last_clean":"2024-05-10T10:36:35.104603+0000","last_became_active":"2024-05-10T10:35:50.932739+0000","last_became_peered":"2024-05-10T10:35:50.932739+0000","last_unstale":"2024-05-10T10:36:35.104603+0000","last_undegraded":"2024-05-10T10:36:35.104603+0000","last_fullsized":"2024-05-10T10:36:35.104603+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:15.302832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142702+0000","last_change":"2024-05-10T10:36:01.988561+0000","last_active":"2024-05-10T10:36:35.142702+0000","last_peered":"2024-05-10T10:36:35.142702+0000","last_clean":"2024-05-10T10:36:35.142702+0000","last_became_active":"2024-05-10T10:36:01.988120+0000","last_became_peered":"2024-05-10T10:36:01.988120+0000","last_unstale":"2024-05-10T10:36:35.142702+0000","last_undegraded":"2024-05-10T10:36:35.142702+0000","last_fullsized":"2024-05-10T10:36:35.142702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:35:37.120469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:59.390590+0000","last_change":"2024-05-10T10:35:47.915506+0000","last_active":"2024-05-10T10:36:59.390590+0000","last_peered":"2024-05-10T10:36:59.390590+0000","last_clean":"2024-05-10T10:36:59.390590+0000","last_became_active":"2024-05-10T10:35:47.915141+0000","last_became_peered":"2024-05-10T10:35:47.915141+0000","last_unstale":"2024-05-10T10:36:59.390590+0000","last_undegraded":"2024-05-10T10:36:59.390590+0000","last_fullsized":"2024-05-10T10:36:59.390590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:07:34.454646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108644+0000","last_change":"2024-05-10T10:35:50.938466+0000","last_active":"2024-05-10T10:36:35.108644+0000","last_peered":"2024-05-10T10:36:35.108644+0000","last_clean":"2024-05-10T10:36:35.108644+0000","last_became_active":"2024-05-10T10:35:50.938167+0000","last_became_peered":"2024-05-10T10:35:50.938167+0000","last_unstale":"2024-05-10T10:36:35.108644+0000","last_undegraded":"2024-05-10T10:36:35.108644+0000","last_fullsized":"2024-05-10T10:36:35.108644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:18.014708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142690+0000","last_change":"2024-05-10T10:36:02.018661+0000","last_active":"2024-05-10T10:36:35.142690+0000","last_peered":"2024-05-10T10:36:35.142690+0000","last_clean":"2024-05-10T10:36:35.142690+0000","last_became_active":"2024-05-10T10:36:02.018196+0000","last_became_peered":"2024-05-10T10:36:02.018196+0000","last_unstale":"2024-05-10T10:36:35.142690+0000","last_undegraded":"2024-05-10T10:36:35.142690+0000","last_fullsized":"2024-05-10T10:36:35.142690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:11.605012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:57.378752+0000","last_change":"2024-05-10T10:35:47.917009+0000","last_active":"2024-05-10T10:36:57.378752+0000","last_peered":"2024-05-10T10:36:57.378752+0000","last_clean":"2024-05-10T10:36:57.378752+0000","last_became_active":"2024-05-10T10:35:47.916536+0000","last_became_peered":"2024-05-10T10:35:47.916536+0000","last_unstale":"2024-05-10T10:36:57.378752+0000","last_undegraded":"2024-05-10T10:36:57.378752+0000","last_fullsized":"2024-05-10T10:36:57.378752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:17.783027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141091+0000","last_change":"2024-05-10T10:35:50.926173+0000","last_active":"2024-05-10T10:36:35.141091+0000","last_peered":"2024-05-10T10:36:35.141091+0000","last_clean":"2024-05-10T10:36:35.141091+0000","last_became_active":"2024-05-10T10:35:50.925846+0000","last_became_peered":"2024-05-10T10:35:50.925846+0000","last_unstale":"2024-05-10T10:36:35.141091+0000","last_undegraded":"2024-05-10T10:36:35.141091+0000","last_fullsized":"2024-05-10T10:36:35.141091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:29:02.587667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109908+0000","last_change":"2024-05-10T10:36:01.981312+0000","last_active":"2024-05-10T10:36:35.109908+0000","last_peered":"2024-05-10T10:36:35.109908+0000","last_clean":"2024-05-10T10:36:35.109908+0000","last_became_active":"2024-05-10T10:36:01.980861+0000","last_became_peered":"2024-05-10T10:36:01.980861+0000","last_unstale":"2024-05-10T10:36:35.109908+0000","last_undegraded":"2024-05-10T10:36:35.109908+0000","last_fullsized":"2024-05-10T10:36:35.109908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:29.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:24.439899+0000","last_change":"2024-05-10T10:35:47.911120+0000","last_active":"2024-05-10T10:37:24.439899+0000","last_peered":"2024-05-10T10:37:24.439899+0000","last_clean":"2024-05-10T10:37:24.439899+0000","last_became_active":"2024-05-10T10:35:47.909945+0000","last_became_peered":"2024-05-10T10:35:47.909945+0000","last_unstale":"2024-05-10T10:37:24.439899+0000","last_undegraded":"2024-05-10T10:37:24.439899+0000","last_fullsized":"2024-05-10T10:37:24.439899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:40.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141549+0000","last_change":"2024-05-10T10:35:50.933855+0000","last_active":"2024-05-10T10:36:35.141549+0000","last_peered":"2024-05-10T10:36:35.141549+0000","last_clean":"2024-05-10T10:36:35.141549+0000","last_became_active":"2024-05-10T10:35:50.933502+0000","last_became_peered":"2024-05-10T10:35:50.933502+0000","last_unstale":"2024-05-10T10:36:35.141549+0000","last_undegraded":"2024-05-10T10:36:35.141549+0000","last_fullsized":"2024-05-10T10:36:35.141549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:36.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107967+0000","last_change":"2024-05-10T10:36:01.985133+0000","last_active":"2024-05-10T10:36:35.107967+0000","last_peered":"2024-05-10T10:36:35.107967+0000","last_clean":"2024-05-10T10:36:35.107967+0000","last_became_active":"2024-05-10T10:36:01.984758+0000","last_became_peered":"2024-05-10T10:36:01.984758+0000","last_unstale":"2024-05-10T10:36:35.107967+0000","last_undegraded":"2024-05-10T10:36:35.107967+0000","last_fullsized":"2024-05-10T10:36:35.107967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:52.733063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402504+0000","last_change":"2024-05-10T10:35:47.908645+0000","last_active":"2024-05-10T10:37:00.402504+0000","last_peered":"2024-05-10T10:37:00.402504+0000","last_clean":"2024-05-10T10:37:00.402504+0000","last_became_active":"2024-05-10T10:35:47.908123+0000","last_became_peered":"2024-05-10T10:35:47.908123+0000","last_unstale":"2024-05-10T10:37:00.402504+0000","last_undegraded":"2024-05-10T10:37:00.402504+0000","last_fullsized":"2024-05-10T10:37:00.402504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:46.181724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141170+0000","last_change":"2024-05-10T10:35:50.929215+0000","last_active":"2024-05-10T10:36:35.141170+0000","last_peered":"2024-05-10T10:36:35.141170+0000","last_clean":"2024-05-10T10:36:35.141170+0000","last_became_active":"2024-05-10T10:35:50.928907+0000","last_became_peered":"2024-05-10T10:35:50.928907+0000","last_unstale":"2024-05-10T10:36:35.141170+0000","last_undegraded":"2024-05-10T10:36:35.141170+0000","last_fullsized":"2024-05-10T10:36:35.141170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:57:17.340687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110495+0000","last_change":"2024-05-10T10:36:01.988548+0000","last_active":"2024-05-10T10:36:35.110495+0000","last_peered":"2024-05-10T10:36:35.110495+0000","last_clean":"2024-05-10T10:36:35.110495+0000","last_became_active":"2024-05-10T10:36:01.987221+0000","last_became_peered":"2024-05-10T10:36:01.987221+0000","last_unstale":"2024-05-10T10:36:35.110495+0000","last_undegraded":"2024-05-10T10:36:35.110495+0000","last_fullsized":"2024-05-10T10:36:35.110495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:55:22.434536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.400132+0000","last_change":"2024-05-10T10:35:47.914022+0000","last_active":"2024-05-10T10:37:00.400132+0000","last_peered":"2024-05-10T10:37:00.400132+0000","last_clean":"2024-05-10T10:37:00.400132+0000","last_became_active":"2024-05-10T10:35:47.913711+0000","last_became_peered":"2024-05-10T10:35:47.913711+0000","last_unstale":"2024-05-10T10:37:00.400132+0000","last_undegraded":"2024-05-10T10:37:00.400132+0000","last_fullsized":"2024-05-10T10:37:00.400132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:13.291307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141777+0000","last_change":"2024-05-10T10:35:50.927452+0000","last_active":"2024-05-10T10:36:35.141777+0000","last_peered":"2024-05-10T10:36:35.141777+0000","last_clean":"2024-05-10T10:36:35.141777+0000","last_became_active":"2024-05-10T10:35:50.927102+0000","last_became_peered":"2024-05-10T10:35:50.927102+0000","last_unstale":"2024-05-10T10:36:35.141777+0000","last_undegraded":"2024-05-10T10:36:35.141777+0000","last_fullsized":"2024-05-10T10:36:35.141777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:07.704196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104723+0000","last_change":"2024-05-10T10:35:50.933072+0000","last_active":"2024-05-10T10:36:35.104723+0000","last_peered":"2024-05-10T10:36:35.104723+0000","last_clean":"2024-05-10T10:36:35.104723+0000","last_became_active":"2024-05-10T10:35:50.932724+0000","last_became_peered":"2024-05-10T10:35:50.932724+0000","last_unstale":"2024-05-10T10:36:35.104723+0000","last_undegraded":"2024-05-10T10:36:35.104723+0000","last_fullsized":"2024-05-10T10:36:35.104723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:08.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107435+0000","last_change":"2024-05-10T10:35:50.931635+0000","last_active":"2024-05-10T10:36:35.107435+0000","last_peered":"2024-05-10T10:36:35.107435+0000","last_clean":"2024-05-10T10:36:35.107435+0000","last_became_active":"2024-05-10T10:35:50.930158+0000","last_became_peered":"2024-05-10T10:35:50.930158+0000","last_unstale":"2024-05-10T10:36:35.107435+0000","last_undegraded":"2024-05-10T10:36:35.107435+0000","last_fullsized":"2024-05-10T10:36:35.107435+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:22.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107239+0000","last_change":"2024-05-10T10:35:50.936947+0000","last_active":"2024-05-10T10:36:35.107239+0000","last_peered":"2024-05-10T10:36:35.107239+0000","last_clean":"2024-05-10T10:36:35.107239+0000","last_became_active":"2024-05-10T10:35:50.936156+0000","last_became_peered":"2024-05-10T10:35:50.936156+0000","last_unstale":"2024-05-10T10:36:35.107239+0000","last_undegraded":"2024-05-10T10:36:35.107239+0000","last_fullsized":"2024-05-10T10:36:35.107239+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:21.167787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108354+0000","last_change":"2024-05-10T10:35:50.935384+0000","last_active":"2024-05-10T10:36:35.108354+0000","last_peered":"2024-05-10T10:36:35.108354+0000","last_clean":"2024-05-10T10:36:35.108354+0000","last_became_active":"2024-05-10T10:35:50.932555+0000","last_became_peered":"2024-05-10T10:35:50.932555+0000","last_unstale":"2024-05-10T10:36:35.108354+0000","last_undegraded":"2024-05-10T10:36:35.108354+0000","last_fullsized":"2024-05-10T10:36:35.108354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:09:11.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141236+0000","last_change":"2024-05-10T10:35:50.929322+0000","last_active":"2024-05-10T10:36:35.141236+0000","last_peered":"2024-05-10T10:36:35.141236+0000","last_clean":"2024-05-10T10:36:35.141236+0000","last_became_active":"2024-05-10T10:35:50.928366+0000","last_became_peered":"2024-05-10T10:35:50.928366+0000","last_unstale":"2024-05-10T10:36:35.141236+0000","last_undegraded":"2024-05-10T10:36:35.141236+0000","last_fullsized":"2024-05-10T10:36:35.141236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:32:28.337962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110922+0000","last_change":"2024-05-10T10:36:35.110922+0000","last_active":"2024-05-10T10:36:35.110922+0000","last_peered":"2024-05-10T10:36:35.110922+0000","last_clean":"2024-05-10T10:36:35.110922+0000","last_became_active":"2024-05-10T10:36:35.110525+0000","last_became_peered":"2024-05-10T10:36:35.110525+0000","last_unstale":"2024-05-10T10:36:35.110922+0000","last_undegraded":"2024-05-10T10:36:35.110922+0000","last_fullsized":"2024-05-10T10:36:35.110922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:47.825168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107795+0000","last_change":"2024-05-10T10:35:50.927869+0000","last_active":"2024-05-10T10:36:35.107795+0000","last_peered":"2024-05-10T10:36:35.107795+0000","last_clean":"2024-05-10T10:36:35.107795+0000","last_became_active":"2024-05-10T10:35:50.927552+0000","last_became_peered":"2024-05-10T10:35:50.927552+0000","last_unstale":"2024-05-10T10:36:35.107795+0000","last_undegraded":"2024-05-10T10:36:35.107795+0000","last_fullsized":"2024-05-10T10:36:35.107795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:36.962316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108617+0000","last_change":"2024-05-10T10:35:50.933099+0000","last_active":"2024-05-10T10:36:35.108617+0000","last_peered":"2024-05-10T10:36:35.108617+0000","last_clean":"2024-05-10T10:36:35.108617+0000","last_became_active":"2024-05-10T10:35:50.932765+0000","last_became_peered":"2024-05-10T10:35:50.932765+0000","last_unstale":"2024-05-10T10:36:35.108617+0000","last_undegraded":"2024-05-10T10:36:35.108617+0000","last_fullsized":"2024-05-10T10:36:35.108617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:19.095217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111181+0000","last_change":"2024-05-10T10:35:50.937704+0000","last_active":"2024-05-10T10:36:35.111181+0000","last_peered":"2024-05-10T10:36:35.111181+0000","last_clean":"2024-05-10T10:36:35.111181+0000","last_became_active":"2024-05-10T10:35:50.937236+0000","last_became_peered":"2024-05-10T10:35:50.937236+0000","last_unstale":"2024-05-10T10:36:35.111181+0000","last_undegraded":"2024-05-10T10:36:35.111181+0000","last_fullsized":"2024-05-10T10:36:35.111181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:46.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108911+0000","last_change":"2024-05-10T10:35:50.935181+0000","last_active":"2024-05-10T10:36:35.108911+0000","last_peered":"2024-05-10T10:36:35.108911+0000","last_clean":"2024-05-10T10:36:35.108911+0000","last_became_active":"2024-05-10T10:35:50.933446+0000","last_became_peered":"2024-05-10T10:35:50.933446+0000","last_unstale":"2024-05-10T10:36:35.108911+0000","last_undegraded":"2024-05-10T10:36:35.108911+0000","last_fullsized":"2024-05-10T10:36:35.108911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:03.972410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110037+0000","last_change":"2024-05-10T10:35:50.931797+0000","last_active":"2024-05-10T10:36:35.110037+0000","last_peered":"2024-05-10T10:36:35.110037+0000","last_clean":"2024-05-10T10:36:35.110037+0000","last_became_active":"2024-05-10T10:35:50.931494+0000","last_became_peered":"2024-05-10T10:35:50.931494+0000","last_unstale":"2024-05-10T10:36:35.110037+0000","last_undegraded":"2024-05-10T10:36:35.110037+0000","last_fullsized":"2024-05-10T10:36:35.110037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:30.797462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141018+0000","last_change":"2024-05-10T10:35:50.931688+0000","last_active":"2024-05-10T10:36:35.141018+0000","last_peered":"2024-05-10T10:36:35.141018+0000","last_clean":"2024-05-10T10:36:35.141018+0000","last_became_active":"2024-05-10T10:35:50.931362+0000","last_became_peered":"2024-05-10T10:35:50.931362+0000","last_unstale":"2024-05-10T10:36:35.141018+0000","last_undegraded":"2024-05-10T10:36:35.141018+0000","last_fullsized":"2024-05-10T10:36:35.141018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:24.263752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104961+0000","last_change":"2024-05-10T10:35:50.930905+0000","last_active":"2024-05-10T10:36:35.104961+0000","last_peered":"2024-05-10T10:36:35.104961+0000","last_clean":"2024-05-10T10:36:35.104961+0000","last_became_active":"2024-05-10T10:35:50.930448+0000","last_became_peered":"2024-05-10T10:35:50.930448+0000","last_unstale":"2024-05-10T10:36:35.104961+0000","last_undegraded":"2024-05-10T10:36:35.104961+0000","last_fullsized":"2024-05-10T10:36:35.104961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:52:24.468600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63482,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":274,"num_read_kb":205,"num_write":240,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":207872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678051,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":502216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874279,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1699840,"data_stored":1146092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037804,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36928,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708224,"statfs":{"total":95995035648,"available":95957221376,"internally_reserved":0,"allocated":1630208,"data_stored":1099021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":497650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":564062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":499106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":497358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":558784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1094212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:37:28.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T10:37:29.075 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:30.697 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:30 smithi112 ceph-mon[30100]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-10T10:37:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:30 smithi148 ceph-mon[33568]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-10T10:37:30.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:30 smithi195 ceph-mon[33773]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.953+0000 7efce7298700 1 -- 172.21.15.112:0/48380687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 msgr2=0x7efce0102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.953+0000 7efce7298700 1 --2- 172.21.15.112:0/48380687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce0102a70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0099c0 tx=0x7efcdc009cd0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.953+0000 7efce7298700 1 -- 172.21.15.112:0/48380687 shutdown_connections 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.953+0000 7efce7298700 1 --2- 172.21.15.112:0/48380687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efce01076e0 0x7efce010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.953+0000 7efce7298700 1 --2- 172.21.15.112:0/48380687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce0102a70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.953+0000 7efce7298700 1 --2- 172.21.15.112:0/48380687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efce0103040 0x7efce0107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.953+0000 7efce7298700 1 -- 172.21.15.112:0/48380687 >> 172.21.15.112:0/48380687 conn(0x7efce00783f0 msgr2=0x7efce0078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.954+0000 7efce7298700 1 -- 172.21.15.112:0/48380687 shutdown_connections 2024-05-10T10:37:30.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.954+0000 7efce7298700 1 -- 172.21.15.112:0/48380687 wait complete. 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.954+0000 7efce7298700 1 Processor -- start 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.954+0000 7efce7298700 1 -- start start 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efce0103040 0x7efce019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efce01076e0 0x7efce0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efce0112420 con 0x7efce0102690 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efce01122a0 con 0x7efce01076e0 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efce01125a0 con 0x7efce0103040 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34794/0 (socket says 172.21.15.112:34794) 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 -- 172.21.15.112:0/1788954239 learned_addr learned my addr 172.21.15.112:0/1788954239 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce4833700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efce0103040 0x7efce019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5835700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efce01076e0 0x7efce0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 -- 172.21.15.112:0/1788954239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efce0103040 msgr2=0x7efce019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efce0103040 0x7efce019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 -- 172.21.15.112:0/1788954239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efce01076e0 msgr2=0x7efce0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efce01076e0 0x7efce0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 -- 172.21.15.112:0/1788954239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce019a000 con 0x7efce0102690 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce5034700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce019f6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efcdc000c00 tx=0x7efcdc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efcd67fc700 1 -- 172.21.15.112:0/1788954239 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc026070 con 0x7efce0102690 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce019a290 con 0x7efce0102690 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a6500 con 0x7efce0102690 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efcd67fc700 1 -- 172.21.15.112:0/1788954239 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcdc0054c0 con 0x7efce0102690 2024-05-10T10:37:30.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.955+0000 7efcd67fc700 1 -- 172.21.15.112:0/1788954239 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc004c20 con 0x7efce0102690 2024-05-10T10:37:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.956+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc4005330 con 0x7efce0102690 2024-05-10T10:37:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.958+0000 7efcd67fc700 1 -- 172.21.15.112:0/1788954239 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7efcdc029070 con 0x7efce0102690 2024-05-10T10:37:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.958+0000 7efcd67fc700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efccc0b2990 0x7efccc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.958+0000 7efce4833700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efccc0b2990 0x7efccc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.958+0000 7efcd67fc700 1 -- 172.21.15.112:0/1788954239 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7efcdc0e3460 con 0x7efce0102690 2024-05-10T10:37:30.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.959+0000 7efce4833700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efccc0b2990 0x7efccc0b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efcd0005ea0 tx=0x7efcd0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:30.962+0000 7efcd67fc700 1 -- 172.21.15.112:0/1788954239 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efcdc030020 con 0x7efce0102690 2024-05-10T10:37:31.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.167+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efcc4000bc0 con 0x7efccc0b2990 2024-05-10T10:37:31.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.176+0000 7efcd67fc700 1 -- 172.21.15.112:0/1788954239 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340401 (secure 0 0 0) 0x7efcc4000bc0 con 0x7efccc0b2990 2024-05-10T10:37:31.181 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:37:31.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efccc0b2990 msgr2=0x7efccc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efccc0b2990 0x7efccc0b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efcd0005ea0 tx=0x7efcd0005e30 comp rx=0 tx=0).stop 2024-05-10T10:37:31.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 msgr2=0x7efce019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce019f6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efcdc000c00 tx=0x7efcdc00f740 comp rx=0 tx=0).stop 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 shutdown_connections 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efce01076e0 0x7efce0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efce0102690 0x7efce019f6a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efccc0b2990 0x7efccc0b4e50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 --2- 172.21.15.112:0/1788954239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efce0103040 0x7efce019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 >> 172.21.15.112:0/1788954239 conn(0x7efce00783f0 msgr2=0x7efce0106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 shutdown_connections 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:31.180+0000 7efce7298700 1 -- 172.21.15.112:0/1788954239 wait complete. 2024-05-10T10:37:31.185 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-10T10:37:32.154 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-05-10T10:37:29.289785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662797,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":376,"num_read_kb":298,"num_write":372,"num_write_kb":2295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2955541,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367304,"kb_used_data":13992,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574520,"statfs":{"total":1151940427776,"available":1151564308480,"internally_reserved":0,"allocated":14327808,"data_stored":7974880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36292,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":129,"num_read_kb":107,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003917"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105537+0000","last_change":"2024-05-10T10:35:50.924449+0000","last_active":"2024-05-10T10:36:35.105537+0000","last_peered":"2024-05-10T10:36:35.105537+0000","last_clean":"2024-05-10T10:36:35.105537+0000","last_became_active":"2024-05-10T10:35:50.924114+0000","last_became_peered":"2024-05-10T10:35:50.924114+0000","last_unstale":"2024-05-10T10:36:35.105537+0000","last_undegraded":"2024-05-10T10:36:35.105537+0000","last_fullsized":"2024-05-10T10:36:35.105537+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:50.636501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108327+0000","last_change":"2024-05-10T10:35:50.928715+0000","last_active":"2024-05-10T10:36:35.108327+0000","last_peered":"2024-05-10T10:36:35.108327+0000","last_clean":"2024-05-10T10:36:35.108327+0000","last_became_active":"2024-05-10T10:35:50.928404+0000","last_became_peered":"2024-05-10T10:35:50.928404+0000","last_unstale":"2024-05-10T10:36:35.108327+0000","last_undegraded":"2024-05-10T10:36:35.108327+0000","last_fullsized":"2024-05-10T10:36:35.108327+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:44.934022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107892+0000","last_change":"2024-05-10T10:35:50.925134+0000","last_active":"2024-05-10T10:36:35.107892+0000","last_peered":"2024-05-10T10:36:35.107892+0000","last_clean":"2024-05-10T10:36:35.107892+0000","last_became_active":"2024-05-10T10:35:50.924797+0000","last_became_peered":"2024-05-10T10:35:50.924797+0000","last_unstale":"2024-05-10T10:36:35.107892+0000","last_undegraded":"2024-05-10T10:36:35.107892+0000","last_fullsized":"2024-05-10T10:36:35.107892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:43:52.666519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142241+0000","last_change":"2024-05-10T10:35:50.930916+0000","last_active":"2024-05-10T10:36:35.142241+0000","last_peered":"2024-05-10T10:36:35.142241+0000","last_clean":"2024-05-10T10:36:35.142241+0000","last_became_active":"2024-05-10T10:35:50.930378+0000","last_became_peered":"2024-05-10T10:35:50.930378+0000","last_unstale":"2024-05-10T10:36:35.142241+0000","last_undegraded":"2024-05-10T10:36:35.142241+0000","last_fullsized":"2024-05-10T10:36:35.142241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:07.165595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108632+0000","last_change":"2024-05-10T10:35:50.925459+0000","last_active":"2024-05-10T10:36:35.108632+0000","last_peered":"2024-05-10T10:36:35.108632+0000","last_clean":"2024-05-10T10:36:35.108632+0000","last_became_active":"2024-05-10T10:35:50.925061+0000","last_became_peered":"2024-05-10T10:35:50.925061+0000","last_unstale":"2024-05-10T10:36:35.108632+0000","last_undegraded":"2024-05-10T10:36:35.108632+0000","last_fullsized":"2024-05-10T10:36:35.108632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:48:06.622369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107623+0000","last_change":"2024-05-10T10:35:50.932052+0000","last_active":"2024-05-10T10:36:35.107623+0000","last_peered":"2024-05-10T10:36:35.107623+0000","last_clean":"2024-05-10T10:36:35.107623+0000","last_became_active":"2024-05-10T10:35:50.931685+0000","last_became_peered":"2024-05-10T10:35:50.931685+0000","last_unstale":"2024-05-10T10:36:35.107623+0000","last_undegraded":"2024-05-10T10:36:35.107623+0000","last_fullsized":"2024-05-10T10:36:35.107623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:13:39.930917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141143+0000","last_change":"2024-05-10T10:35:50.930620+0000","last_active":"2024-05-10T10:36:35.141143+0000","last_peered":"2024-05-10T10:36:35.141143+0000","last_clean":"2024-05-10T10:36:35.141143+0000","last_became_active":"2024-05-10T10:35:50.929696+0000","last_became_peered":"2024-05-10T10:35:50.929696+0000","last_unstale":"2024-05-10T10:36:35.141143+0000","last_undegraded":"2024-05-10T10:36:35.141143+0000","last_fullsized":"2024-05-10T10:36:35.141143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:23.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107834+0000","last_change":"2024-05-10T10:35:50.937517+0000","last_active":"2024-05-10T10:36:35.107834+0000","last_peered":"2024-05-10T10:36:35.107834+0000","last_clean":"2024-05-10T10:36:35.107834+0000","last_became_active":"2024-05-10T10:35:50.937180+0000","last_became_peered":"2024-05-10T10:35:50.937180+0000","last_unstale":"2024-05-10T10:36:35.107834+0000","last_undegraded":"2024-05-10T10:36:35.107834+0000","last_fullsized":"2024-05-10T10:36:35.107834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.568509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110387+0000","last_change":"2024-05-10T10:35:50.937825+0000","last_active":"2024-05-10T10:36:35.110387+0000","last_peered":"2024-05-10T10:36:35.110387+0000","last_clean":"2024-05-10T10:36:35.110387+0000","last_became_active":"2024-05-10T10:35:50.935650+0000","last_became_peered":"2024-05-10T10:35:50.935650+0000","last_unstale":"2024-05-10T10:36:35.110387+0000","last_undegraded":"2024-05-10T10:36:35.110387+0000","last_fullsized":"2024-05-10T10:36:35.110387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:43:55.780266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107567+0000","last_change":"2024-05-10T10:35:50.933598+0000","last_active":"2024-05-10T10:36:35.107567+0000","last_peered":"2024-05-10T10:36:35.107567+0000","last_clean":"2024-05-10T10:36:35.107567+0000","last_became_active":"2024-05-10T10:35:50.932767+0000","last_became_peered":"2024-05-10T10:35:50.932767+0000","last_unstale":"2024-05-10T10:36:35.107567+0000","last_undegraded":"2024-05-10T10:36:35.107567+0000","last_fullsized":"2024-05-10T10:36:35.107567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:40.610376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141292+0000","last_change":"2024-05-10T10:35:50.934882+0000","last_active":"2024-05-10T10:36:35.141292+0000","last_peered":"2024-05-10T10:36:35.141292+0000","last_clean":"2024-05-10T10:36:35.141292+0000","last_became_active":"2024-05-10T10:35:50.934528+0000","last_became_peered":"2024-05-10T10:35:50.934528+0000","last_unstale":"2024-05-10T10:36:35.141292+0000","last_undegraded":"2024-05-10T10:36:35.141292+0000","last_fullsized":"2024-05-10T10:36:35.141292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:15.319765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111545+0000","last_change":"2024-05-10T10:36:35.110939+0000","last_active":"2024-05-10T10:36:35.111545+0000","last_peered":"2024-05-10T10:36:35.111545+0000","last_clean":"2024-05-10T10:36:35.111545+0000","last_became_active":"2024-05-10T10:36:35.110600+0000","last_became_peered":"2024-05-10T10:36:35.110600+0000","last_unstale":"2024-05-10T10:36:35.111545+0000","last_undegraded":"2024-05-10T10:36:35.111545+0000","last_fullsized":"2024-05-10T10:36:35.111545+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:45:58.547301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141104+0000","last_change":"2024-05-10T10:35:50.934161+0000","last_active":"2024-05-10T10:36:35.141104+0000","last_peered":"2024-05-10T10:36:35.141104+0000","last_clean":"2024-05-10T10:36:35.141104+0000","last_became_active":"2024-05-10T10:35:50.933831+0000","last_became_peered":"2024-05-10T10:35:50.933831+0000","last_unstale":"2024-05-10T10:36:35.141104+0000","last_undegraded":"2024-05-10T10:36:35.141104+0000","last_fullsized":"2024-05-10T10:36:35.141104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:09.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107892+0000","last_change":"2024-05-10T10:35:50.927404+0000","last_active":"2024-05-10T10:36:35.107892+0000","last_peered":"2024-05-10T10:36:35.107892+0000","last_clean":"2024-05-10T10:36:35.107892+0000","last_became_active":"2024-05-10T10:35:50.927094+0000","last_became_peered":"2024-05-10T10:35:50.927094+0000","last_unstale":"2024-05-10T10:36:35.107892+0000","last_undegraded":"2024-05-10T10:36:35.107892+0000","last_fullsized":"2024-05-10T10:36:35.107892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:25.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142063+0000","last_change":"2024-05-10T10:36:35.142063+0000","last_active":"2024-05-10T10:36:35.142063+0000","last_peered":"2024-05-10T10:36:35.142063+0000","last_clean":"2024-05-10T10:36:35.142063+0000","last_became_active":"2024-05-10T10:36:35.141685+0000","last_became_peered":"2024-05-10T10:36:35.141685+0000","last_unstale":"2024-05-10T10:36:35.142063+0000","last_undegraded":"2024-05-10T10:36:35.142063+0000","last_fullsized":"2024-05-10T10:36:35.142063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:47.284288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109527+0000","last_change":"2024-05-10T10:35:50.933172+0000","last_active":"2024-05-10T10:36:35.109527+0000","last_peered":"2024-05-10T10:36:35.109527+0000","last_clean":"2024-05-10T10:36:35.109527+0000","last_became_active":"2024-05-10T10:35:50.931864+0000","last_became_peered":"2024-05-10T10:35:50.931864+0000","last_unstale":"2024-05-10T10:36:35.109527+0000","last_undegraded":"2024-05-10T10:36:35.109527+0000","last_fullsized":"2024-05-10T10:36:35.109527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:50.620315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105374+0000","last_change":"2024-05-10T10:35:50.925952+0000","last_active":"2024-05-10T10:36:35.105374+0000","last_peered":"2024-05-10T10:36:35.105374+0000","last_clean":"2024-05-10T10:36:35.105374+0000","last_became_active":"2024-05-10T10:35:50.925610+0000","last_became_peered":"2024-05-10T10:35:50.925610+0000","last_unstale":"2024-05-10T10:36:35.105374+0000","last_undegraded":"2024-05-10T10:36:35.105374+0000","last_fullsized":"2024-05-10T10:36:35.105374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:38.287235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105152+0000","last_change":"2024-05-10T10:35:50.929379+0000","last_active":"2024-05-10T10:36:35.105152+0000","last_peered":"2024-05-10T10:36:35.105152+0000","last_clean":"2024-05-10T10:36:35.105152+0000","last_became_active":"2024-05-10T10:35:50.929082+0000","last_became_peered":"2024-05-10T10:35:50.929082+0000","last_unstale":"2024-05-10T10:36:35.105152+0000","last_undegraded":"2024-05-10T10:36:35.105152+0000","last_fullsized":"2024-05-10T10:36:35.105152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:54.097855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108118+0000","last_change":"2024-05-10T10:35:50.937754+0000","last_active":"2024-05-10T10:36:35.108118+0000","last_peered":"2024-05-10T10:36:35.108118+0000","last_clean":"2024-05-10T10:36:35.108118+0000","last_became_active":"2024-05-10T10:35:50.935662+0000","last_became_peered":"2024-05-10T10:35:50.935662+0000","last_unstale":"2024-05-10T10:36:35.108118+0000","last_undegraded":"2024-05-10T10:36:35.108118+0000","last_fullsized":"2024-05-10T10:36:35.108118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:52.692279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107234+0000","last_change":"2024-05-10T10:36:01.989463+0000","last_active":"2024-05-10T10:36:35.107234+0000","last_peered":"2024-05-10T10:36:35.107234+0000","last_clean":"2024-05-10T10:36:35.107234+0000","last_became_active":"2024-05-10T10:36:01.988949+0000","last_became_peered":"2024-05-10T10:36:01.988949+0000","last_unstale":"2024-05-10T10:36:35.107234+0000","last_undegraded":"2024-05-10T10:36:35.107234+0000","last_fullsized":"2024-05-10T10:36:35.107234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:47.529845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":81,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:26.441164+0000","last_change":"2024-05-10T10:35:47.909033+0000","last_active":"2024-05-10T10:37:26.441164+0000","last_peered":"2024-05-10T10:37:26.441164+0000","last_clean":"2024-05-10T10:37:26.441164+0000","last_became_active":"2024-05-10T10:35:47.908680+0000","last_became_peered":"2024-05-10T10:35:47.908680+0000","last_unstale":"2024-05-10T10:37:26.441164+0000","last_undegraded":"2024-05-10T10:37:26.441164+0000","last_fullsized":"2024-05-10T10:37:26.441164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:46.642113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108365+0000","last_change":"2024-05-10T10:35:50.939233+0000","last_active":"2024-05-10T10:36:35.108365+0000","last_peered":"2024-05-10T10:36:35.108365+0000","last_clean":"2024-05-10T10:36:35.108365+0000","last_became_active":"2024-05-10T10:35:50.938949+0000","last_became_peered":"2024-05-10T10:35:50.938949+0000","last_unstale":"2024-05-10T10:36:35.108365+0000","last_undegraded":"2024-05-10T10:36:35.108365+0000","last_fullsized":"2024-05-10T10:36:35.108365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:06.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141798+0000","last_change":"2024-05-10T10:36:01.990982+0000","last_active":"2024-05-10T10:36:35.141798+0000","last_peered":"2024-05-10T10:36:35.141798+0000","last_clean":"2024-05-10T10:36:35.141798+0000","last_became_active":"2024-05-10T10:36:01.990325+0000","last_became_peered":"2024-05-10T10:36:01.990325+0000","last_unstale":"2024-05-10T10:36:35.141798+0000","last_undegraded":"2024-05-10T10:36:35.141798+0000","last_fullsized":"2024-05-10T10:36:35.141798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:18.424130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108588+0000","last_change":"2024-05-10T10:35:47.911239+0000","last_active":"2024-05-10T10:36:35.108588+0000","last_peered":"2024-05-10T10:36:35.108588+0000","last_clean":"2024-05-10T10:36:35.108588+0000","last_became_active":"2024-05-10T10:35:47.910945+0000","last_became_peered":"2024-05-10T10:35:47.910945+0000","last_unstale":"2024-05-10T10:36:35.108588+0000","last_undegraded":"2024-05-10T10:36:35.108588+0000","last_fullsized":"2024-05-10T10:36:35.108588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:08:52.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108444+0000","last_change":"2024-05-10T10:35:50.923694+0000","last_active":"2024-05-10T10:36:35.108444+0000","last_peered":"2024-05-10T10:36:35.108444+0000","last_clean":"2024-05-10T10:36:35.108444+0000","last_became_active":"2024-05-10T10:35:50.923381+0000","last_became_peered":"2024-05-10T10:35:50.923381+0000","last_unstale":"2024-05-10T10:36:35.108444+0000","last_undegraded":"2024-05-10T10:36:35.108444+0000","last_fullsized":"2024-05-10T10:36:35.108444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:59.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142217+0000","last_change":"2024-05-10T10:36:01.989868+0000","last_active":"2024-05-10T10:36:35.142217+0000","last_peered":"2024-05-10T10:36:35.142217+0000","last_clean":"2024-05-10T10:36:35.142217+0000","last_became_active":"2024-05-10T10:36:01.989283+0000","last_became_peered":"2024-05-10T10:36:01.989283+0000","last_unstale":"2024-05-10T10:36:35.142217+0000","last_undegraded":"2024-05-10T10:36:35.142217+0000","last_fullsized":"2024-05-10T10:36:35.142217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:35.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:01.405807+0000","last_change":"2024-05-10T10:35:47.915555+0000","last_active":"2024-05-10T10:37:01.405807+0000","last_peered":"2024-05-10T10:37:01.405807+0000","last_clean":"2024-05-10T10:37:01.405807+0000","last_became_active":"2024-05-10T10:35:47.912852+0000","last_became_peered":"2024-05-10T10:35:47.912852+0000","last_unstale":"2024-05-10T10:37:01.405807+0000","last_undegraded":"2024-05-10T10:37:01.405807+0000","last_fullsized":"2024-05-10T10:37:01.405807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:56:45.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109741+0000","last_change":"2024-05-10T10:35:50.934010+0000","last_active":"2024-05-10T10:36:35.109741+0000","last_peered":"2024-05-10T10:36:35.109741+0000","last_clean":"2024-05-10T10:36:35.109741+0000","last_became_active":"2024-05-10T10:35:50.933653+0000","last_became_peered":"2024-05-10T10:35:50.933653+0000","last_unstale":"2024-05-10T10:36:35.109741+0000","last_undegraded":"2024-05-10T10:36:35.109741+0000","last_fullsized":"2024-05-10T10:36:35.109741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:34.862734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142118+0000","last_change":"2024-05-10T10:36:01.990524+0000","last_active":"2024-05-10T10:36:35.142118+0000","last_peered":"2024-05-10T10:36:35.142118+0000","last_clean":"2024-05-10T10:36:35.142118+0000","last_became_active":"2024-05-10T10:36:01.990011+0000","last_became_peered":"2024-05-10T10:36:01.990011+0000","last_unstale":"2024-05-10T10:36:35.142118+0000","last_undegraded":"2024-05-10T10:36:35.142118+0000","last_fullsized":"2024-05-10T10:36:35.142118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:06.123095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.403594+0000","last_change":"2024-05-10T10:35:47.910977+0000","last_active":"2024-05-10T10:37:00.403594+0000","last_peered":"2024-05-10T10:37:00.403594+0000","last_clean":"2024-05-10T10:37:00.403594+0000","last_became_active":"2024-05-10T10:35:47.910623+0000","last_became_peered":"2024-05-10T10:35:47.910623+0000","last_unstale":"2024-05-10T10:37:00.403594+0000","last_undegraded":"2024-05-10T10:37:00.403594+0000","last_fullsized":"2024-05-10T10:37:00.403594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:37.760666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.112228+0000","last_change":"2024-05-10T10:35:50.937238+0000","last_active":"2024-05-10T10:36:35.112228+0000","last_peered":"2024-05-10T10:36:35.112228+0000","last_clean":"2024-05-10T10:36:35.112228+0000","last_became_active":"2024-05-10T10:35:50.936298+0000","last_became_peered":"2024-05-10T10:35:50.936298+0000","last_unstale":"2024-05-10T10:36:35.112228+0000","last_undegraded":"2024-05-10T10:36:35.112228+0000","last_fullsized":"2024-05-10T10:36:35.112228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:54.724452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107861+0000","last_change":"2024-05-10T10:36:01.987769+0000","last_active":"2024-05-10T10:36:35.107861+0000","last_peered":"2024-05-10T10:36:35.107861+0000","last_clean":"2024-05-10T10:36:35.107861+0000","last_became_active":"2024-05-10T10:36:01.987367+0000","last_became_peered":"2024-05-10T10:36:01.987367+0000","last_unstale":"2024-05-10T10:36:35.107861+0000","last_undegraded":"2024-05-10T10:36:35.107861+0000","last_fullsized":"2024-05-10T10:36:35.107861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:36.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.438422+0000","last_change":"2024-05-10T10:35:47.915082+0000","last_active":"2024-05-10T10:37:23.438422+0000","last_peered":"2024-05-10T10:37:23.438422+0000","last_clean":"2024-05-10T10:37:23.438422+0000","last_became_active":"2024-05-10T10:35:47.914722+0000","last_became_peered":"2024-05-10T10:35:47.914722+0000","last_unstale":"2024-05-10T10:37:23.438422+0000","last_undegraded":"2024-05-10T10:37:23.438422+0000","last_fullsized":"2024-05-10T10:37:23.438422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:31.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108511+0000","last_change":"2024-05-10T10:35:50.926181+0000","last_active":"2024-05-10T10:36:35.108511+0000","last_peered":"2024-05-10T10:36:35.108511+0000","last_clean":"2024-05-10T10:36:35.108511+0000","last_became_active":"2024-05-10T10:35:50.925865+0000","last_became_peered":"2024-05-10T10:35:50.925865+0000","last_unstale":"2024-05-10T10:36:35.108511+0000","last_undegraded":"2024-05-10T10:36:35.108511+0000","last_fullsized":"2024-05-10T10:36:35.108511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:04.976908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142412+0000","last_change":"2024-05-10T10:36:35.142412+0000","last_active":"2024-05-10T10:36:35.142412+0000","last_peered":"2024-05-10T10:36:35.142412+0000","last_clean":"2024-05-10T10:36:35.142412+0000","last_became_active":"2024-05-10T10:36:35.141959+0000","last_became_peered":"2024-05-10T10:36:35.141959+0000","last_unstale":"2024-05-10T10:36:35.142412+0000","last_undegraded":"2024-05-10T10:36:35.142412+0000","last_fullsized":"2024-05-10T10:36:35.142412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:20:34.133261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:59.392578+0000","last_change":"2024-05-10T10:35:47.912826+0000","last_active":"2024-05-10T10:36:59.392578+0000","last_peered":"2024-05-10T10:36:59.392578+0000","last_clean":"2024-05-10T10:36:59.392578+0000","last_became_active":"2024-05-10T10:35:47.912468+0000","last_became_peered":"2024-05-10T10:35:47.912468+0000","last_unstale":"2024-05-10T10:36:59.392578+0000","last_undegraded":"2024-05-10T10:36:59.392578+0000","last_fullsized":"2024-05-10T10:36:59.392578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:56.482104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.114920+0000","last_change":"2024-05-10T10:36:35.114920+0000","last_active":"2024-05-10T10:36:35.114920+0000","last_peered":"2024-05-10T10:36:35.114920+0000","last_clean":"2024-05-10T10:36:35.114920+0000","last_became_active":"2024-05-10T10:36:35.114546+0000","last_became_peered":"2024-05-10T10:36:35.114546+0000","last_unstale":"2024-05-10T10:36:35.114920+0000","last_undegraded":"2024-05-10T10:36:35.114920+0000","last_fullsized":"2024-05-10T10:36:35.114920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:52.454111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105050+0000","last_change":"2024-05-10T10:36:01.990086+0000","last_active":"2024-05-10T10:36:35.105050+0000","last_peered":"2024-05-10T10:36:35.105050+0000","last_clean":"2024-05-10T10:36:35.105050+0000","last_became_active":"2024-05-10T10:36:01.989624+0000","last_became_peered":"2024-05-10T10:36:01.989624+0000","last_unstale":"2024-05-10T10:36:35.105050+0000","last_undegraded":"2024-05-10T10:36:35.105050+0000","last_fullsized":"2024-05-10T10:36:35.105050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:33.708069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:58.384130+0000","last_change":"2024-05-10T10:35:47.905182+0000","last_active":"2024-05-10T10:36:58.384130+0000","last_peered":"2024-05-10T10:36:58.384130+0000","last_clean":"2024-05-10T10:36:58.384130+0000","last_became_active":"2024-05-10T10:35:47.904855+0000","last_became_peered":"2024-05-10T10:35:47.904855+0000","last_unstale":"2024-05-10T10:36:58.384130+0000","last_undegraded":"2024-05-10T10:36:58.384130+0000","last_fullsized":"2024-05-10T10:36:58.384130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:17:44.844156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107749+0000","last_change":"2024-05-10T10:35:50.935478+0000","last_active":"2024-05-10T10:36:35.107749+0000","last_peered":"2024-05-10T10:36:35.107749+0000","last_clean":"2024-05-10T10:36:35.107749+0000","last_became_active":"2024-05-10T10:35:50.933487+0000","last_became_peered":"2024-05-10T10:35:50.933487+0000","last_unstale":"2024-05-10T10:36:35.107749+0000","last_undegraded":"2024-05-10T10:36:35.107749+0000","last_fullsized":"2024-05-10T10:36:35.107749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:02.433423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108100+0000","last_change":"2024-05-10T10:36:01.984285+0000","last_active":"2024-05-10T10:36:35.108100+0000","last_peered":"2024-05-10T10:36:35.108100+0000","last_clean":"2024-05-10T10:36:35.108100+0000","last_became_active":"2024-05-10T10:36:01.983900+0000","last_became_peered":"2024-05-10T10:36:01.983900+0000","last_unstale":"2024-05-10T10:36:35.108100+0000","last_undegraded":"2024-05-10T10:36:35.108100+0000","last_fullsized":"2024-05-10T10:36:35.108100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:53.406106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.403241+0000","last_change":"2024-05-10T10:35:47.911123+0000","last_active":"2024-05-10T10:37:00.403241+0000","last_peered":"2024-05-10T10:37:00.403241+0000","last_clean":"2024-05-10T10:37:00.403241+0000","last_became_active":"2024-05-10T10:35:47.910762+0000","last_became_peered":"2024-05-10T10:35:47.910762+0000","last_unstale":"2024-05-10T10:37:00.403241+0000","last_undegraded":"2024-05-10T10:37:00.403241+0000","last_fullsized":"2024-05-10T10:37:00.403241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:07.045586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104519+0000","last_change":"2024-05-10T10:35:50.928093+0000","last_active":"2024-05-10T10:36:35.104519+0000","last_peered":"2024-05-10T10:36:35.104519+0000","last_clean":"2024-05-10T10:36:35.104519+0000","last_became_active":"2024-05-10T10:35:50.926945+0000","last_became_peered":"2024-05-10T10:35:50.926945+0000","last_unstale":"2024-05-10T10:36:35.104519+0000","last_undegraded":"2024-05-10T10:36:35.104519+0000","last_fullsized":"2024-05-10T10:36:35.104519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:11.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.143043+0000","last_change":"2024-05-10T10:36:35.143043+0000","last_active":"2024-05-10T10:36:35.143043+0000","last_peered":"2024-05-10T10:36:35.143043+0000","last_clean":"2024-05-10T10:36:35.143043+0000","last_became_active":"2024-05-10T10:36:35.142739+0000","last_became_peered":"2024-05-10T10:36:35.142739+0000","last_unstale":"2024-05-10T10:36:35.143043+0000","last_undegraded":"2024-05-10T10:36:35.143043+0000","last_fullsized":"2024-05-10T10:36:35.143043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:10.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:28.444693+0000","last_change":"2024-05-10T10:35:47.908687+0000","last_active":"2024-05-10T10:37:28.444693+0000","last_peered":"2024-05-10T10:37:28.444693+0000","last_clean":"2024-05-10T10:37:28.444693+0000","last_became_active":"2024-05-10T10:35:47.908349+0000","last_became_peered":"2024-05-10T10:35:47.908349+0000","last_unstale":"2024-05-10T10:37:28.444693+0000","last_undegraded":"2024-05-10T10:37:28.444693+0000","last_fullsized":"2024-05-10T10:37:28.444693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:48.696104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110473+0000","last_change":"2024-05-10T10:35:50.933729+0000","last_active":"2024-05-10T10:36:35.110473+0000","last_peered":"2024-05-10T10:36:35.110473+0000","last_clean":"2024-05-10T10:36:35.110473+0000","last_became_active":"2024-05-10T10:35:50.933109+0000","last_became_peered":"2024-05-10T10:35:50.933109+0000","last_unstale":"2024-05-10T10:36:35.110473+0000","last_undegraded":"2024-05-10T10:36:35.110473+0000","last_fullsized":"2024-05-10T10:36:35.110473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:10:38.305592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104491+0000","last_change":"2024-05-10T10:36:01.991969+0000","last_active":"2024-05-10T10:36:35.104491+0000","last_peered":"2024-05-10T10:36:35.104491+0000","last_clean":"2024-05-10T10:36:35.104491+0000","last_became_active":"2024-05-10T10:36:01.990278+0000","last_became_peered":"2024-05-10T10:36:01.990278+0000","last_unstale":"2024-05-10T10:36:35.104491+0000","last_undegraded":"2024-05-10T10:36:35.104491+0000","last_fullsized":"2024-05-10T10:36:35.104491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:33.070520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.401808+0000","last_change":"2024-05-10T10:35:47.911682+0000","last_active":"2024-05-10T10:37:00.401808+0000","last_peered":"2024-05-10T10:37:00.401808+0000","last_clean":"2024-05-10T10:37:00.401808+0000","last_became_active":"2024-05-10T10:35:47.911328+0000","last_became_peered":"2024-05-10T10:35:47.911328+0000","last_unstale":"2024-05-10T10:37:00.401808+0000","last_undegraded":"2024-05-10T10:37:00.401808+0000","last_fullsized":"2024-05-10T10:37:00.401808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.361340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108539+0000","last_change":"2024-05-10T10:35:50.931541+0000","last_active":"2024-05-10T10:36:35.108539+0000","last_peered":"2024-05-10T10:36:35.108539+0000","last_clean":"2024-05-10T10:36:35.108539+0000","last_became_active":"2024-05-10T10:35:50.931244+0000","last_became_peered":"2024-05-10T10:35:50.931244+0000","last_unstale":"2024-05-10T10:36:35.108539+0000","last_undegraded":"2024-05-10T10:36:35.108539+0000","last_fullsized":"2024-05-10T10:36:35.108539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:38.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142480+0000","last_change":"2024-05-10T10:36:01.990390+0000","last_active":"2024-05-10T10:36:35.142480+0000","last_peered":"2024-05-10T10:36:35.142480+0000","last_clean":"2024-05-10T10:36:35.142480+0000","last_became_active":"2024-05-10T10:36:01.989434+0000","last_became_peered":"2024-05-10T10:36:01.989434+0000","last_unstale":"2024-05-10T10:36:35.142480+0000","last_undegraded":"2024-05-10T10:36:35.142480+0000","last_fullsized":"2024-05-10T10:36:35.142480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:53.129212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:59.392291+0000","last_change":"2024-05-10T10:35:47.911590+0000","last_active":"2024-05-10T10:36:59.392291+0000","last_peered":"2024-05-10T10:36:59.392291+0000","last_clean":"2024-05-10T10:36:59.392291+0000","last_became_active":"2024-05-10T10:35:47.910950+0000","last_became_peered":"2024-05-10T10:35:47.910950+0000","last_unstale":"2024-05-10T10:36:59.392291+0000","last_undegraded":"2024-05-10T10:36:59.392291+0000","last_fullsized":"2024-05-10T10:36:59.392291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:18.844774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110047+0000","last_change":"2024-05-10T10:35:50.938174+0000","last_active":"2024-05-10T10:36:35.110047+0000","last_peered":"2024-05-10T10:36:35.110047+0000","last_clean":"2024-05-10T10:36:35.110047+0000","last_became_active":"2024-05-10T10:35:50.937618+0000","last_became_peered":"2024-05-10T10:35:50.937618+0000","last_unstale":"2024-05-10T10:36:35.110047+0000","last_undegraded":"2024-05-10T10:36:35.110047+0000","last_fullsized":"2024-05-10T10:36:35.110047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:26:48.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104514+0000","last_change":"2024-05-10T10:36:01.979226+0000","last_active":"2024-05-10T10:36:35.104514+0000","last_peered":"2024-05-10T10:36:35.104514+0000","last_clean":"2024-05-10T10:36:35.104514+0000","last_became_active":"2024-05-10T10:36:01.978847+0000","last_became_peered":"2024-05-10T10:36:01.978847+0000","last_unstale":"2024-05-10T10:36:35.104514+0000","last_undegraded":"2024-05-10T10:36:35.104514+0000","last_fullsized":"2024-05-10T10:36:35.104514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:10.629813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'36","reported_seq":156,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:28.445360+0000","last_change":"2024-05-10T10:36:35.150814+0000","last_active":"2024-05-10T10:37:28.445360+0000","last_peered":"2024-05-10T10:37:28.445360+0000","last_clean":"2024-05-10T10:37:28.445360+0000","last_became_active":"2024-05-10T10:36:35.142980+0000","last_became_peered":"2024-05-10T10:36:35.142980+0000","last_unstale":"2024-05-10T10:37:28.445360+0000","last_undegraded":"2024-05-10T10:37:28.445360+0000","last_fullsized":"2024-05-10T10:37:28.445360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:17.907275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":64,"num_read_kb":96,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109665+0000","last_change":"2024-05-10T10:35:50.925584+0000","last_active":"2024-05-10T10:36:35.109665+0000","last_peered":"2024-05-10T10:36:35.109665+0000","last_clean":"2024-05-10T10:36:35.109665+0000","last_became_active":"2024-05-10T10:35:50.925215+0000","last_became_peered":"2024-05-10T10:35:50.925215+0000","last_unstale":"2024-05-10T10:36:35.109665+0000","last_undegraded":"2024-05-10T10:36:35.109665+0000","last_fullsized":"2024-05-10T10:36:35.109665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:03.377110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.112541+0000","last_change":"2024-05-10T10:36:01.985090+0000","last_active":"2024-05-10T10:36:35.112541+0000","last_peered":"2024-05-10T10:36:35.112541+0000","last_clean":"2024-05-10T10:36:35.112541+0000","last_became_active":"2024-05-10T10:36:01.984059+0000","last_became_peered":"2024-05-10T10:36:01.984059+0000","last_unstale":"2024-05-10T10:36:35.112541+0000","last_undegraded":"2024-05-10T10:36:35.112541+0000","last_fullsized":"2024-05-10T10:36:35.112541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:41.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:01.404996+0000","last_change":"2024-05-10T10:35:47.914377+0000","last_active":"2024-05-10T10:37:01.404996+0000","last_peered":"2024-05-10T10:37:01.404996+0000","last_clean":"2024-05-10T10:37:01.404996+0000","last_became_active":"2024-05-10T10:35:47.913830+0000","last_became_peered":"2024-05-10T10:35:47.913830+0000","last_unstale":"2024-05-10T10:37:01.404996+0000","last_undegraded":"2024-05-10T10:37:01.404996+0000","last_fullsized":"2024-05-10T10:37:01.404996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:54:18.540952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.140857+0000","last_change":"2024-05-10T10:35:50.932721+0000","last_active":"2024-05-10T10:36:35.140857+0000","last_peered":"2024-05-10T10:36:35.140857+0000","last_clean":"2024-05-10T10:36:35.140857+0000","last_became_active":"2024-05-10T10:35:50.932410+0000","last_became_peered":"2024-05-10T10:35:50.932410+0000","last_unstale":"2024-05-10T10:36:35.140857+0000","last_undegraded":"2024-05-10T10:36:35.140857+0000","last_fullsized":"2024-05-10T10:36:35.140857+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:42.255534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.143581+0000","last_change":"2024-05-10T10:36:35.143581+0000","last_active":"2024-05-10T10:36:35.143581+0000","last_peered":"2024-05-10T10:36:35.143581+0000","last_clean":"2024-05-10T10:36:35.143581+0000","last_became_active":"2024-05-10T10:36:35.142957+0000","last_became_peered":"2024-05-10T10:36:35.142957+0000","last_unstale":"2024-05-10T10:36:35.143581+0000","last_undegraded":"2024-05-10T10:36:35.143581+0000","last_fullsized":"2024-05-10T10:36:35.143581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:14:39.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402941+0000","last_change":"2024-05-10T10:35:47.908874+0000","last_active":"2024-05-10T10:37:00.402941+0000","last_peered":"2024-05-10T10:37:00.402941+0000","last_clean":"2024-05-10T10:37:00.402941+0000","last_became_active":"2024-05-10T10:35:47.908478+0000","last_became_peered":"2024-05-10T10:35:47.908478+0000","last_unstale":"2024-05-10T10:37:00.402941+0000","last_undegraded":"2024-05-10T10:37:00.402941+0000","last_fullsized":"2024-05-10T10:37:00.402941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:13.147762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141379+0000","last_change":"2024-05-10T10:35:50.925168+0000","last_active":"2024-05-10T10:36:35.141379+0000","last_peered":"2024-05-10T10:36:35.141379+0000","last_clean":"2024-05-10T10:36:35.141379+0000","last_became_active":"2024-05-10T10:35:50.924509+0000","last_became_peered":"2024-05-10T10:35:50.924509+0000","last_unstale":"2024-05-10T10:36:35.141379+0000","last_undegraded":"2024-05-10T10:36:35.141379+0000","last_fullsized":"2024-05-10T10:36:35.141379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:08.458195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.144972+0000","last_change":"2024-05-10T10:36:35.144575+0000","last_active":"2024-05-10T10:36:35.144972+0000","last_peered":"2024-05-10T10:36:35.144972+0000","last_clean":"2024-05-10T10:36:35.144972+0000","last_became_active":"2024-05-10T10:36:35.144165+0000","last_became_peered":"2024-05-10T10:36:35.144165+0000","last_unstale":"2024-05-10T10:36:35.144972+0000","last_undegraded":"2024-05-10T10:36:35.144972+0000","last_fullsized":"2024-05-10T10:36:35.144972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:45.179163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.437814+0000","last_change":"2024-05-10T10:35:47.911208+0000","last_active":"2024-05-10T10:37:23.437814+0000","last_peered":"2024-05-10T10:37:23.437814+0000","last_clean":"2024-05-10T10:37:23.437814+0000","last_became_active":"2024-05-10T10:35:47.910820+0000","last_became_peered":"2024-05-10T10:35:47.910820+0000","last_unstale":"2024-05-10T10:37:23.437814+0000","last_undegraded":"2024-05-10T10:37:23.437814+0000","last_fullsized":"2024-05-10T10:37:23.437814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:04.804203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141550+0000","last_change":"2024-05-10T10:35:50.928195+0000","last_active":"2024-05-10T10:36:35.141550+0000","last_peered":"2024-05-10T10:36:35.141550+0000","last_clean":"2024-05-10T10:36:35.141550+0000","last_became_active":"2024-05-10T10:35:50.927832+0000","last_became_peered":"2024-05-10T10:35:50.927832+0000","last_unstale":"2024-05-10T10:36:35.141550+0000","last_undegraded":"2024-05-10T10:36:35.141550+0000","last_fullsized":"2024-05-10T10:36:35.141550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:03.071525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141227+0000","last_change":"2024-05-10T10:36:01.982483+0000","last_active":"2024-05-10T10:36:35.141227+0000","last_peered":"2024-05-10T10:36:35.141227+0000","last_clean":"2024-05-10T10:36:35.141227+0000","last_became_active":"2024-05-10T10:36:01.982117+0000","last_became_peered":"2024-05-10T10:36:01.982117+0000","last_unstale":"2024-05-10T10:36:35.141227+0000","last_undegraded":"2024-05-10T10:36:35.141227+0000","last_fullsized":"2024-05-10T10:36:35.141227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:08.145035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.401506+0000","last_change":"2024-05-10T10:35:47.912929+0000","last_active":"2024-05-10T10:37:00.401506+0000","last_peered":"2024-05-10T10:37:00.401506+0000","last_clean":"2024-05-10T10:37:00.401506+0000","last_became_active":"2024-05-10T10:35:47.912575+0000","last_became_peered":"2024-05-10T10:35:47.912575+0000","last_unstale":"2024-05-10T10:37:00.401506+0000","last_undegraded":"2024-05-10T10:37:00.401506+0000","last_fullsized":"2024-05-10T10:37:00.401506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:03.235579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111381+0000","last_change":"2024-05-10T10:35:50.935967+0000","last_active":"2024-05-10T10:36:35.111381+0000","last_peered":"2024-05-10T10:36:35.111381+0000","last_clean":"2024-05-10T10:36:35.111381+0000","last_became_active":"2024-05-10T10:35:50.935635+0000","last_became_peered":"2024-05-10T10:35:50.935635+0000","last_unstale":"2024-05-10T10:36:35.111381+0000","last_undegraded":"2024-05-10T10:36:35.111381+0000","last_fullsized":"2024-05-10T10:36:35.111381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:02:53.681522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111262+0000","last_change":"2024-05-10T10:36:01.979488+0000","last_active":"2024-05-10T10:36:35.111262+0000","last_peered":"2024-05-10T10:36:35.111262+0000","last_clean":"2024-05-10T10:36:35.111262+0000","last_became_active":"2024-05-10T10:36:01.978839+0000","last_became_peered":"2024-05-10T10:36:01.978839+0000","last_unstale":"2024-05-10T10:36:35.111262+0000","last_undegraded":"2024-05-10T10:36:35.111262+0000","last_fullsized":"2024-05-10T10:36:35.111262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:20.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.380137+0000","last_change":"2024-05-10T10:35:47.913235+0000","last_active":"2024-05-10T10:37:00.380137+0000","last_peered":"2024-05-10T10:37:00.380137+0000","last_clean":"2024-05-10T10:37:00.380137+0000","last_became_active":"2024-05-10T10:35:47.912788+0000","last_became_peered":"2024-05-10T10:35:47.912788+0000","last_unstale":"2024-05-10T10:37:00.380137+0000","last_undegraded":"2024-05-10T10:37:00.380137+0000","last_fullsized":"2024-05-10T10:37:00.380137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:56:10.275488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.105111+0000","last_change":"2024-05-10T10:35:50.930269+0000","last_active":"2024-05-10T10:36:35.105111+0000","last_peered":"2024-05-10T10:36:35.105111+0000","last_clean":"2024-05-10T10:36:35.105111+0000","last_became_active":"2024-05-10T10:35:50.929918+0000","last_became_peered":"2024-05-10T10:35:50.929918+0000","last_unstale":"2024-05-10T10:36:35.105111+0000","last_undegraded":"2024-05-10T10:36:35.105111+0000","last_fullsized":"2024-05-10T10:36:35.105111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:31.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108511+0000","last_change":"2024-05-10T10:36:02.018506+0000","last_active":"2024-05-10T10:36:35.108511+0000","last_peered":"2024-05-10T10:36:35.108511+0000","last_clean":"2024-05-10T10:36:35.108511+0000","last_became_active":"2024-05-10T10:36:02.018004+0000","last_became_peered":"2024-05-10T10:36:02.018004+0000","last_unstale":"2024-05-10T10:36:35.108511+0000","last_undegraded":"2024-05-10T10:36:35.108511+0000","last_fullsized":"2024-05-10T10:36:35.108511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:37.404970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104862+0000","last_change":"2024-05-10T10:35:47.910654+0000","last_active":"2024-05-10T10:36:35.104862+0000","last_peered":"2024-05-10T10:36:35.104862+0000","last_clean":"2024-05-10T10:36:35.104862+0000","last_became_active":"2024-05-10T10:35:47.910338+0000","last_became_peered":"2024-05-10T10:35:47.910338+0000","last_unstale":"2024-05-10T10:36:35.104862+0000","last_undegraded":"2024-05-10T10:36:35.104862+0000","last_fullsized":"2024-05-10T10:36:35.104862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:07.127574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107328+0000","last_change":"2024-05-10T10:35:50.940950+0000","last_active":"2024-05-10T10:36:35.107328+0000","last_peered":"2024-05-10T10:36:35.107328+0000","last_clean":"2024-05-10T10:36:35.107328+0000","last_became_active":"2024-05-10T10:35:50.940562+0000","last_became_peered":"2024-05-10T10:35:50.940562+0000","last_unstale":"2024-05-10T10:36:35.107328+0000","last_undegraded":"2024-05-10T10:36:35.107328+0000","last_fullsized":"2024-05-10T10:36:35.107328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:24.825900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141056+0000","last_change":"2024-05-10T10:36:01.989277+0000","last_active":"2024-05-10T10:36:35.141056+0000","last_peered":"2024-05-10T10:36:35.141056+0000","last_clean":"2024-05-10T10:36:35.141056+0000","last_became_active":"2024-05-10T10:36:01.988866+0000","last_became_peered":"2024-05-10T10:36:01.988866+0000","last_unstale":"2024-05-10T10:36:35.141056+0000","last_undegraded":"2024-05-10T10:36:35.141056+0000","last_fullsized":"2024-05-10T10:36:35.141056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:08:29.740044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:24.438971+0000","last_change":"2024-05-10T10:35:47.908332+0000","last_active":"2024-05-10T10:37:24.438971+0000","last_peered":"2024-05-10T10:37:24.438971+0000","last_clean":"2024-05-10T10:37:24.438971+0000","last_became_active":"2024-05-10T10:35:47.907933+0000","last_became_peered":"2024-05-10T10:35:47.907933+0000","last_unstale":"2024-05-10T10:37:24.438971+0000","last_undegraded":"2024-05-10T10:37:24.438971+0000","last_fullsized":"2024-05-10T10:37:24.438971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:47:48.702338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107175+0000","last_change":"2024-05-10T10:35:50.934638+0000","last_active":"2024-05-10T10:36:35.107175+0000","last_peered":"2024-05-10T10:36:35.107175+0000","last_clean":"2024-05-10T10:36:35.107175+0000","last_became_active":"2024-05-10T10:35:50.934285+0000","last_became_peered":"2024-05-10T10:35:50.934285+0000","last_unstale":"2024-05-10T10:36:35.107175+0000","last_undegraded":"2024-05-10T10:36:35.107175+0000","last_fullsized":"2024-05-10T10:36:35.107175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:24:31.179005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141235+0000","last_change":"2024-05-10T10:36:02.018468+0000","last_active":"2024-05-10T10:36:35.141235+0000","last_peered":"2024-05-10T10:36:35.141235+0000","last_clean":"2024-05-10T10:36:35.141235+0000","last_became_active":"2024-05-10T10:36:02.018087+0000","last_became_peered":"2024-05-10T10:36:02.018087+0000","last_unstale":"2024-05-10T10:36:35.141235+0000","last_undegraded":"2024-05-10T10:36:35.141235+0000","last_fullsized":"2024-05-10T10:36:35.141235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:55.712012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.112756+0000","last_change":"2024-05-10T10:35:50.933660+0000","last_active":"2024-05-10T10:36:35.112756+0000","last_peered":"2024-05-10T10:36:35.112756+0000","last_clean":"2024-05-10T10:36:35.112756+0000","last_became_active":"2024-05-10T10:35:50.933230+0000","last_became_peered":"2024-05-10T10:35:50.933230+0000","last_unstale":"2024-05-10T10:36:35.112756+0000","last_undegraded":"2024-05-10T10:36:35.112756+0000","last_fullsized":"2024-05-10T10:36:35.112756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:34:54.272278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:23.437181+0000","last_change":"2024-05-10T10:35:47.910161+0000","last_active":"2024-05-10T10:37:23.437181+0000","last_peered":"2024-05-10T10:37:23.437181+0000","last_clean":"2024-05-10T10:37:23.437181+0000","last_became_active":"2024-05-10T10:35:47.909875+0000","last_became_peered":"2024-05-10T10:35:47.909875+0000","last_unstale":"2024-05-10T10:37:23.437181+0000","last_undegraded":"2024-05-10T10:37:23.437181+0000","last_fullsized":"2024-05-10T10:37:23.437181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:21.620921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141225+0000","last_change":"2024-05-10T10:36:01.986755+0000","last_active":"2024-05-10T10:36:35.141225+0000","last_peered":"2024-05-10T10:36:35.141225+0000","last_clean":"2024-05-10T10:36:35.141225+0000","last_became_active":"2024-05-10T10:36:01.986340+0000","last_became_peered":"2024-05-10T10:36:01.986340+0000","last_unstale":"2024-05-10T10:36:35.141225+0000","last_undegraded":"2024-05-10T10:36:35.141225+0000","last_fullsized":"2024-05-10T10:36:35.141225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:38.434323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.114648+0000","last_change":"2024-05-10T10:36:35.114218+0000","last_active":"2024-05-10T10:36:35.114648+0000","last_peered":"2024-05-10T10:36:35.114648+0000","last_clean":"2024-05-10T10:36:35.114648+0000","last_became_active":"2024-05-10T10:36:35.113876+0000","last_became_peered":"2024-05-10T10:36:35.113876+0000","last_unstale":"2024-05-10T10:36:35.114648+0000","last_undegraded":"2024-05-10T10:36:35.114648+0000","last_fullsized":"2024-05-10T10:36:35.114648+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:21.415678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:01.407362+0000","last_change":"2024-05-10T10:35:47.909161+0000","last_active":"2024-05-10T10:37:01.407362+0000","last_peered":"2024-05-10T10:37:01.407362+0000","last_clean":"2024-05-10T10:37:01.407362+0000","last_became_active":"2024-05-10T10:35:47.908546+0000","last_became_peered":"2024-05-10T10:35:47.908546+0000","last_unstale":"2024-05-10T10:37:01.407362+0000","last_undegraded":"2024-05-10T10:37:01.407362+0000","last_fullsized":"2024-05-10T10:37:01.407362+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:52.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141367+0000","last_change":"2024-05-10T10:34:20.980948+0000","last_active":"2024-05-10T10:36:35.141367+0000","last_peered":"2024-05-10T10:36:35.141367+0000","last_clean":"2024-05-10T10:36:35.141367+0000","last_became_active":"2024-05-10T10:34:20.975662+0000","last_became_peered":"2024-05-10T10:34:20.975662+0000","last_unstale":"2024-05-10T10:36:35.141367+0000","last_undegraded":"2024-05-10T10:36:35.141367+0000","last_fullsized":"2024-05-10T10:36:35.141367+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_clean_scrub_stamp":"2024-05-10T10:30:55.419211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:33.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111792+0000","last_change":"2024-05-10T10:36:02.018578+0000","last_active":"2024-05-10T10:36:35.111792+0000","last_peered":"2024-05-10T10:36:35.111792+0000","last_clean":"2024-05-10T10:36:35.111792+0000","last_became_active":"2024-05-10T10:36:02.018126+0000","last_became_peered":"2024-05-10T10:36:02.018126+0000","last_unstale":"2024-05-10T10:36:35.111792+0000","last_undegraded":"2024-05-10T10:36:35.111792+0000","last_fullsized":"2024-05-10T10:36:35.111792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:18:17.112432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.140858+0000","last_change":"2024-05-10T10:35:50.927472+0000","last_active":"2024-05-10T10:36:35.140858+0000","last_peered":"2024-05-10T10:36:35.140858+0000","last_clean":"2024-05-10T10:36:35.140858+0000","last_became_active":"2024-05-10T10:35:50.927030+0000","last_became_peered":"2024-05-10T10:35:50.927030+0000","last_unstale":"2024-05-10T10:36:35.140858+0000","last_undegraded":"2024-05-10T10:36:35.140858+0000","last_fullsized":"2024-05-10T10:36:35.140858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:15:03.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:58.384938+0000","last_change":"2024-05-10T10:35:47.912675+0000","last_active":"2024-05-10T10:36:58.384938+0000","last_peered":"2024-05-10T10:36:58.384938+0000","last_clean":"2024-05-10T10:36:58.384938+0000","last_became_active":"2024-05-10T10:35:47.912297+0000","last_became_peered":"2024-05-10T10:35:47.912297+0000","last_unstale":"2024-05-10T10:36:58.384938+0000","last_undegraded":"2024-05-10T10:36:58.384938+0000","last_fullsized":"2024-05-10T10:36:58.384938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:46.178221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104688+0000","last_change":"2024-05-10T10:36:01.981501+0000","last_active":"2024-05-10T10:36:35.104688+0000","last_peered":"2024-05-10T10:36:35.104688+0000","last_clean":"2024-05-10T10:36:35.104688+0000","last_became_active":"2024-05-10T10:36:01.981029+0000","last_became_peered":"2024-05-10T10:36:01.981029+0000","last_unstale":"2024-05-10T10:36:35.104688+0000","last_undegraded":"2024-05-10T10:36:35.104688+0000","last_fullsized":"2024-05-10T10:36:35.104688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:18.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402125+0000","last_change":"2024-05-10T10:35:47.910701+0000","last_active":"2024-05-10T10:37:00.402125+0000","last_peered":"2024-05-10T10:37:00.402125+0000","last_clean":"2024-05-10T10:37:00.402125+0000","last_became_active":"2024-05-10T10:35:47.910286+0000","last_became_peered":"2024-05-10T10:35:47.910286+0000","last_unstale":"2024-05-10T10:37:00.402125+0000","last_undegraded":"2024-05-10T10:37:00.402125+0000","last_fullsized":"2024-05-10T10:37:00.402125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:07.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141217+0000","last_change":"2024-05-10T10:35:50.926026+0000","last_active":"2024-05-10T10:36:35.141217+0000","last_peered":"2024-05-10T10:36:35.141217+0000","last_clean":"2024-05-10T10:36:35.141217+0000","last_became_active":"2024-05-10T10:35:50.925624+0000","last_became_peered":"2024-05-10T10:35:50.925624+0000","last_unstale":"2024-05-10T10:36:35.141217+0000","last_undegraded":"2024-05-10T10:36:35.141217+0000","last_fullsized":"2024-05-10T10:36:35.141217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:22:02.828681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107718+0000","last_change":"2024-05-10T10:36:02.018403+0000","last_active":"2024-05-10T10:36:35.107718+0000","last_peered":"2024-05-10T10:36:35.107718+0000","last_clean":"2024-05-10T10:36:35.107718+0000","last_became_active":"2024-05-10T10:36:02.017980+0000","last_became_peered":"2024-05-10T10:36:02.017980+0000","last_unstale":"2024-05-10T10:36:35.107718+0000","last_undegraded":"2024-05-10T10:36:35.107718+0000","last_fullsized":"2024-05-10T10:36:35.107718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:07.264730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:21.948396+0000","last_change":"2024-05-10T10:35:47.912737+0000","last_active":"2024-05-10T10:37:21.948396+0000","last_peered":"2024-05-10T10:37:21.948396+0000","last_clean":"2024-05-10T10:37:21.948396+0000","last_became_active":"2024-05-10T10:35:47.912300+0000","last_became_peered":"2024-05-10T10:35:47.912300+0000","last_unstale":"2024-05-10T10:37:21.948396+0000","last_undegraded":"2024-05-10T10:37:21.948396+0000","last_fullsized":"2024-05-10T10:37:21.948396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:31:57.606863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108518+0000","last_change":"2024-05-10T10:35:50.935817+0000","last_active":"2024-05-10T10:36:35.108518+0000","last_peered":"2024-05-10T10:36:35.108518+0000","last_clean":"2024-05-10T10:36:35.108518+0000","last_became_active":"2024-05-10T10:35:50.935515+0000","last_became_peered":"2024-05-10T10:35:50.935515+0000","last_unstale":"2024-05-10T10:36:35.108518+0000","last_undegraded":"2024-05-10T10:36:35.108518+0000","last_fullsized":"2024-05-10T10:36:35.108518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:49.042193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.106866+0000","last_change":"2024-05-10T10:36:01.986002+0000","last_active":"2024-05-10T10:36:35.106866+0000","last_peered":"2024-05-10T10:36:35.106866+0000","last_clean":"2024-05-10T10:36:35.106866+0000","last_became_active":"2024-05-10T10:36:01.985558+0000","last_became_peered":"2024-05-10T10:36:01.985558+0000","last_unstale":"2024-05-10T10:36:35.106866+0000","last_undegraded":"2024-05-10T10:36:35.106866+0000","last_fullsized":"2024-05-10T10:36:35.106866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:27.732316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402360+0000","last_change":"2024-05-10T10:35:47.909254+0000","last_active":"2024-05-10T10:37:00.402360+0000","last_peered":"2024-05-10T10:37:00.402360+0000","last_clean":"2024-05-10T10:37:00.402360+0000","last_became_active":"2024-05-10T10:35:47.908370+0000","last_became_peered":"2024-05-10T10:35:47.908370+0000","last_unstale":"2024-05-10T10:37:00.402360+0000","last_undegraded":"2024-05-10T10:37:00.402360+0000","last_fullsized":"2024-05-10T10:37:00.402360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:44.643592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108194+0000","last_change":"2024-05-10T10:35:50.933001+0000","last_active":"2024-05-10T10:36:35.108194+0000","last_peered":"2024-05-10T10:36:35.108194+0000","last_clean":"2024-05-10T10:36:35.108194+0000","last_became_active":"2024-05-10T10:35:50.932010+0000","last_became_peered":"2024-05-10T10:35:50.932010+0000","last_unstale":"2024-05-10T10:36:35.108194+0000","last_undegraded":"2024-05-10T10:36:35.108194+0000","last_fullsized":"2024-05-10T10:36:35.108194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:01.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142379+0000","last_change":"2024-05-10T10:36:02.018650+0000","last_active":"2024-05-10T10:36:35.142379+0000","last_peered":"2024-05-10T10:36:35.142379+0000","last_clean":"2024-05-10T10:36:35.142379+0000","last_became_active":"2024-05-10T10:36:02.018258+0000","last_became_peered":"2024-05-10T10:36:02.018258+0000","last_unstale":"2024-05-10T10:36:35.142379+0000","last_undegraded":"2024-05-10T10:36:35.142379+0000","last_fullsized":"2024-05-10T10:36:35.142379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:33.590752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:57.376072+0000","last_change":"2024-05-10T10:35:47.910419+0000","last_active":"2024-05-10T10:36:57.376072+0000","last_peered":"2024-05-10T10:36:57.376072+0000","last_clean":"2024-05-10T10:36:57.376072+0000","last_became_active":"2024-05-10T10:35:47.909930+0000","last_became_peered":"2024-05-10T10:35:47.909930+0000","last_unstale":"2024-05-10T10:36:57.376072+0000","last_undegraded":"2024-05-10T10:36:57.376072+0000","last_fullsized":"2024-05-10T10:36:57.376072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:32.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109956+0000","last_change":"2024-05-10T10:35:50.923207+0000","last_active":"2024-05-10T10:36:35.109956+0000","last_peered":"2024-05-10T10:36:35.109956+0000","last_clean":"2024-05-10T10:36:35.109956+0000","last_became_active":"2024-05-10T10:35:50.922866+0000","last_became_peered":"2024-05-10T10:35:50.922866+0000","last_unstale":"2024-05-10T10:36:35.109956+0000","last_undegraded":"2024-05-10T10:36:35.109956+0000","last_fullsized":"2024-05-10T10:36:35.109956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:48.096498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104984+0000","last_change":"2024-05-10T10:36:01.985399+0000","last_active":"2024-05-10T10:36:35.104984+0000","last_peered":"2024-05-10T10:36:35.104984+0000","last_clean":"2024-05-10T10:36:35.104984+0000","last_became_active":"2024-05-10T10:36:01.985016+0000","last_became_peered":"2024-05-10T10:36:01.985016+0000","last_unstale":"2024-05-10T10:36:35.104984+0000","last_undegraded":"2024-05-10T10:36:35.104984+0000","last_fullsized":"2024-05-10T10:36:35.104984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:12.151693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402150+0000","last_change":"2024-05-10T10:35:47.908594+0000","last_active":"2024-05-10T10:37:00.402150+0000","last_peered":"2024-05-10T10:37:00.402150+0000","last_clean":"2024-05-10T10:37:00.402150+0000","last_became_active":"2024-05-10T10:35:47.908278+0000","last_became_peered":"2024-05-10T10:35:47.908278+0000","last_unstale":"2024-05-10T10:37:00.402150+0000","last_undegraded":"2024-05-10T10:37:00.402150+0000","last_fullsized":"2024-05-10T10:37:00.402150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:24.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104603+0000","last_change":"2024-05-10T10:35:50.933089+0000","last_active":"2024-05-10T10:36:35.104603+0000","last_peered":"2024-05-10T10:36:35.104603+0000","last_clean":"2024-05-10T10:36:35.104603+0000","last_became_active":"2024-05-10T10:35:50.932739+0000","last_became_peered":"2024-05-10T10:35:50.932739+0000","last_unstale":"2024-05-10T10:36:35.104603+0000","last_undegraded":"2024-05-10T10:36:35.104603+0000","last_fullsized":"2024-05-10T10:36:35.104603+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:15.302832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142702+0000","last_change":"2024-05-10T10:36:01.988561+0000","last_active":"2024-05-10T10:36:35.142702+0000","last_peered":"2024-05-10T10:36:35.142702+0000","last_clean":"2024-05-10T10:36:35.142702+0000","last_became_active":"2024-05-10T10:36:01.988120+0000","last_became_peered":"2024-05-10T10:36:01.988120+0000","last_unstale":"2024-05-10T10:36:35.142702+0000","last_undegraded":"2024-05-10T10:36:35.142702+0000","last_fullsized":"2024-05-10T10:36:35.142702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:35:37.120469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:59.390590+0000","last_change":"2024-05-10T10:35:47.915506+0000","last_active":"2024-05-10T10:36:59.390590+0000","last_peered":"2024-05-10T10:36:59.390590+0000","last_clean":"2024-05-10T10:36:59.390590+0000","last_became_active":"2024-05-10T10:35:47.915141+0000","last_became_peered":"2024-05-10T10:35:47.915141+0000","last_unstale":"2024-05-10T10:36:59.390590+0000","last_undegraded":"2024-05-10T10:36:59.390590+0000","last_fullsized":"2024-05-10T10:36:59.390590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:07:34.454646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108644+0000","last_change":"2024-05-10T10:35:50.938466+0000","last_active":"2024-05-10T10:36:35.108644+0000","last_peered":"2024-05-10T10:36:35.108644+0000","last_clean":"2024-05-10T10:36:35.108644+0000","last_became_active":"2024-05-10T10:35:50.938167+0000","last_became_peered":"2024-05-10T10:35:50.938167+0000","last_unstale":"2024-05-10T10:36:35.108644+0000","last_undegraded":"2024-05-10T10:36:35.108644+0000","last_fullsized":"2024-05-10T10:36:35.108644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:18.014708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.142690+0000","last_change":"2024-05-10T10:36:02.018661+0000","last_active":"2024-05-10T10:36:35.142690+0000","last_peered":"2024-05-10T10:36:35.142690+0000","last_clean":"2024-05-10T10:36:35.142690+0000","last_became_active":"2024-05-10T10:36:02.018196+0000","last_became_peered":"2024-05-10T10:36:02.018196+0000","last_unstale":"2024-05-10T10:36:35.142690+0000","last_undegraded":"2024-05-10T10:36:35.142690+0000","last_fullsized":"2024-05-10T10:36:35.142690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:11.605012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:57.378752+0000","last_change":"2024-05-10T10:35:47.917009+0000","last_active":"2024-05-10T10:36:57.378752+0000","last_peered":"2024-05-10T10:36:57.378752+0000","last_clean":"2024-05-10T10:36:57.378752+0000","last_became_active":"2024-05-10T10:35:47.916536+0000","last_became_peered":"2024-05-10T10:35:47.916536+0000","last_unstale":"2024-05-10T10:36:57.378752+0000","last_undegraded":"2024-05-10T10:36:57.378752+0000","last_fullsized":"2024-05-10T10:36:57.378752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:17.783027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141091+0000","last_change":"2024-05-10T10:35:50.926173+0000","last_active":"2024-05-10T10:36:35.141091+0000","last_peered":"2024-05-10T10:36:35.141091+0000","last_clean":"2024-05-10T10:36:35.141091+0000","last_became_active":"2024-05-10T10:35:50.925846+0000","last_became_peered":"2024-05-10T10:35:50.925846+0000","last_unstale":"2024-05-10T10:36:35.141091+0000","last_undegraded":"2024-05-10T10:36:35.141091+0000","last_fullsized":"2024-05-10T10:36:35.141091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:29:02.587667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.109908+0000","last_change":"2024-05-10T10:36:01.981312+0000","last_active":"2024-05-10T10:36:35.109908+0000","last_peered":"2024-05-10T10:36:35.109908+0000","last_clean":"2024-05-10T10:36:35.109908+0000","last_became_active":"2024-05-10T10:36:01.980861+0000","last_became_peered":"2024-05-10T10:36:01.980861+0000","last_unstale":"2024-05-10T10:36:35.109908+0000","last_undegraded":"2024-05-10T10:36:35.109908+0000","last_fullsized":"2024-05-10T10:36:35.109908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:29.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:24.439899+0000","last_change":"2024-05-10T10:35:47.911120+0000","last_active":"2024-05-10T10:37:24.439899+0000","last_peered":"2024-05-10T10:37:24.439899+0000","last_clean":"2024-05-10T10:37:24.439899+0000","last_became_active":"2024-05-10T10:35:47.909945+0000","last_became_peered":"2024-05-10T10:35:47.909945+0000","last_unstale":"2024-05-10T10:37:24.439899+0000","last_undegraded":"2024-05-10T10:37:24.439899+0000","last_fullsized":"2024-05-10T10:37:24.439899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:40.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141549+0000","last_change":"2024-05-10T10:35:50.933855+0000","last_active":"2024-05-10T10:36:35.141549+0000","last_peered":"2024-05-10T10:36:35.141549+0000","last_clean":"2024-05-10T10:36:35.141549+0000","last_became_active":"2024-05-10T10:35:50.933502+0000","last_became_peered":"2024-05-10T10:35:50.933502+0000","last_unstale":"2024-05-10T10:36:35.141549+0000","last_undegraded":"2024-05-10T10:36:35.141549+0000","last_fullsized":"2024-05-10T10:36:35.141549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:36.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107967+0000","last_change":"2024-05-10T10:36:01.985133+0000","last_active":"2024-05-10T10:36:35.107967+0000","last_peered":"2024-05-10T10:36:35.107967+0000","last_clean":"2024-05-10T10:36:35.107967+0000","last_became_active":"2024-05-10T10:36:01.984758+0000","last_became_peered":"2024-05-10T10:36:01.984758+0000","last_unstale":"2024-05-10T10:36:35.107967+0000","last_undegraded":"2024-05-10T10:36:35.107967+0000","last_fullsized":"2024-05-10T10:36:35.107967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:52.733063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.402504+0000","last_change":"2024-05-10T10:35:47.908645+0000","last_active":"2024-05-10T10:37:00.402504+0000","last_peered":"2024-05-10T10:37:00.402504+0000","last_clean":"2024-05-10T10:37:00.402504+0000","last_became_active":"2024-05-10T10:35:47.908123+0000","last_became_peered":"2024-05-10T10:35:47.908123+0000","last_unstale":"2024-05-10T10:37:00.402504+0000","last_undegraded":"2024-05-10T10:37:00.402504+0000","last_fullsized":"2024-05-10T10:37:00.402504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:46.181724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141170+0000","last_change":"2024-05-10T10:35:50.929215+0000","last_active":"2024-05-10T10:36:35.141170+0000","last_peered":"2024-05-10T10:36:35.141170+0000","last_clean":"2024-05-10T10:36:35.141170+0000","last_became_active":"2024-05-10T10:35:50.928907+0000","last_became_peered":"2024-05-10T10:35:50.928907+0000","last_unstale":"2024-05-10T10:36:35.141170+0000","last_undegraded":"2024-05-10T10:36:35.141170+0000","last_fullsized":"2024-05-10T10:36:35.141170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:57:17.340687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110495+0000","last_change":"2024-05-10T10:36:01.988548+0000","last_active":"2024-05-10T10:36:35.110495+0000","last_peered":"2024-05-10T10:36:35.110495+0000","last_clean":"2024-05-10T10:36:35.110495+0000","last_became_active":"2024-05-10T10:36:01.987221+0000","last_became_peered":"2024-05-10T10:36:01.987221+0000","last_unstale":"2024-05-10T10:36:35.110495+0000","last_undegraded":"2024-05-10T10:36:35.110495+0000","last_fullsized":"2024-05-10T10:36:35.110495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:55:22.434536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:37:00.400132+0000","last_change":"2024-05-10T10:35:47.914022+0000","last_active":"2024-05-10T10:37:00.400132+0000","last_peered":"2024-05-10T10:37:00.400132+0000","last_clean":"2024-05-10T10:37:00.400132+0000","last_became_active":"2024-05-10T10:35:47.913711+0000","last_became_peered":"2024-05-10T10:35:47.913711+0000","last_unstale":"2024-05-10T10:37:00.400132+0000","last_undegraded":"2024-05-10T10:37:00.400132+0000","last_fullsized":"2024-05-10T10:37:00.400132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:13.291307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141777+0000","last_change":"2024-05-10T10:35:50.927452+0000","last_active":"2024-05-10T10:36:35.141777+0000","last_peered":"2024-05-10T10:36:35.141777+0000","last_clean":"2024-05-10T10:36:35.141777+0000","last_became_active":"2024-05-10T10:35:50.927102+0000","last_became_peered":"2024-05-10T10:35:50.927102+0000","last_unstale":"2024-05-10T10:36:35.141777+0000","last_undegraded":"2024-05-10T10:36:35.141777+0000","last_fullsized":"2024-05-10T10:36:35.141777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:07.704196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104723+0000","last_change":"2024-05-10T10:35:50.933072+0000","last_active":"2024-05-10T10:36:35.104723+0000","last_peered":"2024-05-10T10:36:35.104723+0000","last_clean":"2024-05-10T10:36:35.104723+0000","last_became_active":"2024-05-10T10:35:50.932724+0000","last_became_peered":"2024-05-10T10:35:50.932724+0000","last_unstale":"2024-05-10T10:36:35.104723+0000","last_undegraded":"2024-05-10T10:36:35.104723+0000","last_fullsized":"2024-05-10T10:36:35.104723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:08.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107435+0000","last_change":"2024-05-10T10:35:50.931635+0000","last_active":"2024-05-10T10:36:35.107435+0000","last_peered":"2024-05-10T10:36:35.107435+0000","last_clean":"2024-05-10T10:36:35.107435+0000","last_became_active":"2024-05-10T10:35:50.930158+0000","last_became_peered":"2024-05-10T10:35:50.930158+0000","last_unstale":"2024-05-10T10:36:35.107435+0000","last_undegraded":"2024-05-10T10:36:35.107435+0000","last_fullsized":"2024-05-10T10:36:35.107435+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:22.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107239+0000","last_change":"2024-05-10T10:35:50.936947+0000","last_active":"2024-05-10T10:36:35.107239+0000","last_peered":"2024-05-10T10:36:35.107239+0000","last_clean":"2024-05-10T10:36:35.107239+0000","last_became_active":"2024-05-10T10:35:50.936156+0000","last_became_peered":"2024-05-10T10:35:50.936156+0000","last_unstale":"2024-05-10T10:36:35.107239+0000","last_undegraded":"2024-05-10T10:36:35.107239+0000","last_fullsized":"2024-05-10T10:36:35.107239+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:21.167787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108354+0000","last_change":"2024-05-10T10:35:50.935384+0000","last_active":"2024-05-10T10:36:35.108354+0000","last_peered":"2024-05-10T10:36:35.108354+0000","last_clean":"2024-05-10T10:36:35.108354+0000","last_became_active":"2024-05-10T10:35:50.932555+0000","last_became_peered":"2024-05-10T10:35:50.932555+0000","last_unstale":"2024-05-10T10:36:35.108354+0000","last_undegraded":"2024-05-10T10:36:35.108354+0000","last_fullsized":"2024-05-10T10:36:35.108354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:09:11.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141236+0000","last_change":"2024-05-10T10:35:50.929322+0000","last_active":"2024-05-10T10:36:35.141236+0000","last_peered":"2024-05-10T10:36:35.141236+0000","last_clean":"2024-05-10T10:36:35.141236+0000","last_became_active":"2024-05-10T10:35:50.928366+0000","last_became_peered":"2024-05-10T10:35:50.928366+0000","last_unstale":"2024-05-10T10:36:35.141236+0000","last_undegraded":"2024-05-10T10:36:35.141236+0000","last_fullsized":"2024-05-10T10:36:35.141236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:32:28.337962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110922+0000","last_change":"2024-05-10T10:36:35.110922+0000","last_active":"2024-05-10T10:36:35.110922+0000","last_peered":"2024-05-10T10:36:35.110922+0000","last_clean":"2024-05-10T10:36:35.110922+0000","last_became_active":"2024-05-10T10:36:35.110525+0000","last_became_peered":"2024-05-10T10:36:35.110525+0000","last_unstale":"2024-05-10T10:36:35.110922+0000","last_undegraded":"2024-05-10T10:36:35.110922+0000","last_fullsized":"2024-05-10T10:36:35.110922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:47.825168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.107795+0000","last_change":"2024-05-10T10:35:50.927869+0000","last_active":"2024-05-10T10:36:35.107795+0000","last_peered":"2024-05-10T10:36:35.107795+0000","last_clean":"2024-05-10T10:36:35.107795+0000","last_became_active":"2024-05-10T10:35:50.927552+0000","last_became_peered":"2024-05-10T10:35:50.927552+0000","last_unstale":"2024-05-10T10:36:35.107795+0000","last_undegraded":"2024-05-10T10:36:35.107795+0000","last_fullsized":"2024-05-10T10:36:35.107795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:36.962316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108617+0000","last_change":"2024-05-10T10:35:50.933099+0000","last_active":"2024-05-10T10:36:35.108617+0000","last_peered":"2024-05-10T10:36:35.108617+0000","last_clean":"2024-05-10T10:36:35.108617+0000","last_became_active":"2024-05-10T10:35:50.932765+0000","last_became_peered":"2024-05-10T10:35:50.932765+0000","last_unstale":"2024-05-10T10:36:35.108617+0000","last_undegraded":"2024-05-10T10:36:35.108617+0000","last_fullsized":"2024-05-10T10:36:35.108617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:19.095217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.111181+0000","last_change":"2024-05-10T10:35:50.937704+0000","last_active":"2024-05-10T10:36:35.111181+0000","last_peered":"2024-05-10T10:36:35.111181+0000","last_clean":"2024-05-10T10:36:35.111181+0000","last_became_active":"2024-05-10T10:35:50.937236+0000","last_became_peered":"2024-05-10T10:35:50.937236+0000","last_unstale":"2024-05-10T10:36:35.111181+0000","last_undegraded":"2024-05-10T10:36:35.111181+0000","last_fullsized":"2024-05-10T10:36:35.111181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:46.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.108911+0000","last_change":"2024-05-10T10:35:50.935181+0000","last_active":"2024-05-10T10:36:35.108911+0000","last_peered":"2024-05-10T10:36:35.108911+0000","last_clean":"2024-05-10T10:36:35.108911+0000","last_became_active":"2024-05-10T10:35:50.933446+0000","last_became_peered":"2024-05-10T10:35:50.933446+0000","last_unstale":"2024-05-10T10:36:35.108911+0000","last_undegraded":"2024-05-10T10:36:35.108911+0000","last_fullsized":"2024-05-10T10:36:35.108911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:03.972410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.110037+0000","last_change":"2024-05-10T10:35:50.931797+0000","last_active":"2024-05-10T10:36:35.110037+0000","last_peered":"2024-05-10T10:36:35.110037+0000","last_clean":"2024-05-10T10:36:35.110037+0000","last_became_active":"2024-05-10T10:35:50.931494+0000","last_became_peered":"2024-05-10T10:35:50.931494+0000","last_unstale":"2024-05-10T10:36:35.110037+0000","last_undegraded":"2024-05-10T10:36:35.110037+0000","last_fullsized":"2024-05-10T10:36:35.110037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:30.797462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.141018+0000","last_change":"2024-05-10T10:35:50.931688+0000","last_active":"2024-05-10T10:36:35.141018+0000","last_peered":"2024-05-10T10:36:35.141018+0000","last_clean":"2024-05-10T10:36:35.141018+0000","last_became_active":"2024-05-10T10:35:50.931362+0000","last_became_peered":"2024-05-10T10:35:50.931362+0000","last_unstale":"2024-05-10T10:36:35.141018+0000","last_undegraded":"2024-05-10T10:36:35.141018+0000","last_fullsized":"2024-05-10T10:36:35.141018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:24.263752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T10:36:35.104961+0000","last_change":"2024-05-10T10:35:50.930905+0000","last_active":"2024-05-10T10:36:35.104961+0000","last_peered":"2024-05-10T10:36:35.104961+0000","last_clean":"2024-05-10T10:36:35.104961+0000","last_became_active":"2024-05-10T10:35:50.930448+0000","last_became_peered":"2024-05-10T10:35:50.930448+0000","last_unstale":"2024-05-10T10:36:35.104961+0000","last_undegraded":"2024-05-10T10:36:35.104961+0000","last_fullsized":"2024-05-10T10:36:35.104961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:52:24.468600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72195,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":300,"num_read_kb":234,"num_write":247,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":216585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678051,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":502216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874280,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1708032,"data_stored":1154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037805,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36928,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708224,"statfs":{"total":95995035648,"available":95957221376,"internally_reserved":0,"allocated":1630208,"data_stored":1099021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":497650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":564062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":499106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":497358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":558784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1094212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:37:32.159 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T10:37:32.160 INFO:tasks.cephadm:Setting up client nodes... 2024-05-10T10:37:32.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-10T10:37:32.403 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:32.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:32 smithi112 ceph-mon[30100]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:37:32.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:32 smithi112 ceph-mon[30100]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-10T10:37:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:32 smithi195 ceph-mon[33773]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:37:32.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:32 smithi195 ceph-mon[33773]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-10T10:37:33.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:32 smithi148 ceph-mon[33568]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:37:33.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:32 smithi148 ceph-mon[33568]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-10T10:37:34.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 -- 172.21.15.112:0/46109465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c109230 msgr2=0x7f7b4c10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 --2- 172.21.15.112:0/46109465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c109230 0x7f7b4c10b6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400b0a0 tx=0x7f7b4400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 -- 172.21.15.112:0/46109465 shutdown_connections 2024-05-10T10:37:34.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 --2- 172.21.15.112:0/46109465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c109230 0x7f7b4c10b6a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 --2- 172.21.15.112:0/46109465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b4c103fd0 0x7f7b4c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 --2- 172.21.15.112:0/46109465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c106900 0x7f7b4c108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 -- 172.21.15.112:0/46109465 >> 172.21.15.112:0/46109465 conn(0x7f7b4c0fda00 msgr2=0x7f7b4c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:34.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 -- 172.21.15.112:0/46109465 shutdown_connections 2024-05-10T10:37:34.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.217+0000 7f7b543d6700 1 -- 172.21.15.112:0/46109465 wait complete. 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 Processor -- start 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 -- start start 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c103fd0 0x7f7b4c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c106900 0x7f7b4c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b4c109230 0x7f7b4c1a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10ef50 con 0x7f7b4c103fd0 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10edd0 con 0x7f7b4c106900 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b543d6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10f0d0 con 0x7f7b4c109230 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c103fd0 0x7f7b4c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c103fd0 0x7f7b4c19b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34800/0 (socket says 172.21.15.112:34800) 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 -- 172.21.15.112:0/863203141 learned_addr learned my addr 172.21.15.112:0/863203141 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52973700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b4c109230 0x7f7b4c1a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 -- 172.21.15.112:0/863203141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b4c109230 msgr2=0x7f7b4c1a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b4c109230 0x7f7b4c1a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 -- 172.21.15.112:0/863203141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c106900 msgr2=0x7f7b4c19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c106900 0x7f7b4c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.218+0000 7f7b52172700 1 -- 172.21.15.112:0/863203141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c1a3490 con 0x7f7b4c103fd0 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.219+0000 7f7b52172700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c103fd0 0x7f7b4c19b830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800b560 tx=0x7f7b4800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.219+0000 7f7b437fe700 1 -- 172.21.15.112:0/863203141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48015020 con 0x7f7b4c103fd0 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.219+0000 7f7b437fe700 1 -- 172.21.15.112:0/863203141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b48011cc0 con 0x7f7b4c103fd0 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.219+0000 7f7b437fe700 1 -- 172.21.15.112:0/863203141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48010d30 con 0x7f7b4c103fd0 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.219+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c1a3780 con 0x7f7b4c103fd0 2024-05-10T10:37:34.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.219+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1a3d40 con 0x7f7b4c103fd0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.220+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c0519e0 con 0x7f7b4c103fd0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.221+0000 7f7b437fe700 1 -- 172.21.15.112:0/863203141 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7b48018020 con 0x7f7b4c103fd0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.222+0000 7f7b437fe700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b380b2a30 0x7f7b380b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.222+0000 7f7b51971700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b380b2a30 0x7f7b380b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.222+0000 7f7b437fe700 1 -- 172.21.15.112:0/863203141 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f7b480e1d60 con 0x7f7b4c103fd0 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.222+0000 7f7b51971700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b380b2a30 0x7f7b380b4ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0097b0 tx=0x7f7b3c006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:34.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.226+0000 7f7b437fe700 1 -- 172.21.15.112:0/863203141 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7b4801f020 con 0x7f7b4c103fd0 2024-05-10T10:37:34.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.511+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7b4c02e510 con 0x7f7b4c103fd0 2024-05-10T10:37:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.513+0000 7f7b437fe700 1 -- 172.21.15.112:0/863203141 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7b480a96d0 con 0x7f7b4c103fd0 2024-05-10T10:37:34.514 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-05-10T10:37:34.514 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDu+D1mJGmPHhAAzm3f2spMTN/3uedami6CdA== 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b380b2a30 msgr2=0x7f7b380b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b380b2a30 0x7f7b380b4ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0097b0 tx=0x7f7b3c006d20 comp rx=0 tx=0).stop 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c103fd0 msgr2=0x7f7b4c19b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c103fd0 0x7f7b4c19b830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800b560 tx=0x7f7b4800b870 comp rx=0 tx=0).stop 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 shutdown_connections 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b380b2a30 0x7f7b380b4ef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b4c109230 0x7f7b4c1a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b4c103fd0 0x7f7b4c19b830 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 --2- 172.21.15.112:0/863203141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c106900 0x7f7b4c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.515+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 >> 172.21.15.112:0/863203141 conn(0x7f7b4c0fda00 msgr2=0x7f7b4c103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:34.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.516+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 shutdown_connections 2024-05-10T10:37:34.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:34.516+0000 7f7b543d6700 1 -- 172.21.15.112:0/863203141 wait complete. 2024-05-10T10:37:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:34 smithi195 ceph-mon[33773]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-05-10T10:37:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-10T10:37:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:34 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T10:37:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:34 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/863203141' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T10:37:34.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:34 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/863203141' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T10:37:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:34 smithi112 ceph-mon[30100]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-05-10T10:37:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-10T10:37:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:34 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T10:37:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:34 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/863203141' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T10:37:34.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:34 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/863203141' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T10:37:35.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:34 smithi148 ceph-mon[33568]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-05-10T10:37:35.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-10T10:37:35.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:34 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T10:37:35.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:34 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/863203141' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T10:37:35.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:34 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/863203141' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T10:37:35.357 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:37:35.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-10T10:37:35.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T10:37:35.394 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T10:37:35.395 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-10T10:37:35.395 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-10T10:37:35.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-10T10:37:35.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T10:37:35.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:35 smithi112 ceph-mon[30100]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T10:37:35.680 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-10T10:37:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T10:37:35.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:35 smithi195 ceph-mon[33773]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T10:37:36.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-10T10:37:36.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T10:37:36.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:35 smithi148 ceph-mon[33568]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T10:37:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:36 smithi195 ceph-mon[33773]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-10T10:37:36.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:36 smithi195 ceph-mon[33773]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T10:37:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:36 smithi112 ceph-mon[30100]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-10T10:37:36.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:36 smithi112 ceph-mon[30100]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T10:37:37.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:36 smithi148 ceph-mon[33568]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-10T10:37:37.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:36 smithi148 ceph-mon[33568]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.484+0000 7f278f279700 1 -- 172.21.15.112:0/1767278207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2788102d90 msgr2=0x7f278810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.484+0000 7f278f279700 1 --2- 172.21.15.112:0/1767278207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2788102d90 0x7f278810f250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f278000b0a0 tx=0x7f278000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.484+0000 7f278f279700 1 -- 172.21.15.112:0/1767278207 shutdown_connections 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.484+0000 7f278f279700 1 --2- 172.21.15.112:0/1767278207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2788102d90 0x7f278810f250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.484+0000 7f278f279700 1 --2- 172.21.15.112:0/1767278207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2788108450 0x7f2788108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.484+0000 7f278f279700 1 --2- 172.21.15.112:0/1767278207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27881023d0 0x7f2788102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.484+0000 7f278f279700 1 -- 172.21.15.112:0/1767278207 >> 172.21.15.112:0/1767278207 conn(0x7f2788078410 msgr2=0x7f2788078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.485+0000 7f278f279700 1 -- 172.21.15.112:0/1767278207 shutdown_connections 2024-05-10T10:37:37.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.485+0000 7f278f279700 1 -- 172.21.15.112:0/1767278207 wait complete. 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.485+0000 7f278f279700 1 Processor -- start 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.485+0000 7f278f279700 1 -- start start 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.485+0000 7f278f279700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27881023d0 0x7f278819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.485+0000 7f278f279700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2788102d90 0x7f278819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.485+0000 7f278f279700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2788108450 0x7f27881987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278f279700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27881123d0 con 0x7f27881023d0 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278f279700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2788112250 con 0x7f2788108450 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278f279700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2788112550 con 0x7f2788102d90 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27881023d0 0x7f278819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27881023d0 0x7f278819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55622/0 (socket says 172.21.15.112:55622) 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 -- 172.21.15.112:0/3070946976 learned_addr learned my addr 172.21.15.112:0/3070946976 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d816700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2788108450 0x7f27881987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:37.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278c814700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2788102d90 0x7f278819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 -- 172.21.15.112:0/3070946976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2788102d90 msgr2=0x7f278819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2788102d90 0x7f278819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 -- 172.21.15.112:0/3070946976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2788108450 msgr2=0x7f27881987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2788108450 0x7f27881987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 -- 172.21.15.112:0/3070946976 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2788198fd0 con 0x7f27881023d0 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278c814700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2788102d90 0x7f278819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278d015700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27881023d0 0x7f278819e670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2778007b50 tx=0x7f2778007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f277e7fc700 1 -- 172.21.15.112:0/3070946976 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778019020 con 0x7f27881023d0 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27881992c0 con 0x7f27881023d0 2024-05-10T10:37:37.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f277e7fc700 1 -- 172.21.15.112:0/3070946976 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2778004cc0 con 0x7f27881023d0 2024-05-10T10:37:37.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f277e7fc700 1 -- 172.21.15.112:0/3070946976 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277800e500 con 0x7f27881023d0 2024-05-10T10:37:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.486+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278806bc10 con 0x7f27881023d0 2024-05-10T10:37:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.488+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2788103ac0 con 0x7f27881023d0 2024-05-10T10:37:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.489+0000 7f277e7fc700 1 -- 172.21.15.112:0/3070946976 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2778004750 con 0x7f27881023d0 2024-05-10T10:37:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.490+0000 7f277e7fc700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2910 0x7f27740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.490+0000 7f278c814700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2910 0x7f27740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.490+0000 7f277e7fc700 1 -- 172.21.15.112:0/3070946976 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f27780e3af0 con 0x7f27881023d0 2024-05-10T10:37:37.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.490+0000 7f278c814700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2910 0x7f27740b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2788078cd0 tx=0x7f278400b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.495+0000 7f277e7fc700 1 -- 172.21.15.112:0/3070946976 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2778023020 con 0x7f27881023d0 2024-05-10T10:37:37.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.764+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f278806c700 con 0x7f27881023d0 2024-05-10T10:37:37.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.769+0000 7f277e7fc700 1 -- 172.21.15.112:0/3070946976 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288656 (secure 0 0 0) 0x7f27780ab3b0 con 0x7f27881023d0 2024-05-10T10:37:37.774 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:37:37.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2910 msgr2=0x7f27740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2910 0x7f27740b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2788078cd0 tx=0x7f278400b5f0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27881023d0 msgr2=0x7f278819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27881023d0 0x7f278819e670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2778007b50 tx=0x7f2778007e60 comp rx=0 tx=0).stop 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 shutdown_connections 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2788108450 0x7f27881987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27881023d0 0x7f278819e670 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2910 0x7f27740b4dd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 --2- 172.21.15.112:0/3070946976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2788102d90 0x7f278819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.773+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 >> 172.21.15.112:0/3070946976 conn(0x7f2788078410 msgr2=0x7f2788101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.774+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 shutdown_connections 2024-05-10T10:37:37.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:37.774+0000 7f278f279700 1 -- 172.21.15.112:0/3070946976 wait complete. 2024-05-10T10:37:38.432 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1898750131},{"type":"v1","addr":"172.21.15.112:6801","nonce":1898750131}]},"active_addr":"172.21.15.112:6801/1898750131","active_change":"2024-05-10T10:28:33.178773+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1078512171}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3091790545}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3685786090}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":4212086858}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3835420530}]}]} 2024-05-10T10:37:38.438 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-10T10:37:38.438 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-10T10:37:38.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:37:38.681 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:38.803 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:38 smithi112 ceph-mon[30100]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T10:37:38.803 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:38 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3070946976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T10:37:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:38 smithi148 ceph-mon[33568]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T10:37:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:38 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3070946976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T10:37:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:38 smithi195 ceph-mon[33773]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T10:37:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:38 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3070946976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.414+0000 7f68ea49e700 1 -- 172.21.15.112:0/1230016214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4107730 msgr2=0x7f68e410f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.414+0000 7f68ea49e700 1 --2- 172.21.15.112:0/1230016214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4107730 0x7f68e410f2d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00b0d0 tx=0x7f68dc00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 -- 172.21.15.112:0/1230016214 shutdown_connections 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 --2- 172.21.15.112:0/1230016214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4107730 0x7f68e410f2d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 --2- 172.21.15.112:0/1230016214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e41029e0 0x7f68e4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 --2- 172.21.15.112:0/1230016214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68e4103390 0x7f68e4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 -- 172.21.15.112:0/1230016214 >> 172.21.15.112:0/1230016214 conn(0x7f68e4079350 msgr2=0x7f68e4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 -- 172.21.15.112:0/1230016214 shutdown_connections 2024-05-10T10:37:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 -- 172.21.15.112:0/1230016214 wait complete. 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 Processor -- start 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.415+0000 7f68ea49e700 1 -- start start 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68ea49e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e41029e0 0x7f68e419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68ea49e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4103390 0x7f68e419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68ea49e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68e4107730 0x7f68e41a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68ea49e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4112450 con 0x7f68e4103390 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68ea49e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41122d0 con 0x7f68e41029e0 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68ea49e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41125d0 con 0x7f68e4107730 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4103390 0x7f68e419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4103390 0x7f68e419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55638/0 (socket says 172.21.15.112:55638) 2024-05-10T10:37:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 -- 172.21.15.112:0/2471652767 learned_addr learned my addr 172.21.15.112:0/2471652767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e8a3b700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68e4107730 0x7f68e41a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e3fff700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e41029e0 0x7f68e419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 -- 172.21.15.112:0/2471652767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68e4107730 msgr2=0x7f68e41a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68e4107730 0x7f68e41a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 -- 172.21.15.112:0/2471652767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e41029e0 msgr2=0x7f68e419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e41029e0 0x7f68e419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 -- 172.21.15.112:0/2471652767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e41a5e10 con 0x7f68e4103390 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e37fe700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4103390 0x7f68e419ad90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68d800c9b0 tx=0x7f68d800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e17fa700 1 -- 172.21.15.112:0/2471652767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8007aa0 con 0x7f68e4103390 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e17fa700 1 -- 172.21.15.112:0/2471652767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d8007c40 con 0x7f68e4103390 2024-05-10T10:37:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.416+0000 7f68e17fa700 1 -- 172.21.15.112:0/2471652767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8018660 con 0x7f68e4103390 2024-05-10T10:37:40.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.417+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e41a5fe0 con 0x7f68e4103390 2024-05-10T10:37:40.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.417+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e41a6440 con 0x7f68e4103390 2024-05-10T10:37:40.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.417+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e4103ae0 con 0x7f68e4103390 2024-05-10T10:37:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.419+0000 7f68e17fa700 1 -- 172.21.15.112:0/2471652767 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f68d800f450 con 0x7f68e4103390 2024-05-10T10:37:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.419+0000 7f68e17fa700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f68cc0b2910 0x7f68cc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.420+0000 7f68e3fff700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f68cc0b2910 0x7f68cc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.420+0000 7f68e17fa700 1 -- 172.21.15.112:0/2471652767 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f68d80e29c0 con 0x7f68e4103390 2024-05-10T10:37:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.420+0000 7f68e3fff700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f68cc0b2910 0x7f68cc0b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68d4005fd0 tx=0x7f68d400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.423+0000 7f68e17fa700 1 -- 172.21.15.112:0/2471652767 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f68d8014030 con 0x7f68e4103390 2024-05-10T10:37:40.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.630+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68e402e510 con 0x7f68e4103390 2024-05-10T10:37:40.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.631+0000 7f68e17fa700 1 -- 172.21.15.112:0/2471652767 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f68d80aa1f0 con 0x7f68e4103390 2024-05-10T10:37:40.633 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:37:40.633 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:40.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f68cc0b2910 msgr2=0x7f68cc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:40.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f68cc0b2910 0x7f68cc0b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68d4005fd0 tx=0x7f68d400a560 comp rx=0 tx=0).stop 2024-05-10T10:37:40.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4103390 msgr2=0x7f68e419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:40.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4103390 0x7f68e419ad90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68d800c9b0 tx=0x7f68d800cd70 comp rx=0 tx=0).stop 2024-05-10T10:37:40.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 shutdown_connections 2024-05-10T10:37:40.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f68cc0b2910 0x7f68cc0b4dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68e4107730 0x7f68e41a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e41029e0 0x7f68e419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 --2- 172.21.15.112:0/2471652767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68e4103390 0x7f68e419ad90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:40.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 >> 172.21.15.112:0/2471652767 conn(0x7f68e4079350 msgr2=0x7f68e41007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:40.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 shutdown_connections 2024-05-10T10:37:40.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:40.634+0000 7f68ea49e700 1 -- 172.21.15.112:0/2471652767 wait complete. 2024-05-10T10:37:40.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:40 smithi112 ceph-mon[30100]: pgmap v353: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:41.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:40 smithi148 ceph-mon[33568]: pgmap v353: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:40 smithi195 ceph-mon[33773]: pgmap v353: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:41.339 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-10T10:37:41.339 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:37:41.580 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:41.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:41 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2471652767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:42.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:41 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2471652767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:42.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:41 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2471652767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:42.753 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:42 smithi112 ceph-mon[30100]: pgmap v354: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:43.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:42 smithi148 ceph-mon[33568]: pgmap v354: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:42 smithi195 ceph-mon[33773]: pgmap v354: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 -- 172.21.15.112:0/1431621572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 msgr2=0x7f08fc102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 --2- 172.21.15.112:0/1431621572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc102350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009990 tx=0x7f08f0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 -- 172.21.15.112:0/1431621572 shutdown_connections 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 --2- 172.21.15.112:0/1431621572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08fc102890 0x7f08fc110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 --2- 172.21.15.112:0/1431621572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc109680 0x7f08fc109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 --2- 172.21.15.112:0/1431621572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc102350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 -- 172.21.15.112:0/1431621572 >> 172.21.15.112:0/1431621572 conn(0x7f08fc077610 msgr2=0x7f08fc077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 -- 172.21.15.112:0/1431621572 shutdown_connections 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 -- 172.21.15.112:0/1431621572 wait complete. 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 Processor -- start 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.292+0000 7f09013f6700 1 -- start start 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f09013f6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f09013f6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08fc102890 0x7f08fc1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f09013f6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc109680 0x7f08fc103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f09013f6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc113560 con 0x7f08fc101ed0 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f09013f6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1133e0 con 0x7f08fc109680 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f09013f6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1136e0 con 0x7f08fc102890 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc1030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55662/0 (socket says 172.21.15.112:55662) 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 -- 172.21.15.112:0/1896620421 learned_addr learned my addr 172.21.15.112:0/1896620421 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08fa7fc700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08fc102890 0x7f08fc1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08fb7fe700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc109680 0x7f08fc103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 -- 172.21.15.112:0/1896620421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08fc102890 msgr2=0x7f08fc1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08fc102890 0x7f08fc1035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 -- 172.21.15.112:0/1896620421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc109680 msgr2=0x7f08fc103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc109680 0x7f08fc103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 -- 172.21.15.112:0/1896620421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc1a7f70 con 0x7f08fc101ed0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08fb7fe700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc109680 0x7f08fc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.293+0000 7f08faffd700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc1030b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d950 tx=0x7f08ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.294+0000 7f08e3fff700 1 -- 172.21.15.112:0/1896620421 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec016070 con 0x7f08fc101ed0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.294+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc1a8260 con 0x7f08fc101ed0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.294+0000 7f08e3fff700 1 -- 172.21.15.112:0/1896620421 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08ec009dd0 con 0x7f08fc101ed0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.294+0000 7f08e3fff700 1 -- 172.21.15.112:0/1896620421 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00f3c0 con 0x7f08fc101ed0 2024-05-10T10:37:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.294+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc1a87c0 con 0x7f08fc101ed0 2024-05-10T10:37:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.295+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08fc06a190 con 0x7f08fc101ed0 2024-05-10T10:37:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.296+0000 7f08e3fff700 1 -- 172.21.15.112:0/1896620421 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f08ec019070 con 0x7f08fc101ed0 2024-05-10T10:37:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.297+0000 7f08e3fff700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f08e40b2910 0x7f08e40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.297+0000 7f08fa7fc700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f08e40b2910 0x7f08e40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.297+0000 7f08e3fff700 1 -- 172.21.15.112:0/1896620421 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f08ec0e2760 con 0x7f08fc101ed0 2024-05-10T10:37:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.298+0000 7f08fa7fc700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f08e40b2910 0x7f08e40b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b5c0 tx=0x7f08f0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:43.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.301+0000 7f08e3fff700 1 -- 172.21.15.112:0/1896620421 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f08ec0a9f70 con 0x7f08fc101ed0 2024-05-10T10:37:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.507+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08fc02e6d0 con 0x7f08fc101ed0 2024-05-10T10:37:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.508+0000 7f08e3fff700 1 -- 172.21.15.112:0/1896620421 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f08ec0a9870 con 0x7f08fc101ed0 2024-05-10T10:37:43.515 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:37:43.515 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:43.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f08e40b2910 msgr2=0x7f08e40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f08e40b2910 0x7f08e40b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b5c0 tx=0x7f08f0005960 comp rx=0 tx=0).stop 2024-05-10T10:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 msgr2=0x7f08fc1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc1030b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d950 tx=0x7f08ec00dc60 comp rx=0 tx=0).stop 2024-05-10T10:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 shutdown_connections 2024-05-10T10:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc109680 0x7f08fc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc101ed0 0x7f08fc1030b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f08e40b2910 0x7f08e40b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 --2- 172.21.15.112:0/1896620421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08fc102890 0x7f08fc1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 >> 172.21.15.112:0/1896620421 conn(0x7f08fc077610 msgr2=0x7f08fc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 shutdown_connections 2024-05-10T10:37:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:43.511+0000 7f09013f6700 1 -- 172.21.15.112:0/1896620421 wait complete. 2024-05-10T10:37:43.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:43 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1896620421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:43.967 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:43 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1896620421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:44.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:43 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1896620421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:37:44.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-10T10:37:44.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-10T10:37:44.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-10T10:37:44.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-10T10:37:44.240 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-10T10:37:44.240 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-10T10:37:44.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-10T10:37:44.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-10T10:37:44.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-10T10:37:44.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-10T10:37:44.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-10T10:37:44.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-10T10:37:44.828 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:44.950 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:45.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:44 smithi112 ceph-mon[30100]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 234 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:45.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:44 smithi148 ceph-mon[33568]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 234 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:45.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:44 smithi195 ceph-mon[33773]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 234 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:46.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:46 smithi112 ceph-mon[30100]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:46.790 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:46.791 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:46.846 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:46 smithi148 ceph-mon[33568]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:46 smithi195 ceph-mon[33773]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:47.206 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:47.207 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:47.207 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:47.208 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:47.208 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:47.209 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:47.209 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:37:48.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:48 smithi112 ceph-mon[30100]: pgmap v357: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 174 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:49.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:48 smithi148 ceph-mon[33568]: pgmap v357: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 174 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:48 smithi195 ceph-mon[33773]: pgmap v357: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 174 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:49.761 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:49 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:37:49.886 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:49 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:37:49.922 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:49 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:37:50.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:50 smithi112 ceph-mon[30100]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:51.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:50 smithi148 ceph-mon[33568]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:51.065 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:50 smithi195 ceph-mon[33773]: pgmap v358: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-10T10:37:52.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:37:52.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:52 smithi112 ceph-mon[30100]: pgmap v359: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:37:53.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 -- 172.21.15.112:0/30266681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c107760 msgr2=0x7f437c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:53.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 --2- 172.21.15.112:0/30266681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c107760 0x7f437c10f2e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f437400b0d0 tx=0x7f437400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 -- 172.21.15.112:0/30266681 shutdown_connections 2024-05-10T10:37:53.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 --2- 172.21.15.112:0/30266681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c107760 0x7f437c10f2e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 --2- 172.21.15.112:0/30266681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437c102e00 0x7f437c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 --2- 172.21.15.112:0/30266681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f437c1037b0 0x7f437c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 -- 172.21.15.112:0/30266681 >> 172.21.15.112:0/30266681 conn(0x7f437c078a10 msgr2=0x7f437c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 -- 172.21.15.112:0/30266681 shutdown_connections 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.764+0000 7f43814a8700 1 -- 172.21.15.112:0/30266681 wait complete. 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 Processor -- start 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 -- start start 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f437c102e00 0x7f437c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c1037b0 0x7f437c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437c107760 0x7f437c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1123e0 con 0x7f437c1037b0 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f437c112260 con 0x7f437c107760 2024-05-10T10:37:53.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f43814a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f437c112560 con 0x7f437c102e00 2024-05-10T10:37:53.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c1037b0 0x7f437c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:53.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c1037b0 0x7f437c19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45742/0 (socket says 172.21.15.112:45742) 2024-05-10T10:37:53.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 -- 172.21.15.112:0/914634984 learned_addr learned my addr 172.21.15.112:0/914634984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:53.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437b7fe700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437c107760 0x7f437c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:53.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437affd700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f437c102e00 0x7f437c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 -- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f437c102e00 msgr2=0x7f437c19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f437c102e00 0x7f437c19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 -- 172.21.15.112:0/914634984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437c107760 msgr2=0x7f437c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437c107760 0x7f437c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437a7fc700 1 -- 172.21.15.112:0/914634984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c198fa0 con 0x7f437c1037b0 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.765+0000 7f437affd700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f437c102e00 0x7f437c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.766+0000 7f437a7fc700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c1037b0 0x7f437c19eb30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f437000d980 tx=0x7f437000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.766+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370016070 con 0x7f437c1037b0 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.766+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4370004830 con 0x7f437c1037b0 2024-05-10T10:37:53.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.766+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c199290 con 0x7f437c1037b0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.766+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370005440 con 0x7f437c1037b0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.766+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c1a5570 con 0x7f437c1037b0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.767+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f437c063bf0 con 0x7f437c1037b0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.768+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f43700055e0 con 0x7f437c1037b0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f4363fff700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f43640b2910 0x7f43640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f437affd700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f43640b2910 0x7f43640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f43700e2bf0 con 0x7f437c1037b0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f4363fff700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] conn(0x7f43640bdb70 0x7f43640c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 --> [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f437001ce50 con 0x7f43640bdb70 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f43700e3040 con 0x7f437c1037b0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f437affd700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f43640b2910 0x7f43640b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f436c005fd0 tx=0x7f436c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:53.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.769+0000 7f437b7fe700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] conn(0x7f43640bdb70 0x7f43640c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.770+0000 7f437b7fe700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] conn(0x7f43640bdb70 0x7f43640c0010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.771+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== osd.5 v2:172.21.15.195:6808/465167884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f437001ce50 con 0x7f43640bdb70 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.787+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 --> [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f437c064070 con 0x7f43640bdb70 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.787+0000 7f4363fff700 1 -- 172.21.15.112:0/914634984 <== osd.5 v2:172.21.15.195:6808/465167884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f437c064070 con 0x7f43640bdb70 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] conn(0x7f43640bdb70 msgr2=0x7f43640c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] conn(0x7f43640bdb70 0x7f43640c0010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f43640b2910 msgr2=0x7f43640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f43640b2910 0x7f43640b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f436c005fd0 tx=0x7f436c005ee0 comp rx=0 tx=0).stop 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c1037b0 msgr2=0x7f437c19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 --2- 172.21.15.112:0/914634984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437c1037b0 0x7f437c19eb30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f437000d980 tx=0x7f437000dc90 comp rx=0 tx=0).stop 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f437b7fe700 1 -- 172.21.15.112:0/914634984 reap_dead start 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 shutdown_connections 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 >> 172.21.15.112:0/914634984 conn(0x7f437c078a10 msgr2=0x7f437c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:53.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 shutdown_connections 2024-05-10T10:37:53.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:52.788+0000 7f43814a8700 1 -- 172.21.15.112:0/914634984 wait complete. 2024-05-10T10:37:53.298 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:52 smithi148 ceph-mon[33568]: pgmap v359: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:37:53.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:52 smithi195 ceph-mon[33773]: pgmap v359: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:37:54.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 -- 172.21.15.112:0/1962144463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103f00 msgr2=0x7fc4b81103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 --2- 172.21.15.112:0/1962144463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103f00 0x7fc4b81103b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b0d0 tx=0x7fc4b000b3e0 comp rx=0 tx=0).stop 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 -- 172.21.15.112:0/1962144463 shutdown_connections 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 --2- 172.21.15.112:0/1962144463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103f00 0x7fc4b81103b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 --2- 172.21.15.112:0/1962144463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b81095e0 0x7fc4b81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 --2- 172.21.15.112:0/1962144463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4b8103540 0x7fc4b81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 -- 172.21.15.112:0/1962144463 >> 172.21.15.112:0/1962144463 conn(0x7fc4b807a650 msgr2=0x7fc4b807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 -- 172.21.15.112:0/1962144463 shutdown_connections 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 -- 172.21.15.112:0/1962144463 wait complete. 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.038+0000 7fc4bd409700 1 Processor -- start 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 -- start start 2024-05-10T10:37:54.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103540 0x7fc4b81a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b8103f00 0x7fc4b81a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4b81095e0 0x7fc4b819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b81135f0 con 0x7fc4b8103540 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b8113470 con 0x7fc4b8103f00 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b8113770 con 0x7fc4b81095e0 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103540 0x7fc4b81a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103540 0x7fc4b81a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45762/0 (socket says 172.21.15.112:45762) 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 -- 172.21.15.112:0/1770032526 learned_addr learned my addr 172.21.15.112:0/1770032526 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:54.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4affff700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b8103f00 0x7fc4b81a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b77fe700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4b81095e0 0x7fc4b819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 -- 172.21.15.112:0/1770032526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4b81095e0 msgr2=0x7fc4b819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4b81095e0 0x7fc4b819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 -- 172.21.15.112:0/1770032526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b8103f00 msgr2=0x7fc4b81a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b8103f00 0x7fc4b81a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 -- 172.21.15.112:0/1770032526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b819c0f0 con 0x7fc4b8103540 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b77fe700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4b81095e0 0x7fc4b819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:54.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b6ffd700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103540 0x7fc4b81a17d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800d950 tx=0x7fc4a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8016070 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.039+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b819c3e0 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.040+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4a8009dd0 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.040+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800f3c0 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.040+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b81a86a0 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.040+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4b8077350 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.042+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc4a8019070 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.042+0000 7fc4b4ff9700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc4a40b2910 0x7fc4a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4affff700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc4a40b2910 0x7fc4a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc4a80e28b0 con 0x7fc4b8103540 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4b4ff9700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] conn(0x7fc4a40bdb70 0x7fc4a40c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:54.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 --> [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4a40c06e0 con 0x7fc4a40bdb70 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc4a80e2d00 con 0x7fc4b8103540 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4affff700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc4a40b2910 0x7fc4a40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4b807af50 tx=0x7fc4a000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4b77fe700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] conn(0x7fc4a40bdb70 0x7fc4a40c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.043+0000 7fc4b77fe700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] conn(0x7fc4a40bdb70 0x7fc4a40c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.044+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== osd.3 v2:172.21.15.112:6810/1175679850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4a40c06e0 con 0x7fc4a40bdb70 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.060+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 --> [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4b8104c50 con 0x7fc4a40bdb70 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.060+0000 7fc4b4ff9700 1 -- 172.21.15.112:0/1770032526 <== osd.3 v2:172.21.15.112:6810/1175679850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc4b8104c50 con 0x7fc4a40bdb70 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] conn(0x7fc4a40bdb70 msgr2=0x7fc4a40c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] conn(0x7fc4a40bdb70 0x7fc4a40c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc4a40b2910 msgr2=0x7fc4a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc4a40b2910 0x7fc4a40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4b807af50 tx=0x7fc4a000a380 comp rx=0 tx=0).stop 2024-05-10T10:37:54.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103540 msgr2=0x7fc4b81a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:54.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 --2- 172.21.15.112:0/1770032526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4b8103540 0x7fc4b81a17d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800d950 tx=0x7fc4a800dc60 comp rx=0 tx=0).stop 2024-05-10T10:37:54.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4b77fe700 1 -- 172.21.15.112:0/1770032526 reap_dead start 2024-05-10T10:37:54.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 shutdown_connections 2024-05-10T10:37:54.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 >> 172.21.15.112:0/1770032526 conn(0x7fc4b807a650 msgr2=0x7fc4b8075960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:54.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 shutdown_connections 2024-05-10T10:37:54.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:54.061+0000 7fc4bd409700 1 -- 172.21.15.112:0/1770032526 wait complete. 2024-05-10T10:37:54.967 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:54 smithi112 ceph-mon[30100]: pgmap v360: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:37:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:54 smithi148 ceph-mon[33568]: pgmap v360: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:37:55.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:54 smithi195 ceph-mon[33773]: pgmap v360: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:37:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 -- 172.21.15.112:0/1207646254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8102d90 msgr2=0x7f9cb810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 --2- 172.21.15.112:0/1207646254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8102d90 0x7f9cb810f250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000b0a0 tx=0x7f9cb000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 -- 172.21.15.112:0/1207646254 shutdown_connections 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 --2- 172.21.15.112:0/1207646254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8102d90 0x7f9cb810f250 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 --2- 172.21.15.112:0/1207646254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cb8108450 0x7f9cb8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 --2- 172.21.15.112:0/1207646254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb81023d0 0x7f9cb8102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 -- 172.21.15.112:0/1207646254 >> 172.21.15.112:0/1207646254 conn(0x7f9cb8078410 msgr2=0x7f9cb8078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 -- 172.21.15.112:0/1207646254 shutdown_connections 2024-05-10T10:37:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.002+0000 7f9cbd219700 1 -- 172.21.15.112:0/1207646254 wait complete. 2024-05-10T10:37:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 Processor -- start 2024-05-10T10:37:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 -- start start 2024-05-10T10:37:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb81023d0 0x7f9cb8104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cb8102d90 0x7f9cb81045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8108450 0x7f9cb8104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81123f0 con 0x7f9cb8108450 2024-05-10T10:37:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8112270 con 0x7f9cb8102d90 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cbd219700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8112570 con 0x7f9cb81023d0 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cb759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8108450 0x7f9cb8104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cb759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8108450 0x7f9cb8104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46638/0 (socket says 172.21.15.112:46638) 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cb759e700 1 -- 172.21.15.112:0/2829659755 learned_addr learned my addr 172.21.15.112:0/2829659755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.003+0000 7f9cb659c700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cb8102d90 0x7f9cb81045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb6d9d700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb81023d0 0x7f9cb8104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb759e700 1 -- 172.21.15.112:0/2829659755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb81023d0 msgr2=0x7f9cb8104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb759e700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb81023d0 0x7f9cb8104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb759e700 1 -- 172.21.15.112:0/2829659755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cb8102d90 msgr2=0x7f9cb81045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb759e700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cb8102d90 0x7f9cb81045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb759e700 1 -- 172.21.15.112:0/2829659755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb81a7fe0 con 0x7f9cb8108450 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb6d9d700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb81023d0 0x7f9cb8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:56.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cb759e700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8108450 0x7f9cb8104b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000b830 tx=0x7f9cb0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:56.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0021070 con 0x7f9cb8108450 2024-05-10T10:37:56.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb81a8270 con 0x7f9cb8108450 2024-05-10T10:37:56.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cb0007b80 con 0x7f9cb8108450 2024-05-10T10:37:56.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00052a0 con 0x7f9cb8108450 2024-05-10T10:37:56.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.004+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb81a8770 con 0x7f9cb8108450 2024-05-10T10:37:56.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.005+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9cb81074a0 con 0x7f9cb8108450 2024-05-10T10:37:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.006+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9cb0024070 con 0x7f9cb8108450 2024-05-10T10:37:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.007+0000 7f9caffff700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ca40b2910 0x7f9ca40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.007+0000 7f9cb6d9d700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ca40b2910 0x7f9ca40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.007+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9cb000f020 con 0x7f9cb8108450 2024-05-10T10:37:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.007+0000 7f9caffff700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] conn(0x7f9ca40bdb70 0x7f9ca40c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.007+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 --> [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cb0031e50 con 0x7f9ca40bdb70 2024-05-10T10:37:56.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.007+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9cb00e4610 con 0x7f9cb8108450 2024-05-10T10:37:56.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.008+0000 7f9cb659c700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] conn(0x7f9ca40bdb70 0x7f9ca40c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:56.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.008+0000 7f9cb6d9d700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ca40b2910 0x7f9ca40b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800bb90 tx=0x7f9ca8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:56.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.008+0000 7f9cb659c700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] conn(0x7f9ca40bdb70 0x7f9ca40c0010 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:56.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.009+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== osd.6 v2:172.21.15.112:6818/979146261 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9cb0031e50 con 0x7f9ca40bdb70 2024-05-10T10:37:56.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.025+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 --> [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9cb8103ac0 con 0x7f9ca40bdb70 2024-05-10T10:37:56.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.025+0000 7f9caffff700 1 -- 172.21.15.112:0/2829659755 <== osd.6 v2:172.21.15.112:6818/979146261 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9cb8103ac0 con 0x7f9ca40bdb70 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] conn(0x7f9ca40bdb70 msgr2=0x7f9ca40c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] conn(0x7f9ca40bdb70 0x7f9ca40c0010 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ca40b2910 msgr2=0x7f9ca40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ca40b2910 0x7f9ca40b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800bb90 tx=0x7f9ca8005d80 comp rx=0 tx=0).stop 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8108450 msgr2=0x7f9cb8104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 --2- 172.21.15.112:0/2829659755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8108450 0x7f9cb8104b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000b830 tx=0x7f9cb0004d00 comp rx=0 tx=0).stop 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cb759e700 1 -- 172.21.15.112:0/2829659755 reap_dead start 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 shutdown_connections 2024-05-10T10:37:56.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 >> 172.21.15.112:0/2829659755 conn(0x7f9cb8078410 msgr2=0x7f9cb806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:56.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 shutdown_connections 2024-05-10T10:37:56.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:56.026+0000 7f9cbd219700 1 -- 172.21.15.112:0/2829659755 wait complete. 2024-05-10T10:37:56.767 INFO:teuthology.orchestra.run.smithi112.stdout:107374182482 2024-05-10T10:37:56.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-10T10:37:57.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:56 smithi112 ceph-mon[30100]: pgmap v361: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:37:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:56 smithi148 ceph-mon[33568]: pgmap v361: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:37:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:56 smithi195 ceph-mon[33773]: pgmap v361: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:37:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/2241701676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 msgr2=0x7ff5d010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 --2- 172.21.15.112:0/2241701676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 0x7ff5d010f350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800b0a0 tx=0x7ff5c800b3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/2241701676 shutdown_connections 2024-05-10T10:37:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 --2- 172.21.15.112:0/2241701676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 0x7ff5d010f350 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 --2- 172.21.15.112:0/2241701676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d01042f0 0x7ff5d01046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 --2- 172.21.15.112:0/2241701676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d01023d0 0x7ff5d0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/2241701676 >> 172.21.15.112:0/2241701676 conn(0x7ff5d0078410 msgr2=0x7ff5d0078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/2241701676 shutdown_connections 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/2241701676 wait complete. 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 Processor -- start 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.320+0000 7ff5d7c4b700 1 -- start start 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d7c4b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d01023d0 0x7ff5d019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d7c4b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 0x7ff5d019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d7c4b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d01042f0 0x7ff5d01a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d7c4b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0112540 con 0x7ff5d0102d90 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d7c4b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01123c0 con 0x7ff5d01042f0 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d7c4b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01126c0 con 0x7ff5d01023d0 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 0x7ff5d019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 0x7ff5d019aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46668/0 (socket says 172.21.15.112:46668) 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 -- 172.21.15.112:0/3919256768 learned_addr learned my addr 172.21.15.112:0/3919256768 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:58.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d59e7700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d01023d0 0x7ff5d019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d61e8700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d01042f0 0x7ff5d01a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 -- 172.21.15.112:0/3919256768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d01023d0 msgr2=0x7ff5d019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d01023d0 0x7ff5d019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 -- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d01042f0 msgr2=0x7ff5d01a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d01042f0 0x7ff5d01a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 -- 172.21.15.112:0/3919256768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d01a5f20 con 0x7ff5d0102d90 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5d51e6700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 0x7ff5d019aea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e9a0 tx=0x7ff5c000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.321+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000cbd0 con 0x7ff5d0102d90 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.322+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5c000cd70 con 0x7ff5d0102d90 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.322+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d01a60f0 con 0x7ff5d0102d90 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.322+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0018680 con 0x7ff5d0102d90 2024-05-10T10:37:58.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.322+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01a6630 con 0x7ff5d0102d90 2024-05-10T10:37:58.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.322+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5d0063c30 con 0x7ff5d0102d90 2024-05-10T10:37:58.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.324+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7ff5c0010950 con 0x7ff5d0102d90 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5c6ffd700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff5bc0b2910 0x7ff5bc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5d59e7700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff5bc0b2910 0x7ff5bc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff5c00e2ea0 con 0x7ff5d0102d90 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5c6ffd700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] conn(0x7ff5bc0bdaf0 0x7ff5bc0bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 --> [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5c0024e70 con 0x7ff5bc0bdaf0 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff5c00e32f0 con 0x7ff5d0102d90 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5d59e7700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff5bc0b2910 0x7ff5bc0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005fd0 tx=0x7ff5cc005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:58.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.325+0000 7ff5d61e8700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] conn(0x7ff5bc0bdaf0 0x7ff5bc0bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.326+0000 7ff5d61e8700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] conn(0x7ff5bc0bdaf0 0x7ff5bc0bff90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:58.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.327+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== osd.1 v2:172.21.15.148:6800/3233597314 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5c0024e70 con 0x7ff5bc0bdaf0 2024-05-10T10:37:58.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.343+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 --> [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5d00640b0 con 0x7ff5bc0bdaf0 2024-05-10T10:37:58.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5c6ffd700 1 -- 172.21.15.112:0/3919256768 <== osd.1 v2:172.21.15.148:6800/3233597314 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff5d00640b0 con 0x7ff5bc0bdaf0 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] conn(0x7ff5bc0bdaf0 msgr2=0x7ff5bc0bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] conn(0x7ff5bc0bdaf0 0x7ff5bc0bff90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff5bc0b2910 msgr2=0x7ff5bc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff5bc0b2910 0x7ff5bc0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005fd0 tx=0x7ff5cc005db0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 msgr2=0x7ff5d019aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 --2- 172.21.15.112:0/3919256768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102d90 0x7ff5d019aea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e9a0 tx=0x7ff5c000ed60 comp rx=0 tx=0).stop 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d61e8700 1 -- 172.21.15.112:0/3919256768 reap_dead start 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 shutdown_connections 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.344+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 >> 172.21.15.112:0/3919256768 conn(0x7ff5d0078410 msgr2=0x7ff5d0100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:58.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.345+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 shutdown_connections 2024-05-10T10:37:58.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:58.345+0000 7ff5d7c4b700 1 -- 172.21.15.112:0/3919256768 wait complete. 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2288054729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98102d60 msgr2=0x7f8e9810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 --2- 172.21.15.112:0/2288054729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98102d60 0x7f8e9810f250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000b0a0 tx=0x7f8e9000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2288054729 shutdown_connections 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 --2- 172.21.15.112:0/2288054729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98102d60 0x7f8e9810f250 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 --2- 172.21.15.112:0/2288054729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e98108410 0x7f8e981087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 --2- 172.21.15.112:0/2288054729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e981023a0 0x7f8e98102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2288054729 >> 172.21.15.112:0/2288054729 conn(0x7f8e98079350 msgr2=0x7f8e98079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2288054729 shutdown_connections 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.078+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2288054729 wait complete. 2024-05-10T10:37:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 Processor -- start 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 -- start start 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e981023a0 0x7f8e9819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e98102d60 0x7f8e9819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98108410 0x7f8e981a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e98112430 con 0x7f8e98108410 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e981122b0 con 0x7f8e98102d60 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.079+0000 7f8e9df1a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8e981125b0 con 0x7f8e981023a0 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98108410 0x7f8e981a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98108410 0x7f8e981a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46648/0 (socket says 172.21.15.112:46648) 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 -- 172.21.15.112:0/2945992391 learned_addr learned my addr 172.21.15.112:0/2945992391 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e96ffd700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e98102d60 0x7f8e9819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e977fe700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e981023a0 0x7f8e9819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 -- 172.21.15.112:0/2945992391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e981023a0 msgr2=0x7f8e9819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e981023a0 0x7f8e9819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 -- 172.21.15.112:0/2945992391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e98102d60 msgr2=0x7f8e9819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e98102d60 0x7f8e9819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 -- 172.21.15.112:0/2945992391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e981a5e90 con 0x7f8e98108410 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e97fff700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98108410 0x7f8e981a5950 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000b070 tx=0x7f8e90004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90021070 con 0x7f8e98108410 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e981a6060 con 0x7f8e98108410 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e90007b80 con 0x7f8e98108410 2024-05-10T10:37:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e900046e0 con 0x7f8e98108410 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.080+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e981a64b0 con 0x7f8e98108410 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.081+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e98063bb0 con 0x7f8e98108410 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.083+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8e90003a00 con 0x7f8e98108410 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.083+0000 7f8e94ff9700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8e800b2910 0x7f8e800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.083+0000 7f8e977fe700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8e800b2910 0x7f8e800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.083+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8e9000f020 con 0x7f8e98108410 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.083+0000 7f8e94ff9700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] conn(0x7f8e800bdb70 0x7f8e800c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.083+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 --> [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e90027e70 con 0x7f8e800bdb70 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.084+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8e900e4250 con 0x7f8e98108410 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.084+0000 7f8e977fe700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8e800b2910 0x7f8e800b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e88005fd0 tx=0x7f8e88005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.084+0000 7f8e96ffd700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] conn(0x7f8e800bdb70 0x7f8e800c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.084+0000 7f8e96ffd700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] conn(0x7f8e800bdb70 0x7f8e800c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.085+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== osd.0 v2:172.21.15.112:6802/221387326 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e90027e70 con 0x7f8e800bdb70 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.101+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 --> [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e98064030 con 0x7f8e800bdb70 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.101+0000 7f8e94ff9700 1 -- 172.21.15.112:0/2945992391 <== osd.0 v2:172.21.15.112:6802/221387326 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8e98064030 con 0x7f8e800bdb70 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] conn(0x7f8e800bdb70 msgr2=0x7f8e800c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] conn(0x7f8e800bdb70 0x7f8e800c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8e800b2910 msgr2=0x7f8e800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8e800b2910 0x7f8e800b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e88005fd0 tx=0x7f8e88005db0 comp rx=0 tx=0).stop 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98108410 msgr2=0x7f8e981a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 --2- 172.21.15.112:0/2945992391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e98108410 0x7f8e981a5950 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000b070 tx=0x7f8e90004d50 comp rx=0 tx=0).stop 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e97fff700 1 -- 172.21.15.112:0/2945992391 reap_dead start 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 shutdown_connections 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 >> 172.21.15.112:0/2945992391 conn(0x7f8e98079350 msgr2=0x7f8e98101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 shutdown_connections 2024-05-10T10:37:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:37:57.102+0000 7f8e9df1a700 1 -- 172.21.15.112:0/2945992391 wait complete. 2024-05-10T10:37:59.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:37:58 smithi112 ceph-mon[30100]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:37:59.145 INFO:teuthology.orchestra.run.smithi112.stdout:34359738465 2024-05-10T10:37:59.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-10T10:37:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:37:58 smithi148 ceph-mon[33568]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:37:59.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:37:58 smithi195 ceph-mon[33773]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:37:59.603 INFO:teuthology.orchestra.run.smithi112.stdout:154618822728 2024-05-10T10:37:59.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-10T10:38:00.237 INFO:teuthology.orchestra.run.smithi112.stdout:55834574941 2024-05-10T10:38:00.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-10T10:38:01.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:00 smithi112 ceph-mon[30100]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-05-10T10:38:01.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:00 smithi148 ceph-mon[33568]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-05-10T10:38:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:00 smithi195 ceph-mon[33773]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-05-10T10:38:02.872 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:02 smithi112 ceph-mon[30100]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 -- 172.21.15.112:0/2571944286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f01076c0 msgr2=0x7fa3f010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 --2- 172.21.15.112:0/2571944286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f01076c0 0x7fa3f010f290 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b0d0 tx=0x7fa3e800b3e0 comp rx=0 tx=0).stop 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 -- 172.21.15.112:0/2571944286 shutdown_connections 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 --2- 172.21.15.112:0/2571944286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f01076c0 0x7fa3f010f290 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 --2- 172.21.15.112:0/2571944286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f0102630 0x7fa3f0102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 --2- 172.21.15.112:0/2571944286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f0102fe0 0x7fa3f0106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 -- 172.21.15.112:0/2571944286 >> 172.21.15.112:0/2571944286 conn(0x7fa3f0079370 msgr2=0x7fa3f0079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:02.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 -- 172.21.15.112:0/2571944286 shutdown_connections 2024-05-10T10:38:02.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.973+0000 7fa3f62de700 1 -- 172.21.15.112:0/2571944286 wait complete. 2024-05-10T10:38:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.974+0000 7fa3f62de700 1 Processor -- start 2024-05-10T10:38:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.974+0000 7fa3f62de700 1 -- start start 2024-05-10T10:38:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f62de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0102630 0x7fa3f019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f62de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f0102fe0 0x7fa3f019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f62de700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f01076c0 0x7fa3f0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:02.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f62de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0112410 con 0x7fa3f0102630 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f62de700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0112290 con 0x7fa3f01076c0 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f62de700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0112590 con 0x7fa3f0102fe0 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0102630 0x7fa3f019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0102630 0x7fa3f019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46678/0 (socket says 172.21.15.112:46678) 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 -- 172.21.15.112:0/3487274477 learned_addr learned my addr 172.21.15.112:0/3487274477 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3ef7fe700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f0102fe0 0x7fa3f019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f487b700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f01076c0 0x7fa3f0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 -- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f0102fe0 msgr2=0x7fa3f019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f0102fe0 0x7fa3f019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 -- 172.21.15.112:0/3487274477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f01076c0 msgr2=0x7fa3f0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f01076c0 0x7fa3f0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3effff700 1 -- 172.21.15.112:0/3487274477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f0198f70 con 0x7fa3f0102630 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.975+0000 7fa3f487b700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f01076c0 0x7fa3f0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.976+0000 7fa3effff700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0102630 0x7fa3f019e610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d980 tx=0x7fa3e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.976+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0016070 con 0x7fa3f0102630 2024-05-10T10:38:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.976+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3e0004830 con 0x7fa3f0102630 2024-05-10T10:38:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.976+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f0199260 con 0x7fa3f0102630 2024-05-10T10:38:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.976+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0005440 con 0x7fa3f0102630 2024-05-10T10:38:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.976+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f006bbf0 con 0x7fa3f0102630 2024-05-10T10:38:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.977+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3f006c1a0 con 0x7fa3f0102630 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.978+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa3e00055e0 con 0x7fa3f0102630 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.979+0000 7fa3ed7fa700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3d80b2990 0x7fa3d80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.979+0000 7fa3ef7fe700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3d80b2990 0x7fa3d80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.979+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa3e00e2bc0 con 0x7fa3f0102630 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.979+0000 7fa3ed7fa700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] conn(0x7fa3d80bdbf0 0x7fa3d80c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.980+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 --> [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3e001ce50 con 0x7fa3d80bdbf0 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.980+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa3e00e2f70 con 0x7fa3f0102630 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.980+0000 7fa3ef7fe700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3d80b2990 0x7fa3d80b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0101f80 tx=0x7fa3e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.980+0000 7fa3f487b700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] conn(0x7fa3d80bdbf0 0x7fa3d80c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.980+0000 7fa3f487b700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] conn(0x7fa3d80bdbf0 0x7fa3d80c0090 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.981+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== osd.8 v2:172.21.15.195:6816/3432850695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3e001ce50 con 0x7fa3d80bdbf0 2024-05-10T10:38:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.998+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 --> [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3f006c5d0 con 0x7fa3d80bdbf0 2024-05-10T10:38:02.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.998+0000 7fa3ed7fa700 1 -- 172.21.15.112:0/3487274477 <== osd.8 v2:172.21.15.195:6816/3432850695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa3f006c5d0 con 0x7fa3d80bdbf0 2024-05-10T10:38:03.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] conn(0x7fa3d80bdbf0 msgr2=0x7fa3d80c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] conn(0x7fa3d80bdbf0 0x7fa3d80c0090 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3d80b2990 msgr2=0x7fa3d80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3d80b2990 0x7fa3d80b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0101f80 tx=0x7fa3e4009260 comp rx=0 tx=0).stop 2024-05-10T10:38:03.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0102630 msgr2=0x7fa3f019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 --2- 172.21.15.112:0/3487274477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0102630 0x7fa3f019e610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d980 tx=0x7fa3e000dc90 comp rx=0 tx=0).stop 2024-05-10T10:38:03.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f487b700 1 -- 172.21.15.112:0/3487274477 reap_dead start 2024-05-10T10:38:03.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 shutdown_connections 2024-05-10T10:38:03.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 >> 172.21.15.112:0/3487274477 conn(0x7fa3f0079370 msgr2=0x7fa3f0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:03.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 shutdown_connections 2024-05-10T10:38:03.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:02.999+0000 7fa3f62de700 1 -- 172.21.15.112:0/3487274477 wait complete. 2024-05-10T10:38:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:02 smithi148 ceph-mon[33568]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:02 smithi195 ceph-mon[33773]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 -- 172.21.15.112:0/3443693912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 msgr2=0x7f4e00108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 --2- 172.21.15.112:0/3443693912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 0x7f4e00108940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009990 tx=0x7f4df0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 -- 172.21.15.112:0/3443693912 shutdown_connections 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 --2- 172.21.15.112:0/3443693912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00102800 0x7f4e00110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 --2- 172.21.15.112:0/3443693912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 0x7f4e00108940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 --2- 172.21.15.112:0/3443693912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e00101e40 0x7f4e001022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 -- 172.21.15.112:0/3443693912 >> 172.21.15.112:0/3443693912 conn(0x7f4e000776c0 msgr2=0x7f4e00077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 -- 172.21.15.112:0/3443693912 shutdown_connections 2024-05-10T10:38:03.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.802+0000 7f4e06922700 1 -- 172.21.15.112:0/3443693912 wait complete. 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 Processor -- start 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 -- start start 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e00101e40 0x7f4e0019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00102800 0x7f4e0019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 0x7f4e001998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e00113520 con 0x7f4e00108560 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001133a0 con 0x7f4e00101e40 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.803+0000 7f4e06922700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e001136a0 con 0x7f4e00102800 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 0x7f4e001998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 0x7f4e001998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46710/0 (socket says 172.21.15.112:46710) 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4dfffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e00101e40 0x7f4e0019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 -- 172.21.15.112:0/3497385746 learned_addr learned my addr 172.21.15.112:0/3497385746 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4dff7fe700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00102800 0x7f4e0019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 -- 172.21.15.112:0/3497385746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00102800 msgr2=0x7f4e0019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e00102800 0x7f4e0019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 -- 172.21.15.112:0/3497385746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e00101e40 msgr2=0x7f4e0019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e00101e40 0x7f4e0019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 -- 172.21.15.112:0/3497385746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0019a080 con 0x7f4e00108560 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e04ebf700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 0x7f4e001998a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4df8007ee0 tx=0x7f4df800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8004bb0 con 0x7f4e00108560 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0019a370 con 0x7f4e00108560 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4df8004d50 con 0x7f4e00108560 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df80058b0 con 0x7f4e00108560 2024-05-10T10:38:03.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.804+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e001a6530 con 0x7f4e00108560 2024-05-10T10:38:03.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.805+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e00063c30 con 0x7f4e00108560 2024-05-10T10:38:03.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.807+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4df801a020 con 0x7f4e00108560 2024-05-10T10:38:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.807+0000 7f4dfd7fa700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4de80b2990 0x7f4de80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.808+0000 7f4dfffff700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4de80b2990 0x7f4de80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.808+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f4df80e2440 con 0x7f4e00108560 2024-05-10T10:38:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.808+0000 7f4dfd7fa700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] conn(0x7f4de80bdbf0 0x7f4de80c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.808+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 --> [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4df8005a50 con 0x7f4de80bdbf0 2024-05-10T10:38:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.808+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4df80e2890 con 0x7f4e00108560 2024-05-10T10:38:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.808+0000 7f4dfffff700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4de80b2990 0x7f4de80b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4df0006010 tx=0x7f4df0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.808+0000 7f4dff7fe700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] conn(0x7f4de80bdbf0 0x7f4de80c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.809+0000 7f4dff7fe700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] conn(0x7f4de80bdbf0 0x7f4de80c0090 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.810+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== osd.4 v2:172.21.15.148:6808/455926414 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4df8005a50 con 0x7f4de80bdbf0 2024-05-10T10:38:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.826+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 --> [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e000640b0 con 0x7f4de80bdbf0 2024-05-10T10:38:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.826+0000 7f4dfd7fa700 1 -- 172.21.15.112:0/3497385746 <== osd.4 v2:172.21.15.148:6808/455926414 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4e000640b0 con 0x7f4de80bdbf0 2024-05-10T10:38:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 >> [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] conn(0x7f4de80bdbf0 msgr2=0x7f4de80c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] conn(0x7f4de80bdbf0 0x7f4de80c0090 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4de80b2990 msgr2=0x7f4de80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4de80b2990 0x7f4de80b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4df0006010 tx=0x7f4df0005960 comp rx=0 tx=0).stop 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 msgr2=0x7f4e001998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 --2- 172.21.15.112:0/3497385746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e00108560 0x7f4e001998a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4df8007ee0 tx=0x7f4df800d3b0 comp rx=0 tx=0).stop 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e04ebf700 1 -- 172.21.15.112:0/3497385746 reap_dead start 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 shutdown_connections 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 >> 172.21.15.112:0/3497385746 conn(0x7f4e000776c0 msgr2=0x7f4e00100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 shutdown_connections 2024-05-10T10:38:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.827+0000 7f4e06922700 1 -- 172.21.15.112:0/3497385746 wait complete. 2024-05-10T10:38:04.569 INFO:teuthology.orchestra.run.smithi112.stdout:133143986254 2024-05-10T10:38:04.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-10T10:38:05.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:04 smithi112 ceph-mon[30100]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:05.114 INFO:teuthology.orchestra.run.smithi112.stdout:180388626500 2024-05-10T10:38:05.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-10T10:38:05.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 -- 172.21.15.112:0/840540533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 msgr2=0x7fc518108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:05.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 --2- 172.21.15.112:0/840540533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 0x7fc518108120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0099c0 tx=0x7fc50c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:38:05.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 -- 172.21.15.112:0/840540533 shutdown_connections 2024-05-10T10:38:05.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 --2- 172.21.15.112:0/840540533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc518108850 0x7fc5181103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 --2- 172.21.15.112:0/840540533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518103750 0x7fc518103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 --2- 172.21.15.112:0/840540533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 0x7fc518108120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 -- 172.21.15.112:0/840540533 >> 172.21.15.112:0/840540533 conn(0x7fc51807a600 msgr2=0x7fc51807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 -- 172.21.15.112:0/840540533 shutdown_connections 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.158+0000 7fc5202fc700 1 -- 172.21.15.112:0/840540533 wait complete. 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 Processor -- start 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 -- start start 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518103750 0x7fc5180757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 0x7fc518075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc518108850 0x7fc518076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181134e0 con 0x7fc518104100 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc518113360 con 0x7fc518103750 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc5202fc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc518113660 con 0x7fc518108850 2024-05-10T10:38:05.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc51d897700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 0x7fc518075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc51d897700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 0x7fc518075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46686/0 (socket says 172.21.15.112:46686) 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc51d897700 1 -- 172.21.15.112:0/928960135 learned_addr learned my addr 172.21.15.112:0/928960135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.159+0000 7fc51e098700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518103750 0x7fc5180757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51e899700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc518108850 0x7fc518076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51d897700 1 -- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc518108850 msgr2=0x7fc518076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51d897700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc518108850 0x7fc518076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51d897700 1 -- 172.21.15.112:0/928960135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518103750 msgr2=0x7fc5180757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51d897700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518103750 0x7fc5180757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51d897700 1 -- 172.21.15.112:0/928960135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5181a7f20 con 0x7fc518104100 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51e899700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc518108850 0x7fc518076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc51d897700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 0x7fc518075ce0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc50c006010 tx=0x7fc50c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c026070 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5181a81b0 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc50c003c50 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00fb10 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.160+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5181a8660 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.161+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc518063c30 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.162+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc50c029070 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.163+0000 7fc5177fe700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5000b2910 0x7fc5000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.163+0000 7fc51e098700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5000b2910 0x7fc5000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.163+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc50c0e3650 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.163+0000 7fc5177fe700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] conn(0x7fc5000bdb70 0x7fc5000c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.163+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 --> [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc50c02de50 con 0x7fc5000bdb70 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.163+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc50c0e3aa0 con 0x7fc518104100 2024-05-10T10:38:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.163+0000 7fc51e098700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5000b2910 0x7fc5000b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc50800f540 tx=0x7fc50800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.164+0000 7fc51e899700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] conn(0x7fc5000bdb70 0x7fc5000c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.164+0000 7fc51e899700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] conn(0x7fc5000bdb70 0x7fc5000c0010 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.165+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== osd.2 v2:172.21.15.195:6800/693285528 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc50c02de50 con 0x7fc5000bdb70 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.181+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 --> [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5180640b0 con 0x7fc5000bdb70 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc5177fe700 1 -- 172.21.15.112:0/928960135 <== osd.2 v2:172.21.15.195:6800/693285528 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc5180640b0 con 0x7fc5000bdb70 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] conn(0x7fc5000bdb70 msgr2=0x7fc5000c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc5202fc700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] conn(0x7fc5000bdb70 0x7fc5000c0010 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5000b2910 msgr2=0x7fc5000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc5202fc700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5000b2910 0x7fc5000b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc50800f540 tx=0x7fc50800b040 comp rx=0 tx=0).stop 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 msgr2=0x7fc518075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc5202fc700 1 --2- 172.21.15.112:0/928960135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc518104100 0x7fc518075ce0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc50c006010 tx=0x7fc50c004270 comp rx=0 tx=0).stop 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.182+0000 7fc51e899700 1 -- 172.21.15.112:0/928960135 reap_dead start 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.183+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 shutdown_connections 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.183+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 >> 172.21.15.112:0/928960135 conn(0x7fc51807a600 msgr2=0x7fc518101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.183+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 shutdown_connections 2024-05-10T10:38:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:03.183+0000 7fc5202fc700 1 -- 172.21.15.112:0/928960135 wait complete. 2024-05-10T10:38:05.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:04 smithi148 ceph-mon[33568]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:04 smithi195 ceph-mon[33773]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:06.111 INFO:teuthology.orchestra.run.smithi112.stdout:77309411417 2024-05-10T10:38:06.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-10T10:38:06.967 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:06 smithi112 ceph-mon[30100]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:38:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:06 smithi148 ceph-mon[33568]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:38:07.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:06 smithi195 ceph-mon[33773]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/1619513433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d81076c0 msgr2=0x7fb8d810f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 --2- 172.21.15.112:0/1619513433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d81076c0 0x7fb8d810f290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000b0d0 tx=0x7fb8d000b3e0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/1619513433 shutdown_connections 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 --2- 172.21.15.112:0/1619513433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d81076c0 0x7fb8d810f290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 --2- 172.21.15.112:0/1619513433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8d8102630 0x7fb8d8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 --2- 172.21.15.112:0/1619513433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d8102fe0 0x7fb8d8106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/1619513433 >> 172.21.15.112:0/1619513433 conn(0x7fb8d8079350 msgr2=0x7fb8d8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/1619513433 shutdown_connections 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.023+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/1619513433 wait complete. 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 Processor -- start 2024-05-10T10:38:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 -- start start 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d8102630 0x7fb8d819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d8102fe0 0x7fb8d819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8d81076c0 0x7fb8d81a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8112480 con 0x7fb8d8102fe0 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8112300 con 0x7fb8d8102630 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8dcdc3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8112600 con 0x7fb8d81076c0 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.024+0000 7fb8d5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d8102fe0 0x7fb8d819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d8102fe0 0x7fb8d819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48348/0 (socket says 172.21.15.112:48348) 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 -- 172.21.15.112:0/976471106 learned_addr learned my addr 172.21.15.112:0/976471106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d659c700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d8102630 0x7fb8d819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d6d9d700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8d81076c0 0x7fb8d81a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 -- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8d81076c0 msgr2=0x7fb8d81a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8d81076c0 0x7fb8d81a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 -- 172.21.15.112:0/976471106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d8102630 msgr2=0x7fb8d819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:07.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d8102630 0x7fb8d819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 -- 172.21.15.112:0/976471106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d81a5ee0 con 0x7fb8d8102fe0 2024-05-10T10:38:07.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d6d9d700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8d81076c0 0x7fb8d81a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:07.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d659c700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d8102630 0x7fb8d819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:07.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8d5d9b700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d8102fe0 0x7fb8d819ae60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c8f0 tx=0x7fb8cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:07.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc007b00 con 0x7fb8d8102fe0 2024-05-10T10:38:07.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d81a6110 con 0x7fb8d8102fe0 2024-05-10T10:38:07.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8cc007ca0 con 0x7fb8d8102fe0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc018660 con 0x7fb8d8102fe0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.025+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d81a6650 con 0x7fb8d8102fe0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.026+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb8d8063bb0 con 0x7fb8d8102fe0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.027+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb8cc00f450 con 0x7fb8d8102fe0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.028+0000 7fb8c77fe700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb8c00b2910 0x7fb8c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.028+0000 7fb8d659c700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb8c00b2910 0x7fb8c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.028+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb8cc0e29d0 con 0x7fb8d8102fe0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.028+0000 7fb8c77fe700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] conn(0x7fb8c00bdb70 0x7fb8c00c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.028+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 --> [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8cc015e70 con 0x7fb8c00bdb70 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.028+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb8cc0e2e20 con 0x7fb8d8102fe0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.029+0000 7fb8d659c700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb8c00b2910 0x7fb8c00b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80099d0 tx=0x7fb8c8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.029+0000 7fb8d6d9d700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] conn(0x7fb8c00bdb70 0x7fb8c00c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.029+0000 7fb8d6d9d700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] conn(0x7fb8c00bdb70 0x7fb8c00c0010 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.030+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== osd.11 v2:172.21.15.195:6824/1653173109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb8cc015e70 con 0x7fb8c00bdb70 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.046+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 --> [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb8d8064030 con 0x7fb8c00bdb70 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8c77fe700 1 -- 172.21.15.112:0/976471106 <== osd.11 v2:172.21.15.195:6824/1653173109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb8d8064030 con 0x7fb8c00bdb70 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] conn(0x7fb8c00bdb70 msgr2=0x7fb8c00c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] conn(0x7fb8c00bdb70 0x7fb8c00c0010 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb8c00b2910 msgr2=0x7fb8c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb8c00b2910 0x7fb8c00b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80099d0 tx=0x7fb8c8008040 comp rx=0 tx=0).stop 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d8102fe0 msgr2=0x7fb8d819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 --2- 172.21.15.112:0/976471106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d8102fe0 0x7fb8d819ae60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c8f0 tx=0x7fb8cc00ccb0 comp rx=0 tx=0).stop 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8d6d9d700 1 -- 172.21.15.112:0/976471106 reap_dead start 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 shutdown_connections 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 >> 172.21.15.112:0/976471106 conn(0x7fb8d8079350 msgr2=0x7fb8d81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.047+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 shutdown_connections 2024-05-10T10:38:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:06.048+0000 7fb8dcdc3700 1 -- 172.21.15.112:0/976471106 wait complete. 2024-05-10T10:38:08.046 INFO:teuthology.orchestra.run.smithi112.stdout:304942678059 2024-05-10T10:38:08.047 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-10T10:38:08.592 INFO:teuthology.orchestra.run.smithi112.stdout:231928234043 2024-05-10T10:38:08.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-10T10:38:09.184 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:08 smithi112 ceph-mon[30100]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 -- 172.21.15.112:0/1271116662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8102620 msgr2=0x7f91e8102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 --2- 172.21.15.112:0/1271116662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8102620 0x7f91e8102a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f91d80099c0 tx=0x7f91d8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 -- 172.21.15.112:0/1271116662 shutdown_connections 2024-05-10T10:38:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 --2- 172.21.15.112:0/1271116662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8107700 0x7f91e810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 --2- 172.21.15.112:0/1271116662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8102620 0x7f91e8102a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 --2- 172.21.15.112:0/1271116662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91e8102fd0 0x7f91e8107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 -- 172.21.15.112:0/1271116662 >> 172.21.15.112:0/1271116662 conn(0x7f91e8079440 msgr2=0x7f91e8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 -- 172.21.15.112:0/1271116662 shutdown_connections 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.185+0000 7f91ee519700 1 -- 172.21.15.112:0/1271116662 wait complete. 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 Processor -- start 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 -- start start 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8102620 0x7f91e819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91e8102fd0 0x7f91e819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8107700 0x7f91e81a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8112440 con 0x7f91e8107700 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81122c0 con 0x7f91e8102fd0 2024-05-10T10:38:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ee519700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81125c0 con 0x7f91e8102620 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ecab6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8107700 0x7f91e81a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ecab6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8107700 0x7f91e81a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48356/0 (socket says 172.21.15.112:48356) 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91ecab6700 1 -- 172.21.15.112:0/1393175497 learned_addr learned my addr 172.21.15.112:0/1393175497 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.186+0000 7f91e7fff700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8102620 0x7f91e819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91e77fe700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91e8102fd0 0x7f91e819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ecab6700 1 -- 172.21.15.112:0/1393175497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8102620 msgr2=0x7f91e819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ecab6700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8102620 0x7f91e819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ecab6700 1 -- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91e8102fd0 msgr2=0x7f91e819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ecab6700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91e8102fd0 0x7f91e819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ecab6700 1 -- 172.21.15.112:0/1393175497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e81a5e40 con 0x7f91e8107700 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91e77fe700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91e8102fd0 0x7f91e819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ecab6700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8107700 0x7f91e81a5900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f91e0007d70 tx=0x7f91e000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0016020 con 0x7f91e8107700 2024-05-10T10:38:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91e000ad40 con 0x7f91e8107700 2024-05-10T10:38:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e81a6070 con 0x7f91e8107700 2024-05-10T10:38:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e00115e0 con 0x7f91e8107700 2024-05-10T10:38:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.187+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81a6530 con 0x7f91e8107700 2024-05-10T10:38:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.188+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91e8063bb0 con 0x7f91e8107700 2024-05-10T10:38:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.191+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f91e0019020 con 0x7f91e8107700 2024-05-10T10:38:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.191+0000 7f91e57fa700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91d00b2990 0x7f91d00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.192+0000 7f91e7fff700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91d00b2990 0x7f91d00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.192+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91e00e25b0 con 0x7f91e8107700 2024-05-10T10:38:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.192+0000 7f91e57fa700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] conn(0x7f91d00bdbf0 0x7f91d00c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.192+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 --> [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91e0004880 con 0x7f91d00bdbf0 2024-05-10T10:38:09.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.192+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f91e00e9050 con 0x7f91e8107700 2024-05-10T10:38:09.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.192+0000 7f91e7fff700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91d00b2990 0x7f91d00b4e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91d80058e0 tx=0x7f91d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.192+0000 7f91e77fe700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] conn(0x7f91d00bdbf0 0x7f91d00c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.193+0000 7f91e77fe700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] conn(0x7f91d00bdbf0 0x7f91d00c0090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.193+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== osd.7 v2:172.21.15.148:6816/3546374134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91e0004880 con 0x7f91d00bdbf0 2024-05-10T10:38:09.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.213+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 --> [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91e8064030 con 0x7f91d00bdbf0 2024-05-10T10:38:09.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.214+0000 7f91e57fa700 1 -- 172.21.15.112:0/1393175497 <== osd.7 v2:172.21.15.148:6816/3546374134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91e8064030 con 0x7f91d00bdbf0 2024-05-10T10:38:09.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] conn(0x7f91d00bdbf0 msgr2=0x7f91d00c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] conn(0x7f91d00bdbf0 0x7f91d00c0090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91d00b2990 msgr2=0x7f91d00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91d00b2990 0x7f91d00b4e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91d80058e0 tx=0x7f91d8005fb0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8107700 msgr2=0x7f91e81a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 --2- 172.21.15.112:0/1393175497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e8107700 0x7f91e81a5900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f91e0007d70 tx=0x7f91e000a340 comp rx=0 tx=0).stop 2024-05-10T10:38:09.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ecab6700 1 -- 172.21.15.112:0/1393175497 reap_dead start 2024-05-10T10:38:09.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 shutdown_connections 2024-05-10T10:38:09.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.215+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 >> 172.21.15.112:0/1393175497 conn(0x7f91e8079440 msgr2=0x7f91e8100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.216+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 shutdown_connections 2024-05-10T10:38:09.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:09.216+0000 7f91ee519700 1 -- 172.21.15.112:0/1393175497 wait complete. 2024-05-10T10:38:09.224 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:09.225 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:08 smithi148 ceph-mon[33568]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:08 smithi195 ceph-mon[33773]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:09.596 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:09.604 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:10.749 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:11.176 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:10 smithi112 ceph-mon[30100]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T10:38:11.204 INFO:teuthology.orchestra.run.smithi112.stdout:206158430273 2024-05-10T10:38:11.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-10T10:38:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:10 smithi148 ceph-mon[33568]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T10:38:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:10 smithi195 ceph-mon[33773]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T10:38:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 -- 172.21.15.112:0/941012684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c109540 msgr2=0x7eff8c109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 --2- 172.21.15.112:0/941012684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c109540 0x7eff8c109920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7eff88009990 tx=0x7eff88009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 -- 172.21.15.112:0/941012684 shutdown_connections 2024-05-10T10:38:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 --2- 172.21.15.112:0/941012684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff8c06c570 0x7eff8c110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 --2- 172.21.15.112:0/941012684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c109540 0x7eff8c109920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 --2- 172.21.15.112:0/941012684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8c06bbb0 0x7eff8c06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 -- 172.21.15.112:0/941012684 >> 172.21.15.112:0/941012684 conn(0x7eff8c078410 msgr2=0x7eff8c078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 -- 172.21.15.112:0/941012684 shutdown_connections 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 -- 172.21.15.112:0/941012684 wait complete. 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 Processor -- start 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.784+0000 7eff93a71700 1 -- start start 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c06bbb0 0x7eff8c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8c06c570 0x7eff8c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff8c109540 0x7eff8c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c113550 con 0x7eff8c06bbb0 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1133d0 con 0x7eff8c109540 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1136d0 con 0x7eff8c06c570 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c06bbb0 0x7eff8c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c06bbb0 0x7eff8c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48372/0 (socket says 172.21.15.112:48372) 2024-05-10T10:38:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 -- 172.21.15.112:0/3661370466 learned_addr learned my addr 172.21.15.112:0/3661370466 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9200e700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff8c109540 0x7eff8c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 -- 172.21.15.112:0/3661370466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8c06c570 msgr2=0x7eff8c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9100c700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8c06c570 0x7eff8c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8c06c570 0x7eff8c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 -- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff8c109540 msgr2=0x7eff8c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff8c109540 0x7eff8c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 -- 172.21.15.112:0/3661370466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c19a070 con 0x7eff8c06bbb0 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9100c700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff8c06c570 0x7eff8c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9200e700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff8c109540 0x7eff8c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff9180d700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c06bbb0 0x7eff8c19f750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7eff88000c00 tx=0x7eff88009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88027070 con 0x7eff8c06bbb0 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff8800fc70 con 0x7eff8c06bbb0 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c19a300 con 0x7eff8c06bbb0 2024-05-10T10:38:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88022d40 con 0x7eff8c06bbb0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.785+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1a6620 con 0x7eff8c06bbb0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.786+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff8c063bb0 con 0x7eff8c06bbb0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.788+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7eff8802a070 con 0x7eff8c06bbb0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.788+0000 7eff82ffd700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7eff780b2910 0x7eff780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.788+0000 7eff9100c700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7eff780b2910 0x7eff780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.788+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7eff880e3e90 con 0x7eff8c06bbb0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.789+0000 7eff82ffd700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] conn(0x7eff780bdb70 0x7eff780c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.789+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 --> [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff8800eea0 con 0x7eff780bdb70 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.789+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7eff880e42e0 con 0x7eff8c06bbb0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.789+0000 7eff9100c700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7eff780b2910 0x7eff780b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff8c078b50 tx=0x7eff7c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.789+0000 7eff9200e700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] conn(0x7eff780bdb70 0x7eff780c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.790+0000 7eff9200e700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] conn(0x7eff780bdb70 0x7eff780c0010 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.790+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== osd.10 v2:172.21.15.148:6824/2277563117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff8800eea0 con 0x7eff780bdb70 2024-05-10T10:38:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.807+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 --> [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff8c064030 con 0x7eff780bdb70 2024-05-10T10:38:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.807+0000 7eff82ffd700 1 -- 172.21.15.112:0/3661370466 <== osd.10 v2:172.21.15.148:6824/2277563117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7eff8c064030 con 0x7eff780bdb70 2024-05-10T10:38:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] conn(0x7eff780bdb70 msgr2=0x7eff780c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] conn(0x7eff780bdb70 0x7eff780c0010 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7eff780b2910 msgr2=0x7eff780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7eff780b2910 0x7eff780b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff8c078b50 tx=0x7eff7c008040 comp rx=0 tx=0).stop 2024-05-10T10:38:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c06bbb0 msgr2=0x7eff8c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:12.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 --2- 172.21.15.112:0/3661370466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff8c06bbb0 0x7eff8c19f750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7eff88000c00 tx=0x7eff88009f90 comp rx=0 tx=0).stop 2024-05-10T10:38:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff9200e700 1 -- 172.21.15.112:0/3661370466 reap_dead start 2024-05-10T10:38:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 shutdown_connections 2024-05-10T10:38:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 >> 172.21.15.112:0/3661370466 conn(0x7eff8c078410 msgr2=0x7eff8c1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 shutdown_connections 2024-05-10T10:38:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:12.808+0000 7eff93a71700 1 -- 172.21.15.112:0/3661370466 wait complete. 2024-05-10T10:38:13.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:12 smithi112 ceph-mon[30100]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-10T10:38:13.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:12 smithi148 ceph-mon[33568]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-10T10:38:13.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:12 smithi195 ceph-mon[33773]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-10T10:38:13.791 INFO:teuthology.orchestra.run.smithi112.stdout:279172874289 2024-05-10T10:38:13.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-10T10:38:14.924 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:15.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:14 smithi112 ceph-mon[30100]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T10:38:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:14 smithi148 ceph-mon[33568]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T10:38:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:14 smithi195 ceph-mon[33773]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T10:38:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 -- 172.21.15.112:0/3787560296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 msgr2=0x7fade01087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 --2- 172.21.15.112:0/3787560296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade01087f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009990 tx=0x7fadd0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 -- 172.21.15.112:0/3787560296 shutdown_connections 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 --2- 172.21.15.112:0/3787560296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade0102d60 0x7fade010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 --2- 172.21.15.112:0/3787560296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade01087f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 --2- 172.21.15.112:0/3787560296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade01023a0 0x7fade0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 -- 172.21.15.112:0/3787560296 >> 172.21.15.112:0/3787560296 conn(0x7fade0079350 msgr2=0x7fade0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 -- 172.21.15.112:0/3787560296 shutdown_connections 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.567+0000 7fade62cd700 1 -- 172.21.15.112:0/3787560296 wait complete. 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 Processor -- start 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 -- start start 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade01023a0 0x7fade019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade0102d60 0x7fade019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade01123e0 con 0x7fade0108410 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fade0112260 con 0x7fade0102d60 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade62cd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fade0112560 con 0x7fade01023a0 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade486a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade486a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade0199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34460/0 (socket says 172.21.15.112:34460) 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7fade486a700 1 -- 172.21.15.112:0/1740960849 learned_addr learned my addr 172.21.15.112:0/1740960849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.568+0000 7faddf7fe700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade0102d60 0x7fade019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7faddffff700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade01023a0 0x7fade019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade486a700 1 -- 172.21.15.112:0/1740960849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade01023a0 msgr2=0x7fade019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade486a700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade01023a0 0x7fade019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade486a700 1 -- 172.21.15.112:0/1740960849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade0102d60 msgr2=0x7fade019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade486a700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade0102d60 0x7fade019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade486a700 1 -- 172.21.15.112:0/1740960849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade0199ff0 con 0x7fade0108410 2024-05-10T10:38:15.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7faddffff700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade01023a0 0x7fade019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade486a700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade0199890 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fadd8007ee0 tx=0x7fadd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7faddd7fa700 1 -- 172.21.15.112:0/1740960849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd800dbd0 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7faddd7fa700 1 -- 172.21.15.112:0/1740960849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadd800dd70 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade019a2e0 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7faddd7fa700 1 -- 172.21.15.112:0/1740960849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80058d0 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.569+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01a65a0 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.570+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade006a190 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.571+0000 7faddd7fa700 1 -- 172.21.15.112:0/1740960849 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fadd801a020 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.572+0000 7faddd7fa700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fadc80b2910 0x7fadc80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.572+0000 7faddffff700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fadc80b2910 0x7fadc80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.572+0000 7faddd7fa700 1 -- 172.21.15.112:0/1740960849 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fadd80e1e80 con 0x7fade0108410 2024-05-10T10:38:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.573+0000 7faddffff700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fadc80b2910 0x7fadc80b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fadd0003760 tx=0x7fadd0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:15.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.577+0000 7faddd7fa700 1 -- 172.21.15.112:0/1740960849 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fadd8010070 con 0x7fade0108410 2024-05-10T10:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.793+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fade01a6770 con 0x7fade0108410 2024-05-10T10:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.793+0000 7faddd7fa700 1 -- 172.21.15.112:0/1740960849 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fadd80a96b0 con 0x7fade0108410 2024-05-10T10:38:15.795 INFO:teuthology.orchestra.run.smithi112.stdout:154618822732 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.795+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fadc80b2910 msgr2=0x7fadc80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.795+0000 7fade62cd700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fadc80b2910 0x7fadc80b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fadd0003760 tx=0x7fadd0005fb0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.795+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 msgr2=0x7fade0199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.795+0000 7fade62cd700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade0199890 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fadd8007ee0 tx=0x7fadd800d3b0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 shutdown_connections 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0108410 0x7fade0199890 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fadc80b2910 0x7fadc80b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade01023a0 0x7fade019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 --2- 172.21.15.112:0/1740960849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fade0102d60 0x7fade019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 >> 172.21.15.112:0/1740960849 conn(0x7fade0079350 msgr2=0x7fade0106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 shutdown_connections 2024-05-10T10:38:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:15.796+0000 7fade62cd700 1 -- 172.21.15.112:0/1740960849 wait complete. 2024-05-10T10:38:16.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:15 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1740960849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T10:38:16.195 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:15 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1740960849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T10:38:16.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:15 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1740960849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T10:38:17.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:16 smithi112 ceph-mon[30100]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:17.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:16 smithi148 ceph-mon[33568]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:17.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:16 smithi195 ceph-mon[33773]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:17.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.625+0000 7f7dc3486700 1 -- 172.21.15.112:0/1986009166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 msgr2=0x7f7dbc10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.625+0000 7f7dc3486700 1 --2- 172.21.15.112:0/1986009166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc10f290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7db400b0d0 tx=0x7f7db400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.625+0000 7f7dc3486700 1 -- 172.21.15.112:0/1986009166 shutdown_connections 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.625+0000 7f7dc3486700 1 --2- 172.21.15.112:0/1986009166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc10f290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.625+0000 7f7dc3486700 1 --2- 172.21.15.112:0/1986009166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dbc102630 0x7f7dbc102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.625+0000 7f7dc3486700 1 --2- 172.21.15.112:0/1986009166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc102fe0 0x7f7dbc106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.625+0000 7f7dc3486700 1 -- 172.21.15.112:0/1986009166 >> 172.21.15.112:0/1986009166 conn(0x7f7dbc079350 msgr2=0x7f7dbc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 -- 172.21.15.112:0/1986009166 shutdown_connections 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 -- 172.21.15.112:0/1986009166 wait complete. 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 Processor -- start 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 -- start start 2024-05-10T10:38:17.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc102630 0x7f7dbc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dbc102fe0 0x7f7dbc19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc112430 con 0x7f7dbc1076c0 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc1122b0 con 0x7f7dbc102630 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.626+0000 7f7dc3486700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc1125b0 con 0x7f7dbc102fe0 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34476/0 (socket says 172.21.15.112:34476) 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 -- 172.21.15.112:0/481223809 learned_addr learned my addr 172.21.15.112:0/481223809 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1222700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc102630 0x7f7dbc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc0a21700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dbc102fe0 0x7f7dbc19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 -- 172.21.15.112:0/481223809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dbc102fe0 msgr2=0x7f7dbc19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dbc102fe0 0x7f7dbc19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 -- 172.21.15.112:0/481223809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc102630 msgr2=0x7f7dbc19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc102630 0x7f7dbc19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 -- 172.21.15.112:0/481223809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc19a020 con 0x7f7dbc1076c0 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc1a23700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc1998c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009850 tx=0x7f7db4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7db27fc700 1 -- 172.21.15.112:0/481223809 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db401f070 con 0x7f7dbc1076c0 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7db27fc700 1 -- 172.21.15.112:0/481223809 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db4005500 con 0x7f7dbc1076c0 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7db27fc700 1 -- 172.21.15.112:0/481223809 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db401b9a0 con 0x7f7dbc1076c0 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc19a2b0 con 0x7f7dbc1076c0 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.627+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1a65d0 con 0x7f7dbc1076c0 2024-05-10T10:38:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.628+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc103ac0 con 0x7f7dbc1076c0 2024-05-10T10:38:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.630+0000 7f7db27fc700 1 -- 172.21.15.112:0/481223809 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7db4022070 con 0x7f7dbc1076c0 2024-05-10T10:38:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.630+0000 7f7db27fc700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7da80b2910 0x7f7da80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.631+0000 7f7dc1222700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7da80b2910 0x7f7da80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.631+0000 7f7db27fc700 1 -- 172.21.15.112:0/481223809 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7db40e3ac0 con 0x7f7dbc1076c0 2024-05-10T10:38:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.631+0000 7f7dc1222700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7da80b2910 0x7f7da80b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc19aa00 tx=0x7f7db800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:17.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.634+0000 7f7db27fc700 1 -- 172.21.15.112:0/481223809 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7db40ab380 con 0x7f7dbc1076c0 2024-05-10T10:38:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.852+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7dbc02e510 con 0x7f7dbc1076c0 2024-05-10T10:38:17.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.852+0000 7f7db27fc700 1 -- 172.21.15.112:0/481223809 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7db40aac80 con 0x7f7dbc1076c0 2024-05-10T10:38:17.854 INFO:teuthology.orchestra.run.smithi112.stdout:55834574944 2024-05-10T10:38:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7da80b2910 msgr2=0x7f7da80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7da80b2910 0x7f7da80b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc19aa00 tx=0x7f7db800b3c0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 msgr2=0x7f7dbc1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:17.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc1998c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009850 tx=0x7f7db4007d90 comp rx=0 tx=0).stop 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 shutdown_connections 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dbc1076c0 0x7f7dbc1998c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc102630 0x7f7dbc19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7da80b2910 0x7f7da80b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 --2- 172.21.15.112:0/481223809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dbc102fe0 0x7f7dbc19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 >> 172.21.15.112:0/481223809 conn(0x7f7dbc079350 msgr2=0x7f7dbc06bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 shutdown_connections 2024-05-10T10:38:17.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:17.855+0000 7f7dc3486700 1 -- 172.21.15.112:0/481223809 wait complete. 2024-05-10T10:38:18.076 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:17 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/481223809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T10:38:18.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:17 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/481223809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T10:38:18.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:17 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/481223809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T10:38:19.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:18 smithi112 ceph-mon[30100]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-10T10:38:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:18 smithi148 ceph-mon[33568]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-10T10:38:19.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:18 smithi195 ceph-mon[33773]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-10T10:38:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 -- 172.21.15.112:0/794677450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c102450 msgr2=0x7f283c110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 --2- 172.21.15.112:0/794677450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c102450 0x7f283c110340 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f283400b0a0 tx=0x7f283400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 -- 172.21.15.112:0/794677450 shutdown_connections 2024-05-10T10:38:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 --2- 172.21.15.112:0/794677450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c102450 0x7f283c110340 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 --2- 172.21.15.112:0/794677450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f283c109520 0x7f283c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 --2- 172.21.15.112:0/794677450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f283c101a90 0x7f283c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 -- 172.21.15.112:0/794677450 >> 172.21.15.112:0/794677450 conn(0x7f283c078a10 msgr2=0x7f283c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 -- 172.21.15.112:0/794677450 shutdown_connections 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.515+0000 7f2841270700 1 -- 172.21.15.112:0/794677450 wait complete. 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 Processor -- start 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 -- start start 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f283c101a90 0x7f283c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f283c102450 0x7f283c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c109520 0x7f283c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f283c113560 con 0x7f283c109520 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1133e0 con 0x7f283c102450 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f2841270700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1136e0 con 0x7f283c101a90 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c109520 0x7f283c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c109520 0x7f283c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34502/0 (socket says 172.21.15.112:34502) 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f283a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f283c102450 0x7f283c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f283b7fe700 1 -- 172.21.15.112:0/1739048638 learned_addr learned my addr 172.21.15.112:0/1739048638 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.516+0000 7f283affd700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f283c101a90 0x7f283c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f283b7fe700 1 -- 172.21.15.112:0/1739048638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f283c101a90 msgr2=0x7f283c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f283b7fe700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f283c101a90 0x7f283c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f283b7fe700 1 -- 172.21.15.112:0/1739048638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f283c102450 msgr2=0x7f283c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f283b7fe700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f283c102450 0x7f283c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f283b7fe700 1 -- 172.21.15.112:0/1739048638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c19a000 con 0x7f283c109520 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f283b7fe700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c109520 0x7f283c199820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28340062a0 tx=0x7f2834004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f2823fff700 1 -- 172.21.15.112:0/1739048638 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834021070 con 0x7f283c109520 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f2823fff700 1 -- 172.21.15.112:0/1739048638 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2834007c00 con 0x7f283c109520 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c19a290 con 0x7f283c109520 2024-05-10T10:38:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f2823fff700 1 -- 172.21.15.112:0/1739048638 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340052a0 con 0x7f283c109520 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.517+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1a6500 con 0x7f283c109520 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.518+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c104db0 con 0x7f283c109520 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.519+0000 7f2823fff700 1 -- 172.21.15.112:0/1739048638 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2834024070 con 0x7f283c109520 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.520+0000 7f2823fff700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28240b2910 0x7f28240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.520+0000 7f283affd700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28240b2910 0x7f28240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.520+0000 7f2823fff700 1 -- 172.21.15.112:0/1739048638 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f283400f020 con 0x7f283c109520 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.521+0000 7f283affd700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28240b2910 0x7f28240b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f282c00ac50 tx=0x7f282c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:19.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.524+0000 7f2823fff700 1 -- 172.21.15.112:0/1739048638 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f283402b020 con 0x7f283c109520 2024-05-10T10:38:19.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.742+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f283c19ad50 con 0x7f283c109520 2024-05-10T10:38:19.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.743+0000 7f2823fff700 1 -- 172.21.15.112:0/1739048638 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f28340abb20 con 0x7f283c109520 2024-05-10T10:38:19.744 INFO:teuthology.orchestra.run.smithi112.stdout:133143986257 2024-05-10T10:38:19.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28240b2910 msgr2=0x7f28240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28240b2910 0x7f28240b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f282c00ac50 tx=0x7f282c00a380 comp rx=0 tx=0).stop 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c109520 msgr2=0x7f283c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c109520 0x7f283c199820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28340062a0 tx=0x7f2834004e20 comp rx=0 tx=0).stop 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 shutdown_connections 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c109520 0x7f283c199820 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28240b2910 0x7f28240b4dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f283c101a90 0x7f283c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 --2- 172.21.15.112:0/1739048638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f283c102450 0x7f283c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 >> 172.21.15.112:0/1739048638 conn(0x7f283c078a10 msgr2=0x7f283c100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 shutdown_connections 2024-05-10T10:38:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.745+0000 7f2841270700 1 -- 172.21.15.112:0/1739048638 wait complete. 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.758+0000 7fb1cde9f700 1 -- 172.21.15.112:0/177090260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103540 msgr2=0x7fb1c81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.758+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/177090260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103540 0x7fb1c81039c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0099c0 tx=0x7fb1bc009cd0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.758+0000 7fb1cde9f700 1 -- 172.21.15.112:0/177090260 shutdown_connections 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.758+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/177090260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1c8103f00 0x7fb1c81103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.758+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/177090260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c81095e0 0x7fb1c81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.758+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/177090260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103540 0x7fb1c81039c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.758+0000 7fb1cde9f700 1 -- 172.21.15.112:0/177090260 >> 172.21.15.112:0/177090260 conn(0x7fb1c807a650 msgr2=0x7fb1c807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.759+0000 7fb1cde9f700 1 -- 172.21.15.112:0/177090260 shutdown_connections 2024-05-10T10:38:19.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.759+0000 7fb1cde9f700 1 -- 172.21.15.112:0/177090260 wait complete. 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.759+0000 7fb1cde9f700 1 Processor -- start 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.759+0000 7fb1cde9f700 1 -- start start 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1cde9f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1c8103540 0x7fb1c80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1cde9f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103f00 0x7fb1c8075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1cde9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c81095e0 0x7fb1c8076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1cde9f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81135f0 con 0x7fb1c8103f00 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1cde9f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8113470 con 0x7fb1c81095e0 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1cde9f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8113770 con 0x7fb1c8103540 2024-05-10T10:38:19.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103f00 0x7fb1c8075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103f00 0x7fb1c8075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34510/0 (socket says 172.21.15.112:34510) 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 -- 172.21.15.112:0/1354680730 learned_addr learned my addr 172.21.15.112:0/1354680730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c77fe700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1c8103540 0x7fb1c80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 -- 172.21.15.112:0/1354680730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1c8103540 msgr2=0x7fb1c80757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c7fff700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c81095e0 0x7fb1c8076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1c8103540 0x7fb1c80757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 -- 172.21.15.112:0/1354680730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c81095e0 msgr2=0x7fb1c8076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c81095e0 0x7fb1c8076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 -- 172.21.15.112:0/1354680730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c81a8010 con 0x7fb1c8103f00 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c7fff700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c81095e0 0x7fb1c8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c6ffd700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103f00 0x7fb1c8075ce0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006010 tx=0x7fb1bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.760+0000 7fb1c4ff9700 1 -- 172.21.15.112:0/1354680730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc026070 con 0x7fb1c8103f00 2024-05-10T10:38:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.761+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c81a82a0 con 0x7fb1c8103f00 2024-05-10T10:38:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.761+0000 7fb1c4ff9700 1 -- 172.21.15.112:0/1354680730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1bc00f460 con 0x7fb1c8103f00 2024-05-10T10:38:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.761+0000 7fb1c4ff9700 1 -- 172.21.15.112:0/1354680730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc009e60 con 0x7fb1c8103f00 2024-05-10T10:38:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.761+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81a8750 con 0x7fb1c8103f00 2024-05-10T10:38:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.762+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c8104c50 con 0x7fb1c8103f00 2024-05-10T10:38:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.763+0000 7fb1c4ff9700 1 -- 172.21.15.112:0/1354680730 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb1bc029070 con 0x7fb1c8103f00 2024-05-10T10:38:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.764+0000 7fb1c4ff9700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb1b00b2990 0x7fb1b00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.764+0000 7fb1c4ff9700 1 -- 172.21.15.112:0/1354680730 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb1bc0e3200 con 0x7fb1c8103f00 2024-05-10T10:38:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.764+0000 7fb1c77fe700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb1b00b2990 0x7fb1b00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.764+0000 7fb1c77fe700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb1b00b2990 0x7fb1b00b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8005950 tx=0x7fb1b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.768+0000 7fb1c4ff9700 1 -- 172.21.15.112:0/1354680730 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb1bc030080 con 0x7fb1c8103f00 2024-05-10T10:38:19.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/2183392668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 msgr2=0x7f3f98102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 --2- 172.21.15.112:0/2183392668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 0x7f3f98102820 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009990 tx=0x7f3f88009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/2183392668 shutdown_connections 2024-05-10T10:38:19.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 --2- 172.21.15.112:0/2183392668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f98102d60 0x7f3f9810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 --2- 172.21.15.112:0/2183392668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f98108410 0x7f3f981087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 --2- 172.21.15.112:0/2183392668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 0x7f3f98102820 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/2183392668 >> 172.21.15.112:0/2183392668 conn(0x7f3f98079350 msgr2=0x7f3f98079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/2183392668 shutdown_connections 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.931+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/2183392668 wait complete. 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.932+0000 7f3f9f3d3700 1 Processor -- start 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.932+0000 7f3f9f3d3700 1 -- start start 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.932+0000 7f3f9f3d3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 0x7f3f9819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.932+0000 7f3f9f3d3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f98102d60 0x7f3f9819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.932+0000 7f3f9f3d3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f98108410 0x7f3f98198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9f3d3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98112400 con 0x7f3f981023a0 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9f3d3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98112280 con 0x7f3f98108410 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9f3d3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98112580 con 0x7f3f98102d60 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 0x7f3f9819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 0x7f3f9819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34530/0 (socket says 172.21.15.112:34530) 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 -- 172.21.15.112:0/1726445234 learned_addr learned my addr 172.21.15.112:0/1726445234 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d970700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f98108410 0x7f3f98198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9c96e700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f98102d60 0x7f3f9819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 -- 172.21.15.112:0/1726445234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f98102d60 msgr2=0x7f3f9819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f98102d60 0x7f3f9819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 -- 172.21.15.112:0/1726445234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f98108410 msgr2=0x7f3f98198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f98108410 0x7f3f98198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 -- 172.21.15.112:0/1726445234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f98198fe0 con 0x7f3f981023a0 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9c96e700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f98102d60 0x7f3f9819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9d16f700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 0x7f3f9819e680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400c970 tx=0x7f3f9400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94007aa0 con 0x7f3f981023a0 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f94004cc0 con 0x7f3f981023a0 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981992d0 con 0x7f3f981023a0 2024-05-10T10:38:19.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94005500 con 0x7f3f981023a0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.933+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f981a54e0 con 0x7f3f981023a0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.934+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f98063c30 con 0x7f3f981023a0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.936+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3f94007c40 con 0x7f3f981023a0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.937+0000 7f3f8e7fc700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f840b2990 0x7f3f840b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.937+0000 7f3f9c96e700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f840b2990 0x7f3f840b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.937+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3f940e2940 con 0x7f3f981023a0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.937+0000 7f3f9c96e700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f840b2990 0x7f3f840b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800b5c0 tx=0x7f3f88005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.937+0000 7f3f8e7fc700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] conn(0x7f3f840bdb70 0x7f3f840c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.937+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 --> [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f94004b10 con 0x7f3f840bdb70 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.937+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3f940e2d90 con 0x7f3f981023a0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.938+0000 7f3f9d970700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] conn(0x7f3f840bdb70 0x7f3f840c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.938+0000 7f3f9d970700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] conn(0x7f3f840bdb70 0x7f3f840c0010 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:19.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.939+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== osd.9 v2:172.21.15.112:6826/2726539509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f94004b10 con 0x7f3f840bdb70 2024-05-10T10:38:19.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.956+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 --> [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f980640b0 con 0x7f3f840bdb70 2024-05-10T10:38:19.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.956+0000 7f3f8e7fc700 1 -- 172.21.15.112:0/1726445234 <== osd.9 v2:172.21.15.112:6826/2726539509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3f980640b0 con 0x7f3f840bdb70 2024-05-10T10:38:19.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] conn(0x7f3f840bdb70 msgr2=0x7f3f840c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] conn(0x7f3f840bdb70 0x7f3f840c0010 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f840b2990 msgr2=0x7f3f840b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f840b2990 0x7f3f840b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800b5c0 tx=0x7f3f88005fb0 comp rx=0 tx=0).stop 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 msgr2=0x7f3f9819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 --2- 172.21.15.112:0/1726445234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981023a0 0x7f3f9819e680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400c970 tx=0x7f3f9400cd30 comp rx=0 tx=0).stop 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9d970700 1 -- 172.21.15.112:0/1726445234 reap_dead start 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 shutdown_connections 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 >> 172.21.15.112:0/1726445234 conn(0x7f3f98079350 msgr2=0x7f3f981010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 shutdown_connections 2024-05-10T10:38:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:19.957+0000 7f3f9f3d3700 1 -- 172.21.15.112:0/1726445234 wait complete. 2024-05-10T10:38:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.001+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb1c8076cd0 con 0x7fb1c8103f00 2024-05-10T10:38:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.002+0000 7fb1c4ff9700 1 -- 172.21.15.112:0/1354680730 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb1bc013070 con 0x7fb1c8103f00 2024-05-10T10:38:20.003 INFO:teuthology.orchestra.run.smithi112.stdout:77309411420 2024-05-10T10:38:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb1b00b2990 msgr2=0x7fb1b00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb1b00b2990 0x7fb1b00b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8005950 tx=0x7fb1b800b410 comp rx=0 tx=0).stop 2024-05-10T10:38:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103f00 msgr2=0x7fb1c8075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103f00 0x7fb1c8075ce0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006010 tx=0x7fb1bc0045e0 comp rx=0 tx=0).stop 2024-05-10T10:38:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 shutdown_connections 2024-05-10T10:38:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c81095e0 0x7fb1c8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb1b00b2990 0x7fb1b00b4e50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1c8103540 0x7fb1c80757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 --2- 172.21.15.112:0/1354680730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c8103f00 0x7fb1c8075ce0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 >> 172.21.15.112:0/1354680730 conn(0x7fb1c807a650 msgr2=0x7fb1c81022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 shutdown_connections 2024-05-10T10:38:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:20.004+0000 7fb1cde9f700 1 -- 172.21.15.112:0/1354680730 wait complete. 2024-05-10T10:38:20.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:19 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1739048638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T10:38:20.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:19 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1739048638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T10:38:20.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:19 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1739048638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T10:38:20.658 INFO:teuthology.orchestra.run.smithi112.stdout:257698037816 2024-05-10T10:38:20.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-10T10:38:21.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:20 smithi112 ceph-mon[30100]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:21.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:20 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1354680730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T10:38:21.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:20 smithi148 ceph-mon[33568]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:21.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:20 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1354680730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T10:38:21.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411417 got 77309411420 for osd.2 2024-05-10T10:38:21.316 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:20 smithi195 ceph-mon[33773]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:21.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:20 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1354680730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T10:38:22.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 -- 172.21.15.112:0/910124462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 msgr2=0x7fb7041103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:22.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 --2- 172.21.15.112:0/910124462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb7041103a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00b0d0 tx=0x7fb6fc00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 -- 172.21.15.112:0/910124462 shutdown_connections 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 --2- 172.21.15.112:0/910124462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb7041103a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 --2- 172.21.15.112:0/910124462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7041095d0 0x7fb7041099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 --2- 172.21.15.112:0/910124462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb704103530 0x7fb7041039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 -- 172.21.15.112:0/910124462 >> 172.21.15.112:0/910124462 conn(0x7fb70407a690 msgr2=0x7fb70407aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 -- 172.21.15.112:0/910124462 shutdown_connections 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.418+0000 7fb70b01b700 1 -- 172.21.15.112:0/910124462 wait complete. 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 Processor -- start 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 -- start start 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704103530 0x7fb7040757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb704075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7041095d0 0x7fb704076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:22.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb704113570 con 0x7fb704103ef0 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041133f0 con 0x7fb704103530 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb70b01b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041136f0 con 0x7fb7041095d0 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb703fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb704075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb703fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb704075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34558/0 (socket says 172.21.15.112:34558) 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb703fff700 1 -- 172.21.15.112:0/1163909308 learned_addr learned my addr 172.21.15.112:0/1163909308 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.419+0000 7fb708db7700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704103530 0x7fb7040757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb7095b8700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7041095d0 0x7fb704076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb703fff700 1 -- 172.21.15.112:0/1163909308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7041095d0 msgr2=0x7fb704076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb703fff700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7041095d0 0x7fb704076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb703fff700 1 -- 172.21.15.112:0/1163909308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704103530 msgr2=0x7fb7040757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb703fff700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704103530 0x7fb7040757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb703fff700 1 -- 172.21.15.112:0/1163909308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7041a7f90 con 0x7fb704103ef0 2024-05-10T10:38:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb708db7700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704103530 0x7fb7040757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb7095b8700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7041095d0 0x7fb704076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb703fff700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb704075ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000d950 tx=0x7fb6f000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb701ffb700 1 -- 172.21.15.112:0/1163909308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0016070 con 0x7fb704103ef0 2024-05-10T10:38:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb701ffb700 1 -- 172.21.15.112:0/1163909308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6f0010460 con 0x7fb704103ef0 2024-05-10T10:38:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7041a8280 con 0x7fb704103ef0 2024-05-10T10:38:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb701ffb700 1 -- 172.21.15.112:0/1163909308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000f620 con 0x7fb704103ef0 2024-05-10T10:38:22.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.420+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7041a87c0 con 0x7fb704103ef0 2024-05-10T10:38:22.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.421+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70406a190 con 0x7fb704103ef0 2024-05-10T10:38:22.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.423+0000 7fb701ffb700 1 -- 172.21.15.112:0/1163909308 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb6f0019070 con 0x7fb704103ef0 2024-05-10T10:38:22.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.423+0000 7fb701ffb700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6f40b2910 0x7fb6f40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:22.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.423+0000 7fb708db7700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6f40b2910 0x7fb6f40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:22.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.424+0000 7fb701ffb700 1 -- 172.21.15.112:0/1163909308 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb6f00e2340 con 0x7fb704103ef0 2024-05-10T10:38:22.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.424+0000 7fb708db7700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6f40b2910 0x7fb6f40b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8005e10 tx=0x7fb6f8005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:22.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.431+0000 7fb701ffb700 1 -- 172.21.15.112:0/1163909308 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb6f00a9b70 con 0x7fb704103ef0 2024-05-10T10:38:22.654 INFO:teuthology.orchestra.run.smithi112.stdout:34359738470 2024-05-10T10:38:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.652+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb70402e510 con 0x7fb704103ef0 2024-05-10T10:38:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.653+0000 7fb701ffb700 1 -- 172.21.15.112:0/1163909308 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6f00a9470 con 0x7fb704103ef0 2024-05-10T10:38:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6f40b2910 msgr2=0x7fb6f40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6f40b2910 0x7fb6f40b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8005e10 tx=0x7fb6f8005d60 comp rx=0 tx=0).stop 2024-05-10T10:38:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 msgr2=0x7fb704075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb704075ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000d950 tx=0x7fb6f000dd10 comp rx=0 tx=0).stop 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 shutdown_connections 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6f40b2910 0x7fb6f40b4dd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7041095d0 0x7fb704076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704103530 0x7fb7040757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 --2- 172.21.15.112:0/1163909308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb704103ef0 0x7fb704075ce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 >> 172.21.15.112:0/1163909308 conn(0x7fb70407a690 msgr2=0x7fb704102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 shutdown_connections 2024-05-10T10:38:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:22.655+0000 7fb70b01b700 1 -- 172.21.15.112:0/1163909308 wait complete. 2024-05-10T10:38:22.930 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:22 smithi112 ceph-mon[30100]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-10T10:38:22.930 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:22 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1163909308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T10:38:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:22 smithi148 ceph-mon[33568]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-10T10:38:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:22 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1163909308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T10:38:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:22 smithi195 ceph-mon[33773]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-10T10:38:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:22 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1163909308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T10:38:23.830 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986254 got 133143986257 for osd.4 2024-05-10T10:38:23.830 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:24.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738470 for osd.0 2024-05-10T10:38:24.355 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:24.773 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574944 for osd.1 2024-05-10T10:38:24.773 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:24.807 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:24.809 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:24 smithi112 ceph-mon[30100]: pgmap v375: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:25.238 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:24 smithi148 ceph-mon[33568]: pgmap v375: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:24 smithi195 ceph-mon[33773]: pgmap v375: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:25.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822728 got 154618822732 for osd.5 2024-05-10T10:38:25.684 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:27.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:26 smithi112 ceph-mon[30100]: pgmap v376: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-10T10:38:27.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:26 smithi148 ceph-mon[33568]: pgmap v376: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-10T10:38:27.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:26 smithi195 ceph-mon[33773]: pgmap v376: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-10T10:38:28.257 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:28.258 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:28.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 -- 172.21.15.112:0/1142612091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 msgr2=0x7fc13810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 --2- 172.21.15.112:0/1142612091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13810f240 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc13000b0a0 tx=0x7fc13000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 -- 172.21.15.112:0/1142612091 shutdown_connections 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 --2- 172.21.15.112:0/1142612091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13810f240 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 --2- 172.21.15.112:0/1142612091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc138108400 0x7fc1381087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 --2- 172.21.15.112:0/1142612091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc138102390 0x7fc138102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 -- 172.21.15.112:0/1142612091 >> 172.21.15.112:0/1142612091 conn(0x7fc138079310 msgr2=0x7fc138079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 -- 172.21.15.112:0/1142612091 shutdown_connections 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.545+0000 7fc13de34700 1 -- 172.21.15.112:0/1142612091 wait complete. 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 Processor -- start 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 -- start start 2024-05-10T10:38:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc138102390 0x7fc13819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc138108400 0x7fc1381a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc138112450 con 0x7fc138102d50 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1381122d0 con 0x7fc138108400 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc13de34700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1381125d0 con 0x7fc138102390 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc12ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc12ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51060/0 (socket says 172.21.15.112:51060) 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc12ffff700 1 -- 172.21.15.112:0/3501179384 learned_addr learned my addr 172.21.15.112:0/3501179384 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc1377fe700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc138102390 0x7fc13819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.546+0000 7fc137fff700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc138108400 0x7fc1381a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc12ffff700 1 -- 172.21.15.112:0/3501179384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc138102390 msgr2=0x7fc13819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc12ffff700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc138102390 0x7fc13819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc12ffff700 1 -- 172.21.15.112:0/3501179384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc138108400 msgr2=0x7fc1381a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc12ffff700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc138108400 0x7fc1381a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc12ffff700 1 -- 172.21.15.112:0/3501179384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1381a5ef0 con 0x7fc138102d50 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc12ffff700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13819ae70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc12000b530 tx=0x7fc12000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc1357fa700 1 -- 172.21.15.112:0/3501179384 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120015020 con 0x7fc138102d50 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc1357fa700 1 -- 172.21.15.112:0/3501179384 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc120011cc0 con 0x7fc138102d50 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1381a6120 con 0x7fc138102d50 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc1357fa700 1 -- 172.21.15.112:0/3501179384 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120010dc0 con 0x7fc138102d50 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.547+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1381a6660 con 0x7fc138102d50 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.548+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13806a190 con 0x7fc138102d50 2024-05-10T10:38:28.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.549+0000 7fc1357fa700 1 -- 172.21.15.112:0/3501179384 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc120018020 con 0x7fc138102d50 2024-05-10T10:38:28.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.550+0000 7fc1357fa700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc1240b2910 0x7fc1240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:28.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.550+0000 7fc1377fe700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc1240b2910 0x7fc1240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:28.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.550+0000 7fc1357fa700 1 -- 172.21.15.112:0/3501179384 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc1200e2460 con 0x7fc138102d50 2024-05-10T10:38:28.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.551+0000 7fc1377fe700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc1240b2910 0x7fc1240b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc128005950 tx=0x7fc12800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.555+0000 7fc1357fa700 1 -- 172.21.15.112:0/3501179384 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc12001f020 con 0x7fc138102d50 2024-05-10T10:38:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.781+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc13802e510 con 0x7fc138102d50 2024-05-10T10:38:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.781+0000 7fc1357fa700 1 -- 172.21.15.112:0/3501179384 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc1200a9da0 con 0x7fc138102d50 2024-05-10T10:38:28.783 INFO:teuthology.orchestra.run.smithi112.stdout:107374182488 2024-05-10T10:38:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc1240b2910 msgr2=0x7fc1240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc1240b2910 0x7fc1240b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc128005950 tx=0x7fc12800b410 comp rx=0 tx=0).stop 2024-05-10T10:38:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 msgr2=0x7fc13819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13819ae70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc12000b530 tx=0x7fc12000b840 comp rx=0 tx=0).stop 2024-05-10T10:38:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 shutdown_connections 2024-05-10T10:38:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc138108400 0x7fc1381a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc1240b2910 0x7fc1240b4dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc138102390 0x7fc13819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 --2- 172.21.15.112:0/3501179384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138102d50 0x7fc13819ae70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 >> 172.21.15.112:0/3501179384 conn(0x7fc138079310 msgr2=0x7fc1381010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 shutdown_connections 2024-05-10T10:38:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:28.784+0000 7fc13de34700 1 -- 172.21.15.112:0/3501179384 wait complete. 2024-05-10T10:38:29.150 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:28 smithi112 ceph-mon[30100]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:29.150 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:28 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3501179384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T10:38:29.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:28 smithi148 ceph-mon[33568]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:29.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:28 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3501179384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T10:38:29.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:28 smithi195 ceph-mon[33773]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:29.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:28 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3501179384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T10:38:29.497 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182482 got 107374182488 for osd.3 2024-05-10T10:38:29.497 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 -- 172.21.15.112:0/592868424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 msgr2=0x7f7da0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 --2- 172.21.15.112:0/592868424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da0102820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009990 tx=0x7f7d94009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 -- 172.21.15.112:0/592868424 shutdown_connections 2024-05-10T10:38:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 --2- 172.21.15.112:0/592868424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7da0102d60 0x7f7da010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 --2- 172.21.15.112:0/592868424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da0108410 0x7f7da01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 --2- 172.21.15.112:0/592868424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da0102820 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 -- 172.21.15.112:0/592868424 >> 172.21.15.112:0/592868424 conn(0x7f7da0079350 msgr2=0x7f7da0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 -- 172.21.15.112:0/592868424 shutdown_connections 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 -- 172.21.15.112:0/592868424 wait complete. 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 Processor -- start 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.974+0000 7f7da6472700 1 -- start start 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da6472700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da6472700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7da0102d60 0x7f7da019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da6472700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da0108410 0x7f7da0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da6472700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01123e0 con 0x7f7da01023a0 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da6472700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0112260 con 0x7f7da0108410 2024-05-10T10:38:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da6472700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0112560 con 0x7f7da0102d60 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51076/0 (socket says 172.21.15.112:51076) 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 -- 172.21.15.112:0/4017124511 learned_addr learned my addr 172.21.15.112:0/4017124511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9f7fe700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7da0102d60 0x7f7da019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da4a0f700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da0108410 0x7f7da0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 -- 172.21.15.112:0/4017124511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7da0102d60 msgr2=0x7f7da019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7da0102d60 0x7f7da019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 -- 172.21.15.112:0/4017124511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da0108410 msgr2=0x7f7da0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da0108410 0x7f7da0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 -- 172.21.15.112:0/4017124511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da019a030 con 0x7f7da01023a0 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9ffff700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da019f680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c9b0 tx=0x7f7d9000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:29.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9d7fa700 1 -- 172.21.15.112:0/4017124511 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90007b00 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da019a320 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9d7fa700 1 -- 172.21.15.112:0/4017124511 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d90007ca0 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.975+0000 7f7d9d7fa700 1 -- 172.21.15.112:0/4017124511 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90018660 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.976+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da01a64e0 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.977+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da0103ac0 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.978+0000 7f7d9d7fa700 1 -- 172.21.15.112:0/4017124511 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7d9000f450 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.978+0000 7f7d9d7fa700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2910 0x7f7d880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.978+0000 7f7d9f7fe700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2910 0x7f7d880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.979+0000 7f7d9d7fa700 1 -- 172.21.15.112:0/4017124511 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7d900e26a0 con 0x7f7da01023a0 2024-05-10T10:38:29.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.979+0000 7f7d9f7fe700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2910 0x7f7d880b4dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d940098e0 tx=0x7f7d94005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:29.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:29.983+0000 7f7d9d7fa700 1 -- 172.21.15.112:0/4017124511 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7d90014030 con 0x7f7da01023a0 2024-05-10T10:38:29.985 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.199+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7da01a6770 con 0x7f7da01023a0 2024-05-10T10:38:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.199+0000 7f7d9d7fa700 1 -- 172.21.15.112:0/4017124511 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7d900a9ed0 con 0x7f7da01023a0 2024-05-10T10:38:30.201 INFO:teuthology.orchestra.run.smithi112.stdout:231928234048 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.201+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2910 msgr2=0x7f7d880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.201+0000 7f7da6472700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2910 0x7f7d880b4dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d940098e0 tx=0x7f7d94005960 comp rx=0 tx=0).stop 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.201+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 msgr2=0x7f7da019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.201+0000 7f7da6472700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da019f680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c9b0 tx=0x7f7d9000ccc0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 shutdown_connections 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da0108410 0x7f7da0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7da01023a0 0x7f7da019f680 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2910 0x7f7d880b4dd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 --2- 172.21.15.112:0/4017124511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7da0102d60 0x7f7da019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 >> 172.21.15.112:0/4017124511 conn(0x7f7da0079350 msgr2=0x7f7da0106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:30.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 shutdown_connections 2024-05-10T10:38:30.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.202+0000 7f7da6472700 1 -- 172.21.15.112:0/4017124511 wait complete. 2024-05-10T10:38:30.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 -- 172.21.15.112:0/3835038224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 msgr2=0x7ff8e01098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3835038224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e01098e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009990 tx=0x7ff8d0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 -- 172.21.15.112:0/3835038224 shutdown_connections 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3835038224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e006c3d0 0x7ff8e0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3835038224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e01098e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3835038224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e006ba10 0x7ff8e006be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 -- 172.21.15.112:0/3835038224 >> 172.21.15.112:0/3835038224 conn(0x7ff8e0079310 msgr2=0x7ff8e0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 -- 172.21.15.112:0/3835038224 shutdown_connections 2024-05-10T10:38:30.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.238+0000 7ff8e592b700 1 -- 172.21.15.112:0/3835038224 wait complete. 2024-05-10T10:38:30.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.239+0000 7ff8e592b700 1 Processor -- start 2024-05-10T10:38:30.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.239+0000 7ff8e592b700 1 -- start start 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8e592b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e006ba10 0x7ff8e019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8e592b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e006c3d0 0x7ff8e019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8e592b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8e592b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01134c0 con 0x7ff8e0109500 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8e592b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0113340 con 0x7ff8e006ba10 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8e592b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0113640 con 0x7ff8e006c3d0 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e0199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51102/0 (socket says 172.21.15.112:51102) 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 -- 172.21.15.112:0/3105854078 learned_addr learned my addr 172.21.15.112:0/3105854078 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8deffd700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e006ba10 0x7ff8e019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8de7fc700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e006c3d0 0x7ff8e019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 -- 172.21.15.112:0/3105854078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e006c3d0 msgr2=0x7ff8e019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e006c3d0 0x7ff8e019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 -- 172.21.15.112:0/3105854078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e006ba10 msgr2=0x7ff8e019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e006ba10 0x7ff8e019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 -- 172.21.15.112:0/3105854078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e0199fc0 con 0x7ff8e0109500 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8deffd700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e006ba10 0x7ff8e019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8df7fe700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e0199860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8007ee0 tx=0x7ff8d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.240+0000 7ff8e4929700 1 -- 172.21.15.112:0/3105854078 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800dbd0 con 0x7ff8e0109500 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.241+0000 7ff8e4929700 1 -- 172.21.15.112:0/3105854078 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8d800dd70 con 0x7ff8e0109500 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.241+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e019a2b0 con 0x7ff8e0109500 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.241+0000 7ff8e4929700 1 -- 172.21.15.112:0/3105854078 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80058d0 con 0x7ff8e0109500 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.241+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01a6570 con 0x7ff8e0109500 2024-05-10T10:38:30.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.241+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e010b9e0 con 0x7ff8e0109500 2024-05-10T10:38:30.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.244+0000 7ff8e4929700 1 -- 172.21.15.112:0/3105854078 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7ff8d801a020 con 0x7ff8e0109500 2024-05-10T10:38:30.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.245+0000 7ff8e4929700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8c80b2910 0x7ff8c80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:30.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.245+0000 7ff8deffd700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8c80b2910 0x7ff8c80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:30.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.245+0000 7ff8e4929700 1 -- 172.21.15.112:0/3105854078 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff8d80e25b0 con 0x7ff8e0109500 2024-05-10T10:38:30.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.245+0000 7ff8deffd700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8c80b2910 0x7ff8c80b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0003760 tx=0x7ff8d001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:30.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.248+0000 7ff8e4929700 1 -- 172.21.15.112:0/3105854078 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff8d8010070 con 0x7ff8e0109500 2024-05-10T10:38:30.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.453+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff8e010b3c0 con 0x7ff8e0109500 2024-05-10T10:38:30.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.454+0000 7ff8e4929700 1 -- 172.21.15.112:0/3105854078 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff8d80a9de0 con 0x7ff8e0109500 2024-05-10T10:38:30.455 INFO:teuthology.orchestra.run.smithi112.stdout:180388626506 2024-05-10T10:38:30.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8c80b2910 msgr2=0x7ff8c80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:30.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8c80b2910 0x7ff8c80b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0003760 tx=0x7ff8d001a040 comp rx=0 tx=0).stop 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 msgr2=0x7ff8e0199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e0199860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8007ee0 tx=0x7ff8d800d3b0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 shutdown_connections 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0109500 0x7ff8e0199860 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e006ba10 0x7ff8e019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff8c80b2910 0x7ff8c80b4dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 --2- 172.21.15.112:0/3105854078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e006c3d0 0x7ff8e019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 >> 172.21.15.112:0/3105854078 conn(0x7ff8e0079310 msgr2=0x7ff8e0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 shutdown_connections 2024-05-10T10:38:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:30.456+0000 7ff8e592b700 1 -- 172.21.15.112:0/3105854078 wait complete. 2024-05-10T10:38:31.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:30 smithi112 ceph-mon[30100]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:31.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:30 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4017124511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T10:38:31.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:30 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3105854078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T10:38:31.227 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234043 got 231928234048 for osd.8 2024-05-10T10:38:31.227 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:30 smithi148 ceph-mon[33568]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:30 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4017124511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T10:38:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:30 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3105854078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T10:38:31.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:30 smithi195 ceph-mon[33773]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:38:31.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:30 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4017124511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T10:38:31.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:30 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3105854078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T10:38:32.983 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:32 smithi112 ceph-mon[30100]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:32.983 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:32.983 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:32 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:32 smithi148 ceph-mon[33568]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:32 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:33.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:32 smithi195 ceph-mon[33773]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:33.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:33.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:32 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:35.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:34 smithi112 ceph-mon[30100]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-10T10:38:35.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:34 smithi112 ceph-mon[30100]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T10:38:35.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:34 smithi148 ceph-mon[33568]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-10T10:38:35.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:34 smithi148 ceph-mon[33568]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T10:38:35.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:34 smithi195 ceph-mon[33773]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-10T10:38:35.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:34 smithi195 ceph-mon[33773]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T10:38:35.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626500 got 180388626506 for osd.6 2024-05-10T10:38:35.539 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:36.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 -- 172.21.15.112:0/963095224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0108850 msgr2=0x7f0bc01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 --2- 172.21.15.112:0/963095224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0108850 0x7f0bc01103d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800b0d0 tx=0x7f0bb800b3e0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 -- 172.21.15.112:0/963095224 shutdown_connections 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 --2- 172.21.15.112:0/963095224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0108850 0x7f0bc01103d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 --2- 172.21.15.112:0/963095224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bc0103750 0x7f0bc0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 --2- 172.21.15.112:0/963095224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc0104100 0x7f0bc0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 -- 172.21.15.112:0/963095224 >> 172.21.15.112:0/963095224 conn(0x7f0bc007a600 msgr2=0x7f0bc007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 -- 172.21.15.112:0/963095224 shutdown_connections 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.206+0000 7f0bc8489700 1 -- 172.21.15.112:0/963095224 wait complete. 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 Processor -- start 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 -- start start 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc0103750 0x7f0bc00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0104100 0x7f0bc0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bc0108850 0x7f0bc0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0113540 con 0x7f0bc0104100 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01133c0 con 0x7f0bc0108850 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc8489700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01136c0 con 0x7f0bc0103750 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc5a24700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0104100 0x7f0bc0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc5a24700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0104100 0x7f0bc0075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34022/0 (socket says 172.21.15.112:34022) 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc5a24700 1 -- 172.21.15.112:0/3700885603 learned_addr learned my addr 172.21.15.112:0/3700885603 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc6a26700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bc0108850 0x7f0bc0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.207+0000 7f0bc6225700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc0103750 0x7f0bc00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc5a24700 1 -- 172.21.15.112:0/3700885603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc0103750 msgr2=0x7f0bc00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc5a24700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc0103750 0x7f0bc00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc5a24700 1 -- 172.21.15.112:0/3700885603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bc0108850 msgr2=0x7f0bc0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc5a24700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bc0108850 0x7f0bc0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc5a24700 1 -- 172.21.15.112:0/3700885603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc01a7f40 con 0x7f0bc0104100 2024-05-10T10:38:36.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc6a26700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bc0108850 0x7f0bc0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:36.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc5a24700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0104100 0x7f0bc0075ce0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000b530 tx=0x7f0bb000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:36.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bb77fe700 1 -- 172.21.15.112:0/3700885603 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0015020 con 0x7f0bc0104100 2024-05-10T10:38:36.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc01a8230 con 0x7f0bc0104100 2024-05-10T10:38:36.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bb77fe700 1 -- 172.21.15.112:0/3700885603 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bb0011cc0 con 0x7f0bc0104100 2024-05-10T10:38:36.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bb77fe700 1 -- 172.21.15.112:0/3700885603 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0010d30 con 0x7f0bc0104100 2024-05-10T10:38:36.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.208+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01a8720 con 0x7f0bc0104100 2024-05-10T10:38:36.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.209+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc006a190 con 0x7f0bc0104100 2024-05-10T10:38:36.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.210+0000 7f0bb77fe700 1 -- 172.21.15.112:0/3700885603 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0bb0018020 con 0x7f0bc0104100 2024-05-10T10:38:36.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:38:36.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:38:36.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:35 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:36.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.211+0000 7f0bb77fe700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bac0b2880 0x7f0bac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.211+0000 7f0bc6225700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bac0b2880 0x7f0bac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.211+0000 7f0bb77fe700 1 -- 172.21.15.112:0/3700885603 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0bb00e2430 con 0x7f0bc0104100 2024-05-10T10:38:36.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.211+0000 7f0bc6225700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bac0b2880 0x7f0bac0b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005fd0 tx=0x7f0bbc00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:36.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.215+0000 7f0bb77fe700 1 -- 172.21.15.112:0/3700885603 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0bb001f020 con 0x7f0bc0104100 2024-05-10T10:38:36.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:38:36.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:38:36.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:35 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:38:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:38:36.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:35 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:38:36.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.429+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0bc002e510 con 0x7f0bc0104100 2024-05-10T10:38:36.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.429+0000 7f0bb77fe700 1 -- 172.21.15.112:0/3700885603 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0bb00a9dc0 con 0x7f0bc0104100 2024-05-10T10:38:36.431 INFO:teuthology.orchestra.run.smithi112.stdout:304942678065 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.431+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bac0b2880 msgr2=0x7f0bac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.431+0000 7f0bc8489700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bac0b2880 0x7f0bac0b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005fd0 tx=0x7f0bbc00a560 comp rx=0 tx=0).stop 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.431+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0104100 msgr2=0x7f0bc0075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.431+0000 7f0bc8489700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0104100 0x7f0bc0075ce0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000b530 tx=0x7f0bb000b8f0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 shutdown_connections 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bc0108850 0x7f0bc0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bac0b2880 0x7f0bac0b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0bc0103750 0x7f0bc00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 --2- 172.21.15.112:0/3700885603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc0104100 0x7f0bc0075ce0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 >> 172.21.15.112:0/3700885603 conn(0x7f0bc007a600 msgr2=0x7f0bc0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:36.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 shutdown_connections 2024-05-10T10:38:36.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.432+0000 7f0bc8489700 1 -- 172.21.15.112:0/3700885603 wait complete. 2024-05-10T10:38:36.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 -- 172.21.15.112:0/2357492873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 msgr2=0x7f2154102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 --2- 172.21.15.112:0/2357492873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f2154102820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2144009990 tx=0x7f2144009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 -- 172.21.15.112:0/2357492873 shutdown_connections 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 --2- 172.21.15.112:0/2357492873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154102d60 0x7f215410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 --2- 172.21.15.112:0/2357492873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2154108410 0x7f21541087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 --2- 172.21.15.112:0/2357492873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f2154102820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 -- 172.21.15.112:0/2357492873 >> 172.21.15.112:0/2357492873 conn(0x7f2154079350 msgr2=0x7f2154079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 -- 172.21.15.112:0/2357492873 shutdown_connections 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.947+0000 7f2159509700 1 -- 172.21.15.112:0/2357492873 wait complete. 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 Processor -- start 2024-05-10T10:38:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 -- start start 2024-05-10T10:38:36.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f215419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2154102d60 0x7f215419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154108410 0x7f21541a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21541123e0 con 0x7f21541023a0 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2154112260 con 0x7f2154102d60 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2159509700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2154112560 con 0x7f2154108410 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2152ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f215419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2152ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f215419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34042/0 (socket says 172.21.15.112:34042) 2024-05-10T10:38:36.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.948+0000 7f2152ffd700 1 -- 172.21.15.112:0/3368942698 learned_addr learned my addr 172.21.15.112:0/3368942698 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f21527fc700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2154102d60 0x7f215419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f21537fe700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154108410 0x7f21541a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2152ffd700 1 -- 172.21.15.112:0/3368942698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154108410 msgr2=0x7f21541a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2152ffd700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154108410 0x7f21541a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2152ffd700 1 -- 172.21.15.112:0/3368942698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2154102d60 msgr2=0x7f215419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2152ffd700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2154102d60 0x7f215419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2152ffd700 1 -- 172.21.15.112:0/3368942698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21541a5e00 con 0x7f21541023a0 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f21537fe700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154108410 0x7f21541a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2152ffd700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f215419a840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f214800d950 tx=0x7f214800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f213bfff700 1 -- 172.21.15.112:0/3368942698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148016070 con 0x7f21541023a0 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21541a5fd0 con 0x7f21541023a0 2024-05-10T10:38:36.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f213bfff700 1 -- 172.21.15.112:0/3368942698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2148009dd0 con 0x7f21541023a0 2024-05-10T10:38:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f213bfff700 1 -- 172.21.15.112:0/3368942698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214800f3c0 con 0x7f21541023a0 2024-05-10T10:38:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.949+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21541a6480 con 0x7f21541023a0 2024-05-10T10:38:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.950+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215406a190 con 0x7f21541023a0 2024-05-10T10:38:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.951+0000 7f213bfff700 1 -- 172.21.15.112:0/3368942698 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2148019070 con 0x7f21541023a0 2024-05-10T10:38:36.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.952+0000 7f213bfff700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f213c0b2880 0x7f213c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:36.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.953+0000 7f213bfff700 1 -- 172.21.15.112:0/3368942698 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f21480e2100 con 0x7f21541023a0 2024-05-10T10:38:36.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.953+0000 7f21527fc700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f213c0b2880 0x7f213c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:36.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.953+0000 7f21527fc700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f213c0b2880 0x7f213c0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21440098e0 tx=0x7f2144005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:36.957+0000 7f213bfff700 1 -- 172.21.15.112:0/3368942698 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f21480a9a90 con 0x7f21541023a0 2024-05-10T10:38:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.164+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f215402e6d0 con 0x7f21541023a0 2024-05-10T10:38:37.166 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:36 smithi112 ceph-mon[30100]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:37.166 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:36 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3700885603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T10:38:37.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.165+0000 7f213bfff700 1 -- 172.21.15.112:0/3368942698 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f21480a9390 con 0x7f21541023a0 2024-05-10T10:38:37.167 INFO:teuthology.orchestra.run.smithi112.stdout:206158430278 2024-05-10T10:38:37.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f213c0b2880 msgr2=0x7f213c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f213c0b2880 0x7f213c0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21440098e0 tx=0x7f2144005960 comp rx=0 tx=0).stop 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 msgr2=0x7f215419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f215419a840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f214800d950 tx=0x7f214800dc60 comp rx=0 tx=0).stop 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 shutdown_connections 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f213c0b2880 0x7f213c0b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2154108410 0x7f21541a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21541023a0 0x7f215419a840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 --2- 172.21.15.112:0/3368942698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2154102d60 0x7f215419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 >> 172.21.15.112:0/3368942698 conn(0x7f2154079350 msgr2=0x7f2154100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 shutdown_connections 2024-05-10T10:38:37.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:37.167+0000 7f2159509700 1 -- 172.21.15.112:0/3368942698 wait complete. 2024-05-10T10:38:37.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:36 smithi148 ceph-mon[33568]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:37.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:36 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3700885603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T10:38:37.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:36 smithi195 ceph-mon[33773]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:37.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:36 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3700885603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T10:38:38.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:37 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3368942698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T10:38:38.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:37 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3368942698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T10:38:38.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:37 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3368942698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T10:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 -- 172.21.15.112:0/1736506236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102630 msgr2=0x7f9f34102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/1736506236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102630 0x7f9f34102a10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9f300099c0 tx=0x7f9f30009cd0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 -- 172.21.15.112:0/1736506236 shutdown_connections 2024-05-10T10:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/1736506236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f341076c0 0x7f9f3410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/1736506236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102630 0x7f9f34102a10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/1736506236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f34102fe0 0x7f9f34106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 -- 172.21.15.112:0/1736506236 >> 172.21.15.112:0/1736506236 conn(0x7f9f34079350 msgr2=0x7f9f34079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 -- 172.21.15.112:0/1736506236 shutdown_connections 2024-05-10T10:38:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 -- 172.21.15.112:0/1736506236 wait complete. 2024-05-10T10:38:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 Processor -- start 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.317+0000 7f9f3c03a700 1 -- start start 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f3c03a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f34102630 0x7f9f34104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f3c03a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102fe0 0x7f9f341045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f3c03a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f341076c0 0x7f9f34104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f3c03a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341124d0 con 0x7f9f34102fe0 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f3c03a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34112350 con 0x7f9f34102630 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f3c03a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34112650 con 0x7f9f341076c0 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102fe0 0x7f9f341045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102fe0 0x7f9f341045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34048/0 (socket says 172.21.15.112:34048) 2024-05-10T10:38:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 -- 172.21.15.112:0/313568069 learned_addr learned my addr 172.21.15.112:0/313568069 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f39dd6700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f34102630 0x7f9f34104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f3a5d7700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f341076c0 0x7f9f34104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 -- 172.21.15.112:0/313568069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f341076c0 msgr2=0x7f9f34104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f341076c0 0x7f9f34104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 -- 172.21.15.112:0/313568069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f34102630 msgr2=0x7f9f34104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f34102630 0x7f9f34104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 -- 172.21.15.112:0/313568069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f341a8090 con 0x7f9f34102fe0 2024-05-10T10:38:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f395d5700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102fe0 0x7f9f341045c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c8f0 tx=0x7f9f2400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.318+0000 7f9f2affd700 1 -- 172.21.15.112:0/313568069 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240078d0 con 0x7f9f34102fe0 2024-05-10T10:38:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.319+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f341a8380 con 0x7f9f34102fe0 2024-05-10T10:38:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.319+0000 7f9f2affd700 1 -- 172.21.15.112:0/313568069 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2400f450 con 0x7f9f34102fe0 2024-05-10T10:38:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.319+0000 7f9f2affd700 1 -- 172.21.15.112:0/313568069 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24018660 con 0x7f9f34102fe0 2024-05-10T10:38:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.319+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f341a88c0 con 0x7f9f34102fe0 2024-05-10T10:38:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.319+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3406a190 con 0x7f9f34102fe0 2024-05-10T10:38:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.322+0000 7f9f2affd700 1 -- 172.21.15.112:0/313568069 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9f2400f900 con 0x7f9f34102fe0 2024-05-10T10:38:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.322+0000 7f9f2affd700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9f200b2880 0x7f9f200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.322+0000 7f9f39dd6700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9f200b2880 0x7f9f200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.322+0000 7f9f2affd700 1 -- 172.21.15.112:0/313568069 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9f240e2490 con 0x7f9f34102fe0 2024-05-10T10:38:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.323+0000 7f9f39dd6700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9f200b2880 0x7f9f200b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f30000c00 tx=0x7f9f30005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.326+0000 7f9f2affd700 1 -- 172.21.15.112:0/313568069 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9f24014030 con 0x7f9f34102fe0 2024-05-10T10:38:38.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.532+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f9f3402e510 con 0x7f9f34102fe0 2024-05-10T10:38:38.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.532+0000 7f9f2affd700 1 -- 172.21.15.112:0/313568069 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9f240a9eb0 con 0x7f9f34102fe0 2024-05-10T10:38:38.533 INFO:teuthology.orchestra.run.smithi112.stdout:279172874294 2024-05-10T10:38:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9f200b2880 msgr2=0x7f9f200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9f200b2880 0x7f9f200b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f30000c00 tx=0x7f9f30005fb0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102fe0 msgr2=0x7f9f341045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102fe0 0x7f9f341045c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c8f0 tx=0x7f9f2400cc00 comp rx=0 tx=0).stop 2024-05-10T10:38:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 shutdown_connections 2024-05-10T10:38:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9f200b2880 0x7f9f200b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f341076c0 0x7f9f34104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f34102630 0x7f9f34104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 --2- 172.21.15.112:0/313568069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f34102fe0 0x7f9f341045c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 >> 172.21.15.112:0/313568069 conn(0x7f9f34079350 msgr2=0x7f9f3406c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 shutdown_connections 2024-05-10T10:38:38.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:38.535+0000 7f9f3c03a700 1 -- 172.21.15.112:0/313568069 wait complete. 2024-05-10T10:38:39.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430273 got 206158430278 for osd.7 2024-05-10T10:38:39.091 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:39.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:38 smithi148 ceph-mon[33568]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:38:39.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:38 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/313568069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T10:38:39.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:38 smithi195 ceph-mon[33773]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:38:39.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:38 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/313568069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T10:38:39.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:38 smithi112 ceph-mon[30100]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:38:39.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:38 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/313568069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T10:38:39.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.553+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/570418114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 msgr2=0x7f8b7410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.553+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/570418114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b7410f250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00b0a0 tx=0x7f8b6c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/570418114 shutdown_connections 2024-05-10T10:38:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/570418114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b7410f250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/570418114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b74108450 0x7f8b74108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/570418114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b741023d0 0x7f8b74102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/570418114 >> 172.21.15.112:0/570418114 conn(0x7f8b74078410 msgr2=0x7f8b74078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:39.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/570418114 shutdown_connections 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/570418114 wait complete. 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 Processor -- start 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 -- start start 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.554+0000 7f8b7c1d5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b741023d0 0x7f8b74114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7c1d5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b74112bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7c1d5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b74108450 0x7f8b741132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7c1d5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741123f0 con 0x7f8b74102d90 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7c1d5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74112270 con 0x7f8b741023d0 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7c1d5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74112570 con 0x7f8b74108450 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b74112bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b74112bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34054/0 (socket says 172.21.15.112:34054) 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 -- 172.21.15.112:0/2703252360 learned_addr learned my addr 172.21.15.112:0/2703252360 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7a772700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b74108450 0x7f8b741132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79f71700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b741023d0 0x7f8b74114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 -- 172.21.15.112:0/2703252360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b74108450 msgr2=0x7f8b741132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b74108450 0x7f8b741132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 -- 172.21.15.112:0/2703252360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b741023d0 msgr2=0x7f8b74114880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b741023d0 0x7f8b74114880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 -- 172.21.15.112:0/2703252360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b741a7f50 con 0x7f8b74102d90 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7a772700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b74108450 0x7f8b741132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79f71700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b741023d0 0x7f8b74114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b79770700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b74112bb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400d980 tx=0x7f8b6400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b6affd700 1 -- 172.21.15.112:0/2703252360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64017070 con 0x7f8b74102d90 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b6affd700 1 -- 172.21.15.112:0/2703252360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b64004830 con 0x7f8b74102d90 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b741a8240 con 0x7f8b74102d90 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b6affd700 1 -- 172.21.15.112:0/2703252360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640053a0 con 0x7f8b74102d90 2024-05-10T10:38:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.555+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b741a8700 con 0x7f8b74102d90 2024-05-10T10:38:39.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.556+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b74103ac0 con 0x7f8b74102d90 2024-05-10T10:38:39.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.558+0000 7f8b6affd700 1 -- 172.21.15.112:0/2703252360 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8b640049e0 con 0x7f8b74102d90 2024-05-10T10:38:39.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.558+0000 7f8b6affd700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b600b2880 0x7f8b600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:39.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.558+0000 7f8b79f71700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b600b2880 0x7f8b600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:39.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.558+0000 7f8b6affd700 1 -- 172.21.15.112:0/2703252360 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8b640e2940 con 0x7f8b74102d90 2024-05-10T10:38:39.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.559+0000 7f8b79f71700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b600b2880 0x7f8b600b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005e10 tx=0x7f8b70005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:39.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.561+0000 7f8b6affd700 1 -- 172.21.15.112:0/2703252360 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8b64020020 con 0x7f8b74102d90 2024-05-10T10:38:39.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.770+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8b741a8960 con 0x7f8b74102d90 2024-05-10T10:38:39.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.770+0000 7f8b6affd700 1 -- 172.21.15.112:0/2703252360 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8b640aa2d0 con 0x7f8b74102d90 2024-05-10T10:38:39.772 INFO:teuthology.orchestra.run.smithi112.stdout:257698037819 2024-05-10T10:38:39.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b600b2880 msgr2=0x7f8b600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:39.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b600b2880 0x7f8b600b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005e10 tx=0x7f8b70005d20 comp rx=0 tx=0).stop 2024-05-10T10:38:39.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 msgr2=0x7f8b74112bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:39.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b74112bb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400d980 tx=0x7f8b6400dd40 comp rx=0 tx=0).stop 2024-05-10T10:38:39.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 shutdown_connections 2024-05-10T10:38:39.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b600b2880 0x7f8b600b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b74108450 0x7f8b741132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b741023d0 0x7f8b74114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 --2- 172.21.15.112:0/2703252360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b74102d90 0x7f8b74112bb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:39.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 >> 172.21.15.112:0/2703252360 conn(0x7f8b74078410 msgr2=0x7f8b741005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:39.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 shutdown_connections 2024-05-10T10:38:39.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:39.773+0000 7f8b7c1d5700 1 -- 172.21.15.112:0/2703252360 wait complete. 2024-05-10T10:38:40.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874289 got 279172874294 for osd.10 2024-05-10T10:38:40.048 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:40.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:39 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2703252360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T10:38:40.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:39 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2703252360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T10:38:40.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:39 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2703252360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T10:38:40.568 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678059 got 304942678065 for osd.11 2024-05-10T10:38:40.569 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:41.237 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:40 smithi112 ceph-mon[30100]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:41.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037816 got 257698037819 for osd.9 2024-05-10T10:38:41.270 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:41.271 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T10:38:41.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T10:38:41.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:40 smithi148 ceph-mon[33568]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:40 smithi195 ceph-mon[33773]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:43.148 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:42 smithi112 ceph-mon[30100]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:42 smithi148 ceph-mon[33568]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:42 smithi195 ceph-mon[33773]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.376+0000 7fbe33cda700 1 -- 172.21.15.112:0/2725507614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c106320 msgr2=0x7fbe2c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.376+0000 7fbe33cda700 1 --2- 172.21.15.112:0/2725507614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c106320 0x7fbe2c106700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe280099c0 tx=0x7fbe28009cd0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 -- 172.21.15.112:0/2725507614 shutdown_connections 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 --2- 172.21.15.112:0/2725507614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe2c108b50 0x7fbe2c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 --2- 172.21.15.112:0/2725507614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c106320 0x7fbe2c106700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 --2- 172.21.15.112:0/2725507614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe2c108190 0x7fbe2c108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 -- 172.21.15.112:0/2725507614 >> 172.21.15.112:0/2725507614 conn(0x7fbe2c0776c0 msgr2=0x7fbe2c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 -- 172.21.15.112:0/2725507614 shutdown_connections 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 -- 172.21.15.112:0/2725507614 wait complete. 2024-05-10T10:38:43.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 Processor -- start 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.377+0000 7fbe33cda700 1 -- start start 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe33cda700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe2c106320 0x7fbe2c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe33cda700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe2c108190 0x7fbe2c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe33cda700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c108b50 0x7fbe2c1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe33cda700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1124a0 con 0x7fbe2c108b50 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe33cda700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c112320 con 0x7fbe2c108190 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe33cda700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c112620 con 0x7fbe2c106320 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c108b50 0x7fbe2c1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c108b50 0x7fbe2c1a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34064/0 (socket says 172.21.15.112:34064) 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 -- 172.21.15.112:0/570013108 learned_addr learned my addr 172.21.15.112:0/570013108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe31275700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe2c108190 0x7fbe2c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe31a76700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe2c106320 0x7fbe2c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 -- 172.21.15.112:0/570013108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe2c106320 msgr2=0x7fbe2c19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe2c106320 0x7fbe2c19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 -- 172.21.15.112:0/570013108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe2c108190 msgr2=0x7fbe2c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe2c108190 0x7fbe2c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 -- 172.21.15.112:0/570013108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c1a5ea0 con 0x7fbe2c108b50 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe32277700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c108b50 0x7fbe2c1a5960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe24007d70 tx=0x7fbe2400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe22ffd700 1 -- 172.21.15.112:0/570013108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24016020 con 0x7fbe2c108b50 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe22ffd700 1 -- 172.21.15.112:0/570013108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe2400ac90 con 0x7fbe2c108b50 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c1a60d0 con 0x7fbe2c108b50 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.378+0000 7fbe22ffd700 1 -- 172.21.15.112:0/570013108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe240115e0 con 0x7fbe2c108b50 2024-05-10T10:38:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.379+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c1a6590 con 0x7fbe2c108b50 2024-05-10T10:38:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.379+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2c10a9d0 con 0x7fbe2c108b50 2024-05-10T10:38:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.381+0000 7fbe22ffd700 1 -- 172.21.15.112:0/570013108 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbe24019020 con 0x7fbe2c108b50 2024-05-10T10:38:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.381+0000 7fbe22ffd700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbe180b2880 0x7fbe180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.381+0000 7fbe31a76700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbe180b2880 0x7fbe180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.382+0000 7fbe22ffd700 1 -- 172.21.15.112:0/570013108 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbe240e29e0 con 0x7fbe2c108b50 2024-05-10T10:38:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.382+0000 7fbe31a76700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbe180b2880 0x7fbe180b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe28000c00 tx=0x7fbe28005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.385+0000 7fbe22ffd700 1 -- 172.21.15.112:0/570013108 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbe24020020 con 0x7fbe2c108b50 2024-05-10T10:38:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.584+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe2c0638e0 con 0x7fbe180b2880 2024-05-10T10:38:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.594+0000 7fbe22ffd700 1 -- 172.21.15.112:0/570013108 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340535 (secure 0 0 0) 0x7fbe2807b250 con 0x7fbe180b2880 2024-05-10T10:38:43.600 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbe180b2880 msgr2=0x7fbe180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbe180b2880 0x7fbe180b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe28000c00 tx=0x7fbe28005fb0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c108b50 msgr2=0x7fbe2c1a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c108b50 0x7fbe2c1a5960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe24007d70 tx=0x7fbe2400a340 comp rx=0 tx=0).stop 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 shutdown_connections 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe2c108b50 0x7fbe2c1a5960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbe180b2880 0x7fbe180b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe2c106320 0x7fbe2c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 --2- 172.21.15.112:0/570013108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe2c108190 0x7fbe2c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 >> 172.21.15.112:0/570013108 conn(0x7fbe2c0776c0 msgr2=0x7fbe2c0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 shutdown_connections 2024-05-10T10:38:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:43.597+0000 7fbe33cda700 1 -- 172.21.15.112:0/570013108 wait complete. 2024-05-10T10:38:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-10T10:38:44.427 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-10T10:38:43.310920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664839,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1108,"num_read_kb":662,"num_write":380,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2955541,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367708,"kb_used_data":14396,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574116,"statfs":{"total":1151940427776,"available":1151563894784,"internally_reserved":0,"allocated":14741504,"data_stored":8215007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":57,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003488"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295460+0000","last_change":"2024-05-10T10:35:50.924449+0000","last_active":"2024-05-10T10:38:33.295460+0000","last_peered":"2024-05-10T10:38:33.295460+0000","last_clean":"2024-05-10T10:38:33.295460+0000","last_became_active":"2024-05-10T10:35:50.924114+0000","last_became_peered":"2024-05-10T10:35:50.924114+0000","last_unstale":"2024-05-10T10:38:33.295460+0000","last_undegraded":"2024-05-10T10:38:33.295460+0000","last_fullsized":"2024-05-10T10:38:33.295460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:50.636501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292146+0000","last_change":"2024-05-10T10:35:50.928715+0000","last_active":"2024-05-10T10:38:33.292146+0000","last_peered":"2024-05-10T10:38:33.292146+0000","last_clean":"2024-05-10T10:38:33.292146+0000","last_became_active":"2024-05-10T10:35:50.928404+0000","last_became_peered":"2024-05-10T10:35:50.928404+0000","last_unstale":"2024-05-10T10:38:33.292146+0000","last_undegraded":"2024-05-10T10:38:33.292146+0000","last_fullsized":"2024-05-10T10:38:33.292146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:44.934022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294960+0000","last_change":"2024-05-10T10:35:50.925134+0000","last_active":"2024-05-10T10:38:33.294960+0000","last_peered":"2024-05-10T10:38:33.294960+0000","last_clean":"2024-05-10T10:38:33.294960+0000","last_became_active":"2024-05-10T10:35:50.924797+0000","last_became_peered":"2024-05-10T10:35:50.924797+0000","last_unstale":"2024-05-10T10:38:33.294960+0000","last_undegraded":"2024-05-10T10:38:33.294960+0000","last_fullsized":"2024-05-10T10:38:33.294960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:43:52.666519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.269216+0000","last_change":"2024-05-10T10:35:50.930916+0000","last_active":"2024-05-10T10:38:33.269216+0000","last_peered":"2024-05-10T10:38:33.269216+0000","last_clean":"2024-05-10T10:38:33.269216+0000","last_became_active":"2024-05-10T10:35:50.930378+0000","last_became_peered":"2024-05-10T10:35:50.930378+0000","last_unstale":"2024-05-10T10:38:33.269216+0000","last_undegraded":"2024-05-10T10:38:33.269216+0000","last_fullsized":"2024-05-10T10:38:33.269216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:07.165595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.269516+0000","last_change":"2024-05-10T10:35:50.925459+0000","last_active":"2024-05-10T10:38:33.269516+0000","last_peered":"2024-05-10T10:38:33.269516+0000","last_clean":"2024-05-10T10:38:33.269516+0000","last_became_active":"2024-05-10T10:35:50.925061+0000","last_became_peered":"2024-05-10T10:35:50.925061+0000","last_unstale":"2024-05-10T10:38:33.269516+0000","last_undegraded":"2024-05-10T10:38:33.269516+0000","last_fullsized":"2024-05-10T10:38:33.269516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:48:06.622369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290891+0000","last_change":"2024-05-10T10:35:50.932052+0000","last_active":"2024-05-10T10:38:33.290891+0000","last_peered":"2024-05-10T10:38:33.290891+0000","last_clean":"2024-05-10T10:38:33.290891+0000","last_became_active":"2024-05-10T10:35:50.931685+0000","last_became_peered":"2024-05-10T10:35:50.931685+0000","last_unstale":"2024-05-10T10:38:33.290891+0000","last_undegraded":"2024-05-10T10:38:33.290891+0000","last_fullsized":"2024-05-10T10:38:33.290891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:13:39.930917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289511+0000","last_change":"2024-05-10T10:35:50.930620+0000","last_active":"2024-05-10T10:38:33.289511+0000","last_peered":"2024-05-10T10:38:33.289511+0000","last_clean":"2024-05-10T10:38:33.289511+0000","last_became_active":"2024-05-10T10:35:50.929696+0000","last_became_peered":"2024-05-10T10:35:50.929696+0000","last_unstale":"2024-05-10T10:38:33.289511+0000","last_undegraded":"2024-05-10T10:38:33.289511+0000","last_fullsized":"2024-05-10T10:38:33.289511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:23.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296397+0000","last_change":"2024-05-10T10:35:50.937517+0000","last_active":"2024-05-10T10:38:33.296397+0000","last_peered":"2024-05-10T10:38:33.296397+0000","last_clean":"2024-05-10T10:38:33.296397+0000","last_became_active":"2024-05-10T10:35:50.937180+0000","last_became_peered":"2024-05-10T10:35:50.937180+0000","last_unstale":"2024-05-10T10:38:33.296397+0000","last_undegraded":"2024-05-10T10:38:33.296397+0000","last_fullsized":"2024-05-10T10:38:33.296397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.568509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293452+0000","last_change":"2024-05-10T10:35:50.937825+0000","last_active":"2024-05-10T10:38:33.293452+0000","last_peered":"2024-05-10T10:38:33.293452+0000","last_clean":"2024-05-10T10:38:33.293452+0000","last_became_active":"2024-05-10T10:35:50.935650+0000","last_became_peered":"2024-05-10T10:35:50.935650+0000","last_unstale":"2024-05-10T10:38:33.293452+0000","last_undegraded":"2024-05-10T10:38:33.293452+0000","last_fullsized":"2024-05-10T10:38:33.293452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:43:55.780266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285811+0000","last_change":"2024-05-10T10:35:50.933598+0000","last_active":"2024-05-10T10:38:33.285811+0000","last_peered":"2024-05-10T10:38:33.285811+0000","last_clean":"2024-05-10T10:38:33.285811+0000","last_became_active":"2024-05-10T10:35:50.932767+0000","last_became_peered":"2024-05-10T10:35:50.932767+0000","last_unstale":"2024-05-10T10:38:33.285811+0000","last_undegraded":"2024-05-10T10:38:33.285811+0000","last_fullsized":"2024-05-10T10:38:33.285811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:40.610376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287212+0000","last_change":"2024-05-10T10:35:50.934882+0000","last_active":"2024-05-10T10:38:33.287212+0000","last_peered":"2024-05-10T10:38:33.287212+0000","last_clean":"2024-05-10T10:38:33.287212+0000","last_became_active":"2024-05-10T10:35:50.934528+0000","last_became_peered":"2024-05-10T10:35:50.934528+0000","last_unstale":"2024-05-10T10:38:33.287212+0000","last_undegraded":"2024-05-10T10:38:33.287212+0000","last_fullsized":"2024-05-10T10:38:33.287212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:15.319765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289469+0000","last_change":"2024-05-10T10:36:35.110939+0000","last_active":"2024-05-10T10:38:33.289469+0000","last_peered":"2024-05-10T10:38:33.289469+0000","last_clean":"2024-05-10T10:38:33.289469+0000","last_became_active":"2024-05-10T10:36:35.110600+0000","last_became_peered":"2024-05-10T10:36:35.110600+0000","last_unstale":"2024-05-10T10:38:33.289469+0000","last_undegraded":"2024-05-10T10:38:33.289469+0000","last_fullsized":"2024-05-10T10:38:33.289469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:45:58.547301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.297102+0000","last_change":"2024-05-10T10:35:50.934161+0000","last_active":"2024-05-10T10:38:33.297102+0000","last_peered":"2024-05-10T10:38:33.297102+0000","last_clean":"2024-05-10T10:38:33.297102+0000","last_became_active":"2024-05-10T10:35:50.933831+0000","last_became_peered":"2024-05-10T10:35:50.933831+0000","last_unstale":"2024-05-10T10:38:33.297102+0000","last_undegraded":"2024-05-10T10:38:33.297102+0000","last_fullsized":"2024-05-10T10:38:33.297102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:09.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286140+0000","last_change":"2024-05-10T10:35:50.927404+0000","last_active":"2024-05-10T10:38:33.286140+0000","last_peered":"2024-05-10T10:38:33.286140+0000","last_clean":"2024-05-10T10:38:33.286140+0000","last_became_active":"2024-05-10T10:35:50.927094+0000","last_became_peered":"2024-05-10T10:35:50.927094+0000","last_unstale":"2024-05-10T10:38:33.286140+0000","last_undegraded":"2024-05-10T10:38:33.286140+0000","last_fullsized":"2024-05-10T10:38:33.286140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:25.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293207+0000","last_change":"2024-05-10T10:36:35.142063+0000","last_active":"2024-05-10T10:38:33.293207+0000","last_peered":"2024-05-10T10:38:33.293207+0000","last_clean":"2024-05-10T10:38:33.293207+0000","last_became_active":"2024-05-10T10:36:35.141685+0000","last_became_peered":"2024-05-10T10:36:35.141685+0000","last_unstale":"2024-05-10T10:38:33.293207+0000","last_undegraded":"2024-05-10T10:38:33.293207+0000","last_fullsized":"2024-05-10T10:38:33.293207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:47.284288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294424+0000","last_change":"2024-05-10T10:35:50.933172+0000","last_active":"2024-05-10T10:38:33.294424+0000","last_peered":"2024-05-10T10:38:33.294424+0000","last_clean":"2024-05-10T10:38:33.294424+0000","last_became_active":"2024-05-10T10:35:50.931864+0000","last_became_peered":"2024-05-10T10:35:50.931864+0000","last_unstale":"2024-05-10T10:38:33.294424+0000","last_undegraded":"2024-05-10T10:38:33.294424+0000","last_fullsized":"2024-05-10T10:38:33.294424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:50.620315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295591+0000","last_change":"2024-05-10T10:35:50.925952+0000","last_active":"2024-05-10T10:38:33.295591+0000","last_peered":"2024-05-10T10:38:33.295591+0000","last_clean":"2024-05-10T10:38:33.295591+0000","last_became_active":"2024-05-10T10:35:50.925610+0000","last_became_peered":"2024-05-10T10:35:50.925610+0000","last_unstale":"2024-05-10T10:38:33.295591+0000","last_undegraded":"2024-05-10T10:38:33.295591+0000","last_fullsized":"2024-05-10T10:38:33.295591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:38.287235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291553+0000","last_change":"2024-05-10T10:35:50.929379+0000","last_active":"2024-05-10T10:38:33.291553+0000","last_peered":"2024-05-10T10:38:33.291553+0000","last_clean":"2024-05-10T10:38:33.291553+0000","last_became_active":"2024-05-10T10:35:50.929082+0000","last_became_peered":"2024-05-10T10:35:50.929082+0000","last_unstale":"2024-05-10T10:38:33.291553+0000","last_undegraded":"2024-05-10T10:38:33.291553+0000","last_fullsized":"2024-05-10T10:38:33.291553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:54.097855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.283992+0000","last_change":"2024-05-10T10:35:50.937754+0000","last_active":"2024-05-10T10:38:33.283992+0000","last_peered":"2024-05-10T10:38:33.283992+0000","last_clean":"2024-05-10T10:38:33.283992+0000","last_became_active":"2024-05-10T10:35:50.935662+0000","last_became_peered":"2024-05-10T10:35:50.935662+0000","last_unstale":"2024-05-10T10:38:33.283992+0000","last_undegraded":"2024-05-10T10:38:33.283992+0000","last_fullsized":"2024-05-10T10:38:33.283992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:52.692279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571937+0000","last_change":"2024-05-10T10:36:01.989463+0000","last_active":"2024-05-10T10:37:35.571937+0000","last_peered":"2024-05-10T10:37:35.571937+0000","last_clean":"2024-05-10T10:37:35.571937+0000","last_became_active":"2024-05-10T10:36:01.988949+0000","last_became_peered":"2024-05-10T10:36:01.988949+0000","last_unstale":"2024-05-10T10:37:35.571937+0000","last_undegraded":"2024-05-10T10:37:35.571937+0000","last_fullsized":"2024-05-10T10:37:35.571937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:47.529845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:41.539200+0000","last_change":"2024-05-10T10:35:47.909033+0000","last_active":"2024-05-10T10:38:41.539200+0000","last_peered":"2024-05-10T10:38:41.539200+0000","last_clean":"2024-05-10T10:38:41.539200+0000","last_became_active":"2024-05-10T10:35:47.908680+0000","last_became_peered":"2024-05-10T10:35:47.908680+0000","last_unstale":"2024-05-10T10:38:41.539200+0000","last_undegraded":"2024-05-10T10:38:41.539200+0000","last_fullsized":"2024-05-10T10:38:41.539200+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:46.642113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294517+0000","last_change":"2024-05-10T10:35:50.939233+0000","last_active":"2024-05-10T10:38:33.294517+0000","last_peered":"2024-05-10T10:38:33.294517+0000","last_clean":"2024-05-10T10:38:33.294517+0000","last_became_active":"2024-05-10T10:35:50.938949+0000","last_became_peered":"2024-05-10T10:35:50.938949+0000","last_unstale":"2024-05-10T10:38:33.294517+0000","last_undegraded":"2024-05-10T10:38:33.294517+0000","last_fullsized":"2024-05-10T10:38:33.294517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:06.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718685+0000","last_change":"2024-05-10T10:36:01.990982+0000","last_active":"2024-05-10T10:37:35.718685+0000","last_peered":"2024-05-10T10:37:35.718685+0000","last_clean":"2024-05-10T10:37:35.718685+0000","last_became_active":"2024-05-10T10:36:01.990325+0000","last_became_peered":"2024-05-10T10:36:01.990325+0000","last_unstale":"2024-05-10T10:37:35.718685+0000","last_undegraded":"2024-05-10T10:37:35.718685+0000","last_fullsized":"2024-05-10T10:37:35.718685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:18.424130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569481+0000","last_change":"2024-05-10T10:35:47.911239+0000","last_active":"2024-05-10T10:37:35.569481+0000","last_peered":"2024-05-10T10:37:35.569481+0000","last_clean":"2024-05-10T10:37:35.569481+0000","last_became_active":"2024-05-10T10:35:47.910945+0000","last_became_peered":"2024-05-10T10:35:47.910945+0000","last_unstale":"2024-05-10T10:37:35.569481+0000","last_undegraded":"2024-05-10T10:37:35.569481+0000","last_fullsized":"2024-05-10T10:37:35.569481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:08:52.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294558+0000","last_change":"2024-05-10T10:35:50.923694+0000","last_active":"2024-05-10T10:38:33.294558+0000","last_peered":"2024-05-10T10:38:33.294558+0000","last_clean":"2024-05-10T10:38:33.294558+0000","last_became_active":"2024-05-10T10:35:50.923381+0000","last_became_peered":"2024-05-10T10:35:50.923381+0000","last_unstale":"2024-05-10T10:38:33.294558+0000","last_undegraded":"2024-05-10T10:38:33.294558+0000","last_fullsized":"2024-05-10T10:38:33.294558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:59.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718663+0000","last_change":"2024-05-10T10:36:01.989868+0000","last_active":"2024-05-10T10:37:35.718663+0000","last_peered":"2024-05-10T10:37:35.718663+0000","last_clean":"2024-05-10T10:37:35.718663+0000","last_became_active":"2024-05-10T10:36:01.989283+0000","last_became_peered":"2024-05-10T10:36:01.989283+0000","last_unstale":"2024-05-10T10:37:35.718663+0000","last_undegraded":"2024-05-10T10:37:35.718663+0000","last_fullsized":"2024-05-10T10:37:35.718663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:35.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570220+0000","last_change":"2024-05-10T10:35:47.915555+0000","last_active":"2024-05-10T10:37:35.570220+0000","last_peered":"2024-05-10T10:37:35.570220+0000","last_clean":"2024-05-10T10:37:35.570220+0000","last_became_active":"2024-05-10T10:35:47.912852+0000","last_became_peered":"2024-05-10T10:35:47.912852+0000","last_unstale":"2024-05-10T10:37:35.570220+0000","last_undegraded":"2024-05-10T10:37:35.570220+0000","last_fullsized":"2024-05-10T10:37:35.570220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:56:45.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291583+0000","last_change":"2024-05-10T10:35:50.934010+0000","last_active":"2024-05-10T10:38:33.291583+0000","last_peered":"2024-05-10T10:38:33.291583+0000","last_clean":"2024-05-10T10:38:33.291583+0000","last_became_active":"2024-05-10T10:35:50.933653+0000","last_became_peered":"2024-05-10T10:35:50.933653+0000","last_unstale":"2024-05-10T10:38:33.291583+0000","last_undegraded":"2024-05-10T10:38:33.291583+0000","last_fullsized":"2024-05-10T10:38:33.291583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:34.862734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718657+0000","last_change":"2024-05-10T10:36:01.990524+0000","last_active":"2024-05-10T10:37:35.718657+0000","last_peered":"2024-05-10T10:37:35.718657+0000","last_clean":"2024-05-10T10:37:35.718657+0000","last_became_active":"2024-05-10T10:36:01.990011+0000","last_became_peered":"2024-05-10T10:36:01.990011+0000","last_unstale":"2024-05-10T10:37:35.718657+0000","last_undegraded":"2024-05-10T10:37:35.718657+0000","last_fullsized":"2024-05-10T10:37:35.718657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:06.123095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570506+0000","last_change":"2024-05-10T10:35:47.910977+0000","last_active":"2024-05-10T10:37:35.570506+0000","last_peered":"2024-05-10T10:37:35.570506+0000","last_clean":"2024-05-10T10:37:35.570506+0000","last_became_active":"2024-05-10T10:35:47.910623+0000","last_became_peered":"2024-05-10T10:35:47.910623+0000","last_unstale":"2024-05-10T10:37:35.570506+0000","last_undegraded":"2024-05-10T10:37:35.570506+0000","last_fullsized":"2024-05-10T10:37:35.570506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:37.760666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296932+0000","last_change":"2024-05-10T10:35:50.937238+0000","last_active":"2024-05-10T10:38:33.296932+0000","last_peered":"2024-05-10T10:38:33.296932+0000","last_clean":"2024-05-10T10:38:33.296932+0000","last_became_active":"2024-05-10T10:35:50.936298+0000","last_became_peered":"2024-05-10T10:35:50.936298+0000","last_unstale":"2024-05-10T10:38:33.296932+0000","last_undegraded":"2024-05-10T10:38:33.296932+0000","last_fullsized":"2024-05-10T10:38:33.296932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:54.724452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571851+0000","last_change":"2024-05-10T10:36:01.987769+0000","last_active":"2024-05-10T10:37:35.571851+0000","last_peered":"2024-05-10T10:37:35.571851+0000","last_clean":"2024-05-10T10:37:35.571851+0000","last_became_active":"2024-05-10T10:36:01.987367+0000","last_became_peered":"2024-05-10T10:36:01.987367+0000","last_unstale":"2024-05-10T10:37:35.571851+0000","last_undegraded":"2024-05-10T10:37:35.571851+0000","last_fullsized":"2024-05-10T10:37:35.571851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:36.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:38.536810+0000","last_change":"2024-05-10T10:35:47.915082+0000","last_active":"2024-05-10T10:38:38.536810+0000","last_peered":"2024-05-10T10:38:38.536810+0000","last_clean":"2024-05-10T10:38:38.536810+0000","last_became_active":"2024-05-10T10:35:47.914722+0000","last_became_peered":"2024-05-10T10:35:47.914722+0000","last_unstale":"2024-05-10T10:38:38.536810+0000","last_undegraded":"2024-05-10T10:38:38.536810+0000","last_fullsized":"2024-05-10T10:38:38.536810+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:31.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285496+0000","last_change":"2024-05-10T10:35:50.926181+0000","last_active":"2024-05-10T10:38:33.285496+0000","last_peered":"2024-05-10T10:38:33.285496+0000","last_clean":"2024-05-10T10:38:33.285496+0000","last_became_active":"2024-05-10T10:35:50.925865+0000","last_became_peered":"2024-05-10T10:35:50.925865+0000","last_unstale":"2024-05-10T10:38:33.285496+0000","last_undegraded":"2024-05-10T10:38:33.285496+0000","last_fullsized":"2024-05-10T10:38:33.285496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:04.976908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.572140+0000","last_change":"2024-05-10T10:36:35.142412+0000","last_active":"2024-05-10T10:37:35.572140+0000","last_peered":"2024-05-10T10:37:35.572140+0000","last_clean":"2024-05-10T10:37:35.572140+0000","last_became_active":"2024-05-10T10:36:35.141959+0000","last_became_peered":"2024-05-10T10:36:35.141959+0000","last_unstale":"2024-05-10T10:37:35.572140+0000","last_undegraded":"2024-05-10T10:37:35.572140+0000","last_fullsized":"2024-05-10T10:37:35.572140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:20:34.133261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720063+0000","last_change":"2024-05-10T10:35:47.912826+0000","last_active":"2024-05-10T10:37:35.720063+0000","last_peered":"2024-05-10T10:37:35.720063+0000","last_clean":"2024-05-10T10:37:35.720063+0000","last_became_active":"2024-05-10T10:35:47.912468+0000","last_became_peered":"2024-05-10T10:35:47.912468+0000","last_unstale":"2024-05-10T10:37:35.720063+0000","last_undegraded":"2024-05-10T10:37:35.720063+0000","last_fullsized":"2024-05-10T10:37:35.720063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:56.482104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293989+0000","last_change":"2024-05-10T10:36:35.114920+0000","last_active":"2024-05-10T10:38:33.293989+0000","last_peered":"2024-05-10T10:38:33.293989+0000","last_clean":"2024-05-10T10:38:33.293989+0000","last_became_active":"2024-05-10T10:36:35.114546+0000","last_became_peered":"2024-05-10T10:36:35.114546+0000","last_unstale":"2024-05-10T10:38:33.293989+0000","last_undegraded":"2024-05-10T10:38:33.293989+0000","last_fullsized":"2024-05-10T10:38:33.293989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:52.454111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570005+0000","last_change":"2024-05-10T10:36:01.990086+0000","last_active":"2024-05-10T10:37:35.570005+0000","last_peered":"2024-05-10T10:37:35.570005+0000","last_clean":"2024-05-10T10:37:35.570005+0000","last_became_active":"2024-05-10T10:36:01.989624+0000","last_became_peered":"2024-05-10T10:36:01.989624+0000","last_unstale":"2024-05-10T10:37:35.570005+0000","last_undegraded":"2024-05-10T10:37:35.570005+0000","last_fullsized":"2024-05-10T10:37:35.570005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:33.708069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564634+0000","last_change":"2024-05-10T10:35:47.905182+0000","last_active":"2024-05-10T10:37:35.564634+0000","last_peered":"2024-05-10T10:37:35.564634+0000","last_clean":"2024-05-10T10:37:35.564634+0000","last_became_active":"2024-05-10T10:35:47.904855+0000","last_became_peered":"2024-05-10T10:35:47.904855+0000","last_unstale":"2024-05-10T10:37:35.564634+0000","last_undegraded":"2024-05-10T10:37:35.564634+0000","last_fullsized":"2024-05-10T10:37:35.564634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:17:44.844156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292710+0000","last_change":"2024-05-10T10:35:50.935478+0000","last_active":"2024-05-10T10:38:33.292710+0000","last_peered":"2024-05-10T10:38:33.292710+0000","last_clean":"2024-05-10T10:38:33.292710+0000","last_became_active":"2024-05-10T10:35:50.933487+0000","last_became_peered":"2024-05-10T10:35:50.933487+0000","last_unstale":"2024-05-10T10:38:33.292710+0000","last_undegraded":"2024-05-10T10:38:33.292710+0000","last_fullsized":"2024-05-10T10:38:33.292710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:02.433423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718440+0000","last_change":"2024-05-10T10:36:01.984285+0000","last_active":"2024-05-10T10:37:35.718440+0000","last_peered":"2024-05-10T10:37:35.718440+0000","last_clean":"2024-05-10T10:37:35.718440+0000","last_became_active":"2024-05-10T10:36:01.983900+0000","last_became_peered":"2024-05-10T10:36:01.983900+0000","last_unstale":"2024-05-10T10:37:35.718440+0000","last_undegraded":"2024-05-10T10:37:35.718440+0000","last_fullsized":"2024-05-10T10:37:35.718440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:53.406106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565788+0000","last_change":"2024-05-10T10:35:47.911123+0000","last_active":"2024-05-10T10:37:35.565788+0000","last_peered":"2024-05-10T10:37:35.565788+0000","last_clean":"2024-05-10T10:37:35.565788+0000","last_became_active":"2024-05-10T10:35:47.910762+0000","last_became_peered":"2024-05-10T10:35:47.910762+0000","last_unstale":"2024-05-10T10:37:35.565788+0000","last_undegraded":"2024-05-10T10:37:35.565788+0000","last_fullsized":"2024-05-10T10:37:35.565788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:07.045586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288933+0000","last_change":"2024-05-10T10:35:50.928093+0000","last_active":"2024-05-10T10:38:33.288933+0000","last_peered":"2024-05-10T10:38:33.288933+0000","last_clean":"2024-05-10T10:38:33.288933+0000","last_became_active":"2024-05-10T10:35:50.926945+0000","last_became_peered":"2024-05-10T10:35:50.926945+0000","last_unstale":"2024-05-10T10:38:33.288933+0000","last_undegraded":"2024-05-10T10:38:33.288933+0000","last_fullsized":"2024-05-10T10:38:33.288933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:11.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569138+0000","last_change":"2024-05-10T10:36:35.143043+0000","last_active":"2024-05-10T10:37:35.569138+0000","last_peered":"2024-05-10T10:37:35.569138+0000","last_clean":"2024-05-10T10:37:35.569138+0000","last_became_active":"2024-05-10T10:36:35.142739+0000","last_became_peered":"2024-05-10T10:36:35.142739+0000","last_unstale":"2024-05-10T10:37:35.569138+0000","last_undegraded":"2024-05-10T10:37:35.569138+0000","last_fullsized":"2024-05-10T10:37:35.569138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:10.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":245,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:38.536069+0000","last_change":"2024-05-10T10:35:47.908687+0000","last_active":"2024-05-10T10:38:38.536069+0000","last_peered":"2024-05-10T10:38:38.536069+0000","last_clean":"2024-05-10T10:38:38.536069+0000","last_became_active":"2024-05-10T10:35:47.908349+0000","last_became_peered":"2024-05-10T10:35:47.908349+0000","last_unstale":"2024-05-10T10:38:38.536069+0000","last_undegraded":"2024-05-10T10:38:38.536069+0000","last_fullsized":"2024-05-10T10:38:38.536069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:48.696104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291371+0000","last_change":"2024-05-10T10:35:50.933729+0000","last_active":"2024-05-10T10:38:33.291371+0000","last_peered":"2024-05-10T10:38:33.291371+0000","last_clean":"2024-05-10T10:38:33.291371+0000","last_became_active":"2024-05-10T10:35:50.933109+0000","last_became_peered":"2024-05-10T10:35:50.933109+0000","last_unstale":"2024-05-10T10:38:33.291371+0000","last_undegraded":"2024-05-10T10:38:33.291371+0000","last_fullsized":"2024-05-10T10:38:33.291371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:10:38.305592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567135+0000","last_change":"2024-05-10T10:36:01.991969+0000","last_active":"2024-05-10T10:37:35.567135+0000","last_peered":"2024-05-10T10:37:35.567135+0000","last_clean":"2024-05-10T10:37:35.567135+0000","last_became_active":"2024-05-10T10:36:01.990278+0000","last_became_peered":"2024-05-10T10:36:01.990278+0000","last_unstale":"2024-05-10T10:37:35.567135+0000","last_undegraded":"2024-05-10T10:37:35.567135+0000","last_fullsized":"2024-05-10T10:37:35.567135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:33.070520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719303+0000","last_change":"2024-05-10T10:35:47.911682+0000","last_active":"2024-05-10T10:37:35.719303+0000","last_peered":"2024-05-10T10:37:35.719303+0000","last_clean":"2024-05-10T10:37:35.719303+0000","last_became_active":"2024-05-10T10:35:47.911328+0000","last_became_peered":"2024-05-10T10:35:47.911328+0000","last_unstale":"2024-05-10T10:37:35.719303+0000","last_undegraded":"2024-05-10T10:37:35.719303+0000","last_fullsized":"2024-05-10T10:37:35.719303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.361340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285082+0000","last_change":"2024-05-10T10:35:50.931541+0000","last_active":"2024-05-10T10:38:33.285082+0000","last_peered":"2024-05-10T10:38:33.285082+0000","last_clean":"2024-05-10T10:38:33.285082+0000","last_became_active":"2024-05-10T10:35:50.931244+0000","last_became_peered":"2024-05-10T10:35:50.931244+0000","last_unstale":"2024-05-10T10:38:33.285082+0000","last_undegraded":"2024-05-10T10:38:33.285082+0000","last_fullsized":"2024-05-10T10:38:33.285082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:38.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720689+0000","last_change":"2024-05-10T10:36:01.990390+0000","last_active":"2024-05-10T10:37:35.720689+0000","last_peered":"2024-05-10T10:37:35.720689+0000","last_clean":"2024-05-10T10:37:35.720689+0000","last_became_active":"2024-05-10T10:36:01.989434+0000","last_became_peered":"2024-05-10T10:36:01.989434+0000","last_unstale":"2024-05-10T10:37:35.720689+0000","last_undegraded":"2024-05-10T10:37:35.720689+0000","last_fullsized":"2024-05-10T10:37:35.720689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:53.129212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566392+0000","last_change":"2024-05-10T10:35:47.911590+0000","last_active":"2024-05-10T10:37:35.566392+0000","last_peered":"2024-05-10T10:37:35.566392+0000","last_clean":"2024-05-10T10:37:35.566392+0000","last_became_active":"2024-05-10T10:35:47.910950+0000","last_became_peered":"2024-05-10T10:35:47.910950+0000","last_unstale":"2024-05-10T10:37:35.566392+0000","last_undegraded":"2024-05-10T10:37:35.566392+0000","last_fullsized":"2024-05-10T10:37:35.566392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:18.844774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292200+0000","last_change":"2024-05-10T10:35:50.938174+0000","last_active":"2024-05-10T10:38:33.292200+0000","last_peered":"2024-05-10T10:38:33.292200+0000","last_clean":"2024-05-10T10:38:33.292200+0000","last_became_active":"2024-05-10T10:35:50.937618+0000","last_became_peered":"2024-05-10T10:35:50.937618+0000","last_unstale":"2024-05-10T10:38:33.292200+0000","last_undegraded":"2024-05-10T10:38:33.292200+0000","last_fullsized":"2024-05-10T10:38:33.292200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:26:48.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567141+0000","last_change":"2024-05-10T10:36:01.979226+0000","last_active":"2024-05-10T10:37:35.567141+0000","last_peered":"2024-05-10T10:37:35.567141+0000","last_clean":"2024-05-10T10:37:35.567141+0000","last_became_active":"2024-05-10T10:36:01.978847+0000","last_became_peered":"2024-05-10T10:36:01.978847+0000","last_unstale":"2024-05-10T10:37:35.567141+0000","last_undegraded":"2024-05-10T10:37:35.567141+0000","last_fullsized":"2024-05-10T10:37:35.567141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:10.629813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":307,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:38.536585+0000","last_change":"2024-05-10T10:36:35.150814+0000","last_active":"2024-05-10T10:38:38.536585+0000","last_peered":"2024-05-10T10:38:38.536585+0000","last_clean":"2024-05-10T10:38:38.536585+0000","last_became_active":"2024-05-10T10:36:35.142980+0000","last_became_peered":"2024-05-10T10:36:35.142980+0000","last_unstale":"2024-05-10T10:38:38.536585+0000","last_undegraded":"2024-05-10T10:38:38.536585+0000","last_fullsized":"2024-05-10T10:38:38.536585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:17.907275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67283,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":169,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290064+0000","last_change":"2024-05-10T10:35:50.925584+0000","last_active":"2024-05-10T10:38:33.290064+0000","last_peered":"2024-05-10T10:38:33.290064+0000","last_clean":"2024-05-10T10:38:33.290064+0000","last_became_active":"2024-05-10T10:35:50.925215+0000","last_became_peered":"2024-05-10T10:35:50.925215+0000","last_unstale":"2024-05-10T10:38:33.290064+0000","last_undegraded":"2024-05-10T10:38:33.290064+0000","last_fullsized":"2024-05-10T10:38:33.290064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:03.377110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566636+0000","last_change":"2024-05-10T10:36:01.985090+0000","last_active":"2024-05-10T10:37:35.566636+0000","last_peered":"2024-05-10T10:37:35.566636+0000","last_clean":"2024-05-10T10:37:35.566636+0000","last_became_active":"2024-05-10T10:36:01.984059+0000","last_became_peered":"2024-05-10T10:36:01.984059+0000","last_unstale":"2024-05-10T10:37:35.566636+0000","last_undegraded":"2024-05-10T10:37:35.566636+0000","last_fullsized":"2024-05-10T10:37:35.566636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:41.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571763+0000","last_change":"2024-05-10T10:35:47.914377+0000","last_active":"2024-05-10T10:37:35.571763+0000","last_peered":"2024-05-10T10:37:35.571763+0000","last_clean":"2024-05-10T10:37:35.571763+0000","last_became_active":"2024-05-10T10:35:47.913830+0000","last_became_peered":"2024-05-10T10:35:47.913830+0000","last_unstale":"2024-05-10T10:37:35.571763+0000","last_undegraded":"2024-05-10T10:37:35.571763+0000","last_fullsized":"2024-05-10T10:37:35.571763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:54:18.540952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284127+0000","last_change":"2024-05-10T10:35:50.932721+0000","last_active":"2024-05-10T10:38:33.284127+0000","last_peered":"2024-05-10T10:38:33.284127+0000","last_clean":"2024-05-10T10:38:33.284127+0000","last_became_active":"2024-05-10T10:35:50.932410+0000","last_became_peered":"2024-05-10T10:35:50.932410+0000","last_unstale":"2024-05-10T10:38:33.284127+0000","last_undegraded":"2024-05-10T10:38:33.284127+0000","last_fullsized":"2024-05-10T10:38:33.284127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:42.255534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569383+0000","last_change":"2024-05-10T10:36:35.143581+0000","last_active":"2024-05-10T10:37:35.569383+0000","last_peered":"2024-05-10T10:37:35.569383+0000","last_clean":"2024-05-10T10:37:35.569383+0000","last_became_active":"2024-05-10T10:36:35.142957+0000","last_became_peered":"2024-05-10T10:36:35.142957+0000","last_unstale":"2024-05-10T10:37:35.569383+0000","last_undegraded":"2024-05-10T10:37:35.569383+0000","last_fullsized":"2024-05-10T10:37:35.569383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:14:39.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566100+0000","last_change":"2024-05-10T10:35:47.908874+0000","last_active":"2024-05-10T10:37:35.566100+0000","last_peered":"2024-05-10T10:37:35.566100+0000","last_clean":"2024-05-10T10:37:35.566100+0000","last_became_active":"2024-05-10T10:35:47.908478+0000","last_became_peered":"2024-05-10T10:35:47.908478+0000","last_unstale":"2024-05-10T10:37:35.566100+0000","last_undegraded":"2024-05-10T10:37:35.566100+0000","last_fullsized":"2024-05-10T10:37:35.566100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:13.147762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285463+0000","last_change":"2024-05-10T10:35:50.925168+0000","last_active":"2024-05-10T10:38:33.285463+0000","last_peered":"2024-05-10T10:38:33.285463+0000","last_clean":"2024-05-10T10:38:33.285463+0000","last_became_active":"2024-05-10T10:35:50.924509+0000","last_became_peered":"2024-05-10T10:35:50.924509+0000","last_unstale":"2024-05-10T10:38:33.285463+0000","last_undegraded":"2024-05-10T10:38:33.285463+0000","last_fullsized":"2024-05-10T10:38:33.285463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:08.458195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569666+0000","last_change":"2024-05-10T10:36:35.144575+0000","last_active":"2024-05-10T10:37:35.569666+0000","last_peered":"2024-05-10T10:37:35.569666+0000","last_clean":"2024-05-10T10:37:35.569666+0000","last_became_active":"2024-05-10T10:36:35.144165+0000","last_became_peered":"2024-05-10T10:36:35.144165+0000","last_unstale":"2024-05-10T10:37:35.569666+0000","last_undegraded":"2024-05-10T10:37:35.569666+0000","last_fullsized":"2024-05-10T10:37:35.569666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:45.179163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:38.536715+0000","last_change":"2024-05-10T10:35:47.911208+0000","last_active":"2024-05-10T10:38:38.536715+0000","last_peered":"2024-05-10T10:38:38.536715+0000","last_clean":"2024-05-10T10:38:38.536715+0000","last_became_active":"2024-05-10T10:35:47.910820+0000","last_became_peered":"2024-05-10T10:35:47.910820+0000","last_unstale":"2024-05-10T10:38:38.536715+0000","last_undegraded":"2024-05-10T10:38:38.536715+0000","last_fullsized":"2024-05-10T10:38:38.536715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:04.804203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289463+0000","last_change":"2024-05-10T10:35:50.928195+0000","last_active":"2024-05-10T10:38:33.289463+0000","last_peered":"2024-05-10T10:38:33.289463+0000","last_clean":"2024-05-10T10:38:33.289463+0000","last_became_active":"2024-05-10T10:35:50.927832+0000","last_became_peered":"2024-05-10T10:35:50.927832+0000","last_unstale":"2024-05-10T10:38:33.289463+0000","last_undegraded":"2024-05-10T10:38:33.289463+0000","last_fullsized":"2024-05-10T10:38:33.289463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:03.071525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565054+0000","last_change":"2024-05-10T10:36:01.982483+0000","last_active":"2024-05-10T10:37:35.565054+0000","last_peered":"2024-05-10T10:37:35.565054+0000","last_clean":"2024-05-10T10:37:35.565054+0000","last_became_active":"2024-05-10T10:36:01.982117+0000","last_became_peered":"2024-05-10T10:36:01.982117+0000","last_unstale":"2024-05-10T10:37:35.565054+0000","last_undegraded":"2024-05-10T10:37:35.565054+0000","last_fullsized":"2024-05-10T10:37:35.565054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:08.145035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569474+0000","last_change":"2024-05-10T10:35:47.912929+0000","last_active":"2024-05-10T10:37:35.569474+0000","last_peered":"2024-05-10T10:37:35.569474+0000","last_clean":"2024-05-10T10:37:35.569474+0000","last_became_active":"2024-05-10T10:35:47.912575+0000","last_became_peered":"2024-05-10T10:35:47.912575+0000","last_unstale":"2024-05-10T10:37:35.569474+0000","last_undegraded":"2024-05-10T10:37:35.569474+0000","last_fullsized":"2024-05-10T10:37:35.569474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:03.235579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296070+0000","last_change":"2024-05-10T10:35:50.935967+0000","last_active":"2024-05-10T10:38:33.296070+0000","last_peered":"2024-05-10T10:38:33.296070+0000","last_clean":"2024-05-10T10:38:33.296070+0000","last_became_active":"2024-05-10T10:35:50.935635+0000","last_became_peered":"2024-05-10T10:35:50.935635+0000","last_unstale":"2024-05-10T10:38:33.296070+0000","last_undegraded":"2024-05-10T10:38:33.296070+0000","last_fullsized":"2024-05-10T10:38:33.296070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:02:53.681522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565799+0000","last_change":"2024-05-10T10:36:01.979488+0000","last_active":"2024-05-10T10:37:35.565799+0000","last_peered":"2024-05-10T10:37:35.565799+0000","last_clean":"2024-05-10T10:37:35.565799+0000","last_became_active":"2024-05-10T10:36:01.978839+0000","last_became_peered":"2024-05-10T10:36:01.978839+0000","last_unstale":"2024-05-10T10:37:35.565799+0000","last_undegraded":"2024-05-10T10:37:35.565799+0000","last_fullsized":"2024-05-10T10:37:35.565799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:20.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570423+0000","last_change":"2024-05-10T10:35:47.913235+0000","last_active":"2024-05-10T10:37:35.570423+0000","last_peered":"2024-05-10T10:37:35.570423+0000","last_clean":"2024-05-10T10:37:35.570423+0000","last_became_active":"2024-05-10T10:35:47.912788+0000","last_became_peered":"2024-05-10T10:35:47.912788+0000","last_unstale":"2024-05-10T10:37:35.570423+0000","last_undegraded":"2024-05-10T10:37:35.570423+0000","last_fullsized":"2024-05-10T10:37:35.570423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:56:10.275488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290951+0000","last_change":"2024-05-10T10:35:50.930269+0000","last_active":"2024-05-10T10:38:33.290951+0000","last_peered":"2024-05-10T10:38:33.290951+0000","last_clean":"2024-05-10T10:38:33.290951+0000","last_became_active":"2024-05-10T10:35:50.929918+0000","last_became_peered":"2024-05-10T10:35:50.929918+0000","last_unstale":"2024-05-10T10:38:33.290951+0000","last_undegraded":"2024-05-10T10:38:33.290951+0000","last_fullsized":"2024-05-10T10:38:33.290951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:31.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569677+0000","last_change":"2024-05-10T10:36:02.018506+0000","last_active":"2024-05-10T10:37:35.569677+0000","last_peered":"2024-05-10T10:37:35.569677+0000","last_clean":"2024-05-10T10:37:35.569677+0000","last_became_active":"2024-05-10T10:36:02.018004+0000","last_became_peered":"2024-05-10T10:36:02.018004+0000","last_unstale":"2024-05-10T10:37:35.569677+0000","last_undegraded":"2024-05-10T10:37:35.569677+0000","last_fullsized":"2024-05-10T10:37:35.569677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:37.404970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567545+0000","last_change":"2024-05-10T10:35:47.910654+0000","last_active":"2024-05-10T10:37:35.567545+0000","last_peered":"2024-05-10T10:37:35.567545+0000","last_clean":"2024-05-10T10:37:35.567545+0000","last_became_active":"2024-05-10T10:35:47.910338+0000","last_became_peered":"2024-05-10T10:35:47.910338+0000","last_unstale":"2024-05-10T10:37:35.567545+0000","last_undegraded":"2024-05-10T10:37:35.567545+0000","last_fullsized":"2024-05-10T10:37:35.567545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:07.127574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288361+0000","last_change":"2024-05-10T10:35:50.940950+0000","last_active":"2024-05-10T10:38:33.288361+0000","last_peered":"2024-05-10T10:38:33.288361+0000","last_clean":"2024-05-10T10:38:33.288361+0000","last_became_active":"2024-05-10T10:35:50.940562+0000","last_became_peered":"2024-05-10T10:35:50.940562+0000","last_unstale":"2024-05-10T10:38:33.288361+0000","last_undegraded":"2024-05-10T10:38:33.288361+0000","last_fullsized":"2024-05-10T10:38:33.288361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:24.825900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564778+0000","last_change":"2024-05-10T10:36:01.989277+0000","last_active":"2024-05-10T10:37:35.564778+0000","last_peered":"2024-05-10T10:37:35.564778+0000","last_clean":"2024-05-10T10:37:35.564778+0000","last_became_active":"2024-05-10T10:36:01.988866+0000","last_became_peered":"2024-05-10T10:36:01.988866+0000","last_unstale":"2024-05-10T10:37:35.564778+0000","last_undegraded":"2024-05-10T10:37:35.564778+0000","last_fullsized":"2024-05-10T10:37:35.564778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:08:29.740044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:39.537275+0000","last_change":"2024-05-10T10:35:47.908332+0000","last_active":"2024-05-10T10:38:39.537275+0000","last_peered":"2024-05-10T10:38:39.537275+0000","last_clean":"2024-05-10T10:38:39.537275+0000","last_became_active":"2024-05-10T10:35:47.907933+0000","last_became_peered":"2024-05-10T10:35:47.907933+0000","last_unstale":"2024-05-10T10:38:39.537275+0000","last_undegraded":"2024-05-10T10:38:39.537275+0000","last_fullsized":"2024-05-10T10:38:39.537275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:47:48.702338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288662+0000","last_change":"2024-05-10T10:35:50.934638+0000","last_active":"2024-05-10T10:38:33.288662+0000","last_peered":"2024-05-10T10:38:33.288662+0000","last_clean":"2024-05-10T10:38:33.288662+0000","last_became_active":"2024-05-10T10:35:50.934285+0000","last_became_peered":"2024-05-10T10:35:50.934285+0000","last_unstale":"2024-05-10T10:38:33.288662+0000","last_undegraded":"2024-05-10T10:38:33.288662+0000","last_fullsized":"2024-05-10T10:38:33.288662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:24:31.179005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565003+0000","last_change":"2024-05-10T10:36:02.018468+0000","last_active":"2024-05-10T10:37:35.565003+0000","last_peered":"2024-05-10T10:37:35.565003+0000","last_clean":"2024-05-10T10:37:35.565003+0000","last_became_active":"2024-05-10T10:36:02.018087+0000","last_became_peered":"2024-05-10T10:36:02.018087+0000","last_unstale":"2024-05-10T10:37:35.565003+0000","last_undegraded":"2024-05-10T10:37:35.565003+0000","last_fullsized":"2024-05-10T10:37:35.565003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:55.712012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295331+0000","last_change":"2024-05-10T10:35:50.933660+0000","last_active":"2024-05-10T10:38:33.295331+0000","last_peered":"2024-05-10T10:38:33.295331+0000","last_clean":"2024-05-10T10:38:33.295331+0000","last_became_active":"2024-05-10T10:35:50.933230+0000","last_became_peered":"2024-05-10T10:35:50.933230+0000","last_unstale":"2024-05-10T10:38:33.295331+0000","last_undegraded":"2024-05-10T10:38:33.295331+0000","last_fullsized":"2024-05-10T10:38:33.295331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:34:54.272278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:38.536191+0000","last_change":"2024-05-10T10:35:47.910161+0000","last_active":"2024-05-10T10:38:38.536191+0000","last_peered":"2024-05-10T10:38:38.536191+0000","last_clean":"2024-05-10T10:38:38.536191+0000","last_became_active":"2024-05-10T10:35:47.909875+0000","last_became_peered":"2024-05-10T10:35:47.909875+0000","last_unstale":"2024-05-10T10:38:38.536191+0000","last_undegraded":"2024-05-10T10:38:38.536191+0000","last_fullsized":"2024-05-10T10:38:38.536191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:21.620921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719393+0000","last_change":"2024-05-10T10:36:01.986755+0000","last_active":"2024-05-10T10:37:35.719393+0000","last_peered":"2024-05-10T10:37:35.719393+0000","last_clean":"2024-05-10T10:37:35.719393+0000","last_became_active":"2024-05-10T10:36:01.986340+0000","last_became_peered":"2024-05-10T10:36:01.986340+0000","last_unstale":"2024-05-10T10:37:35.719393+0000","last_undegraded":"2024-05-10T10:37:35.719393+0000","last_fullsized":"2024-05-10T10:37:35.719393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:38.434323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293573+0000","last_change":"2024-05-10T10:36:35.114218+0000","last_active":"2024-05-10T10:38:33.293573+0000","last_peered":"2024-05-10T10:38:33.293573+0000","last_clean":"2024-05-10T10:38:33.293573+0000","last_became_active":"2024-05-10T10:36:35.113876+0000","last_became_peered":"2024-05-10T10:36:35.113876+0000","last_unstale":"2024-05-10T10:38:33.293573+0000","last_undegraded":"2024-05-10T10:38:33.293573+0000","last_fullsized":"2024-05-10T10:38:33.293573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:21.415678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718798+0000","last_change":"2024-05-10T10:35:47.909161+0000","last_active":"2024-05-10T10:37:35.718798+0000","last_peered":"2024-05-10T10:37:35.718798+0000","last_clean":"2024-05-10T10:37:35.718798+0000","last_became_active":"2024-05-10T10:35:47.908546+0000","last_became_peered":"2024-05-10T10:35:47.908546+0000","last_unstale":"2024-05-10T10:37:35.718798+0000","last_undegraded":"2024-05-10T10:37:35.718798+0000","last_fullsized":"2024-05-10T10:37:35.718798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:52.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719177+0000","last_change":"2024-05-10T10:34:20.980948+0000","last_active":"2024-05-10T10:37:35.719177+0000","last_peered":"2024-05-10T10:37:35.719177+0000","last_clean":"2024-05-10T10:37:35.719177+0000","last_became_active":"2024-05-10T10:34:20.975662+0000","last_became_peered":"2024-05-10T10:34:20.975662+0000","last_unstale":"2024-05-10T10:37:35.719177+0000","last_undegraded":"2024-05-10T10:37:35.719177+0000","last_fullsized":"2024-05-10T10:37:35.719177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_clean_scrub_stamp":"2024-05-10T10:30:55.419211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:33.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566301+0000","last_change":"2024-05-10T10:36:02.018578+0000","last_active":"2024-05-10T10:37:35.566301+0000","last_peered":"2024-05-10T10:37:35.566301+0000","last_clean":"2024-05-10T10:37:35.566301+0000","last_became_active":"2024-05-10T10:36:02.018126+0000","last_became_peered":"2024-05-10T10:36:02.018126+0000","last_unstale":"2024-05-10T10:37:35.566301+0000","last_undegraded":"2024-05-10T10:37:35.566301+0000","last_fullsized":"2024-05-10T10:37:35.566301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:18:17.112432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285100+0000","last_change":"2024-05-10T10:35:50.927472+0000","last_active":"2024-05-10T10:38:33.285100+0000","last_peered":"2024-05-10T10:38:33.285100+0000","last_clean":"2024-05-10T10:38:33.285100+0000","last_became_active":"2024-05-10T10:35:50.927030+0000","last_became_peered":"2024-05-10T10:35:50.927030+0000","last_unstale":"2024-05-10T10:38:33.285100+0000","last_undegraded":"2024-05-10T10:38:33.285100+0000","last_fullsized":"2024-05-10T10:38:33.285100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:15:03.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719958+0000","last_change":"2024-05-10T10:35:47.912675+0000","last_active":"2024-05-10T10:37:35.719958+0000","last_peered":"2024-05-10T10:37:35.719958+0000","last_clean":"2024-05-10T10:37:35.719958+0000","last_became_active":"2024-05-10T10:35:47.912297+0000","last_became_peered":"2024-05-10T10:35:47.912297+0000","last_unstale":"2024-05-10T10:37:35.719958+0000","last_undegraded":"2024-05-10T10:37:35.719958+0000","last_fullsized":"2024-05-10T10:37:35.719958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:46.178221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569247+0000","last_change":"2024-05-10T10:36:01.981501+0000","last_active":"2024-05-10T10:37:35.569247+0000","last_peered":"2024-05-10T10:37:35.569247+0000","last_clean":"2024-05-10T10:37:35.569247+0000","last_became_active":"2024-05-10T10:36:01.981029+0000","last_became_peered":"2024-05-10T10:36:01.981029+0000","last_unstale":"2024-05-10T10:37:35.569247+0000","last_undegraded":"2024-05-10T10:37:35.569247+0000","last_fullsized":"2024-05-10T10:37:35.569247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:18.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570023+0000","last_change":"2024-05-10T10:35:47.910701+0000","last_active":"2024-05-10T10:37:35.570023+0000","last_peered":"2024-05-10T10:37:35.570023+0000","last_clean":"2024-05-10T10:37:35.570023+0000","last_became_active":"2024-05-10T10:35:47.910286+0000","last_became_peered":"2024-05-10T10:35:47.910286+0000","last_unstale":"2024-05-10T10:37:35.570023+0000","last_undegraded":"2024-05-10T10:37:35.570023+0000","last_fullsized":"2024-05-10T10:37:35.570023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:07.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286038+0000","last_change":"2024-05-10T10:35:50.926026+0000","last_active":"2024-05-10T10:38:33.286038+0000","last_peered":"2024-05-10T10:38:33.286038+0000","last_clean":"2024-05-10T10:38:33.286038+0000","last_became_active":"2024-05-10T10:35:50.925624+0000","last_became_peered":"2024-05-10T10:35:50.925624+0000","last_unstale":"2024-05-10T10:38:33.286038+0000","last_undegraded":"2024-05-10T10:38:33.286038+0000","last_fullsized":"2024-05-10T10:38:33.286038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:22:02.828681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569074+0000","last_change":"2024-05-10T10:36:02.018403+0000","last_active":"2024-05-10T10:37:35.569074+0000","last_peered":"2024-05-10T10:37:35.569074+0000","last_clean":"2024-05-10T10:37:35.569074+0000","last_became_active":"2024-05-10T10:36:02.017980+0000","last_became_peered":"2024-05-10T10:36:02.017980+0000","last_unstale":"2024-05-10T10:37:35.569074+0000","last_undegraded":"2024-05-10T10:37:35.569074+0000","last_fullsized":"2024-05-10T10:37:35.569074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:07.264730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:57.730122+0000","last_change":"2024-05-10T10:37:35.577657+0000","last_active":"2024-05-10T10:37:57.730122+0000","last_peered":"2024-05-10T10:37:57.730122+0000","last_clean":"2024-05-10T10:37:57.730122+0000","last_became_active":"2024-05-10T10:37:35.570901+0000","last_became_peered":"2024-05-10T10:37:35.570901+0000","last_unstale":"2024-05-10T10:37:57.730122+0000","last_undegraded":"2024-05-10T10:37:57.730122+0000","last_fullsized":"2024-05-10T10:37:57.730122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:30.088160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293071+0000","last_change":"2024-05-10T10:35:50.935817+0000","last_active":"2024-05-10T10:38:33.293071+0000","last_peered":"2024-05-10T10:38:33.293071+0000","last_clean":"2024-05-10T10:38:33.293071+0000","last_became_active":"2024-05-10T10:35:50.935515+0000","last_became_peered":"2024-05-10T10:35:50.935515+0000","last_unstale":"2024-05-10T10:38:33.293071+0000","last_undegraded":"2024-05-10T10:38:33.293071+0000","last_fullsized":"2024-05-10T10:38:33.293071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:49.042193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571516+0000","last_change":"2024-05-10T10:36:01.986002+0000","last_active":"2024-05-10T10:37:35.571516+0000","last_peered":"2024-05-10T10:37:35.571516+0000","last_clean":"2024-05-10T10:37:35.571516+0000","last_became_active":"2024-05-10T10:36:01.985558+0000","last_became_peered":"2024-05-10T10:36:01.985558+0000","last_unstale":"2024-05-10T10:37:35.571516+0000","last_undegraded":"2024-05-10T10:37:35.571516+0000","last_fullsized":"2024-05-10T10:37:35.571516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:27.732316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720203+0000","last_change":"2024-05-10T10:35:47.909254+0000","last_active":"2024-05-10T10:37:35.720203+0000","last_peered":"2024-05-10T10:37:35.720203+0000","last_clean":"2024-05-10T10:37:35.720203+0000","last_became_active":"2024-05-10T10:35:47.908370+0000","last_became_peered":"2024-05-10T10:35:47.908370+0000","last_unstale":"2024-05-10T10:37:35.720203+0000","last_undegraded":"2024-05-10T10:37:35.720203+0000","last_fullsized":"2024-05-10T10:37:35.720203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:44.643592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287742+0000","last_change":"2024-05-10T10:35:50.933001+0000","last_active":"2024-05-10T10:38:33.287742+0000","last_peered":"2024-05-10T10:38:33.287742+0000","last_clean":"2024-05-10T10:38:33.287742+0000","last_became_active":"2024-05-10T10:35:50.932010+0000","last_became_peered":"2024-05-10T10:35:50.932010+0000","last_unstale":"2024-05-10T10:38:33.287742+0000","last_undegraded":"2024-05-10T10:38:33.287742+0000","last_fullsized":"2024-05-10T10:38:33.287742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:01.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720023+0000","last_change":"2024-05-10T10:36:02.018650+0000","last_active":"2024-05-10T10:37:35.720023+0000","last_peered":"2024-05-10T10:37:35.720023+0000","last_clean":"2024-05-10T10:37:35.720023+0000","last_became_active":"2024-05-10T10:36:02.018258+0000","last_became_peered":"2024-05-10T10:36:02.018258+0000","last_unstale":"2024-05-10T10:37:35.720023+0000","last_undegraded":"2024-05-10T10:37:35.720023+0000","last_fullsized":"2024-05-10T10:37:35.720023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:33.590752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565584+0000","last_change":"2024-05-10T10:35:47.910419+0000","last_active":"2024-05-10T10:37:35.565584+0000","last_peered":"2024-05-10T10:37:35.565584+0000","last_clean":"2024-05-10T10:37:35.565584+0000","last_became_active":"2024-05-10T10:35:47.909930+0000","last_became_peered":"2024-05-10T10:35:47.909930+0000","last_unstale":"2024-05-10T10:37:35.565584+0000","last_undegraded":"2024-05-10T10:37:35.565584+0000","last_fullsized":"2024-05-10T10:37:35.565584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:32.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.250953+0000","last_change":"2024-05-10T10:35:50.923207+0000","last_active":"2024-05-10T10:38:33.250953+0000","last_peered":"2024-05-10T10:38:33.250953+0000","last_clean":"2024-05-10T10:38:33.250953+0000","last_became_active":"2024-05-10T10:35:50.922866+0000","last_became_peered":"2024-05-10T10:35:50.922866+0000","last_unstale":"2024-05-10T10:38:33.250953+0000","last_undegraded":"2024-05-10T10:38:33.250953+0000","last_fullsized":"2024-05-10T10:38:33.250953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:48.096498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567305+0000","last_change":"2024-05-10T10:36:01.985399+0000","last_active":"2024-05-10T10:37:35.567305+0000","last_peered":"2024-05-10T10:37:35.567305+0000","last_clean":"2024-05-10T10:37:35.567305+0000","last_became_active":"2024-05-10T10:36:01.985016+0000","last_became_peered":"2024-05-10T10:36:01.985016+0000","last_unstale":"2024-05-10T10:37:35.567305+0000","last_undegraded":"2024-05-10T10:37:35.567305+0000","last_fullsized":"2024-05-10T10:37:35.567305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:12.151693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569591+0000","last_change":"2024-05-10T10:35:47.908594+0000","last_active":"2024-05-10T10:37:35.569591+0000","last_peered":"2024-05-10T10:37:35.569591+0000","last_clean":"2024-05-10T10:37:35.569591+0000","last_became_active":"2024-05-10T10:35:47.908278+0000","last_became_peered":"2024-05-10T10:35:47.908278+0000","last_unstale":"2024-05-10T10:37:35.569591+0000","last_undegraded":"2024-05-10T10:37:35.569591+0000","last_fullsized":"2024-05-10T10:37:35.569591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:24.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294756+0000","last_change":"2024-05-10T10:35:50.933089+0000","last_active":"2024-05-10T10:38:33.294756+0000","last_peered":"2024-05-10T10:38:33.294756+0000","last_clean":"2024-05-10T10:38:33.294756+0000","last_became_active":"2024-05-10T10:35:50.932739+0000","last_became_peered":"2024-05-10T10:35:50.932739+0000","last_unstale":"2024-05-10T10:38:33.294756+0000","last_undegraded":"2024-05-10T10:38:33.294756+0000","last_fullsized":"2024-05-10T10:38:33.294756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:15.302832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720006+0000","last_change":"2024-05-10T10:36:01.988561+0000","last_active":"2024-05-10T10:37:35.720006+0000","last_peered":"2024-05-10T10:37:35.720006+0000","last_clean":"2024-05-10T10:37:35.720006+0000","last_became_active":"2024-05-10T10:36:01.988120+0000","last_became_peered":"2024-05-10T10:36:01.988120+0000","last_unstale":"2024-05-10T10:37:35.720006+0000","last_undegraded":"2024-05-10T10:37:35.720006+0000","last_fullsized":"2024-05-10T10:37:35.720006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:35:37.120469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.581302+0000","last_change":"2024-05-10T10:37:35.580875+0000","last_active":"2024-05-10T10:37:35.581302+0000","last_peered":"2024-05-10T10:37:35.581302+0000","last_clean":"2024-05-10T10:37:35.581302+0000","last_became_active":"2024-05-10T10:37:35.572187+0000","last_became_peered":"2024-05-10T10:37:35.572187+0000","last_unstale":"2024-05-10T10:37:35.581302+0000","last_undegraded":"2024-05-10T10:37:35.581302+0000","last_fullsized":"2024-05-10T10:37:35.581302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:06.330404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290565+0000","last_change":"2024-05-10T10:35:50.938466+0000","last_active":"2024-05-10T10:38:33.290565+0000","last_peered":"2024-05-10T10:38:33.290565+0000","last_clean":"2024-05-10T10:38:33.290565+0000","last_became_active":"2024-05-10T10:35:50.938167+0000","last_became_peered":"2024-05-10T10:35:50.938167+0000","last_unstale":"2024-05-10T10:38:33.290565+0000","last_undegraded":"2024-05-10T10:38:33.290565+0000","last_fullsized":"2024-05-10T10:38:33.290565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:18.014708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719597+0000","last_change":"2024-05-10T10:36:02.018661+0000","last_active":"2024-05-10T10:37:35.719597+0000","last_peered":"2024-05-10T10:37:35.719597+0000","last_clean":"2024-05-10T10:37:35.719597+0000","last_became_active":"2024-05-10T10:36:02.018196+0000","last_became_peered":"2024-05-10T10:36:02.018196+0000","last_unstale":"2024-05-10T10:37:35.719597+0000","last_undegraded":"2024-05-10T10:37:35.719597+0000","last_fullsized":"2024-05-10T10:37:35.719597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:11.605012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570017+0000","last_change":"2024-05-10T10:35:47.917009+0000","last_active":"2024-05-10T10:37:35.570017+0000","last_peered":"2024-05-10T10:37:35.570017+0000","last_clean":"2024-05-10T10:37:35.570017+0000","last_became_active":"2024-05-10T10:35:47.916536+0000","last_became_peered":"2024-05-10T10:35:47.916536+0000","last_unstale":"2024-05-10T10:37:35.570017+0000","last_undegraded":"2024-05-10T10:37:35.570017+0000","last_fullsized":"2024-05-10T10:37:35.570017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:17.783027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287009+0000","last_change":"2024-05-10T10:35:50.926173+0000","last_active":"2024-05-10T10:38:33.287009+0000","last_peered":"2024-05-10T10:38:33.287009+0000","last_clean":"2024-05-10T10:38:33.287009+0000","last_became_active":"2024-05-10T10:35:50.925846+0000","last_became_peered":"2024-05-10T10:35:50.925846+0000","last_unstale":"2024-05-10T10:38:33.287009+0000","last_undegraded":"2024-05-10T10:38:33.287009+0000","last_fullsized":"2024-05-10T10:38:33.287009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:29:02.587667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565679+0000","last_change":"2024-05-10T10:36:01.981312+0000","last_active":"2024-05-10T10:37:35.565679+0000","last_peered":"2024-05-10T10:37:35.565679+0000","last_clean":"2024-05-10T10:37:35.565679+0000","last_became_active":"2024-05-10T10:36:01.980861+0000","last_became_peered":"2024-05-10T10:36:01.980861+0000","last_unstale":"2024-05-10T10:37:35.565679+0000","last_undegraded":"2024-05-10T10:37:35.565679+0000","last_fullsized":"2024-05-10T10:37:35.565679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:29.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:38.536724+0000","last_change":"2024-05-10T10:35:47.911120+0000","last_active":"2024-05-10T10:38:38.536724+0000","last_peered":"2024-05-10T10:38:38.536724+0000","last_clean":"2024-05-10T10:38:38.536724+0000","last_became_active":"2024-05-10T10:35:47.909945+0000","last_became_peered":"2024-05-10T10:35:47.909945+0000","last_unstale":"2024-05-10T10:38:38.536724+0000","last_undegraded":"2024-05-10T10:38:38.536724+0000","last_fullsized":"2024-05-10T10:38:38.536724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:40.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287370+0000","last_change":"2024-05-10T10:35:50.933855+0000","last_active":"2024-05-10T10:38:33.287370+0000","last_peered":"2024-05-10T10:38:33.287370+0000","last_clean":"2024-05-10T10:38:33.287370+0000","last_became_active":"2024-05-10T10:35:50.933502+0000","last_became_peered":"2024-05-10T10:35:50.933502+0000","last_unstale":"2024-05-10T10:38:33.287370+0000","last_undegraded":"2024-05-10T10:38:33.287370+0000","last_fullsized":"2024-05-10T10:38:33.287370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:36.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718378+0000","last_change":"2024-05-10T10:36:01.985133+0000","last_active":"2024-05-10T10:37:35.718378+0000","last_peered":"2024-05-10T10:37:35.718378+0000","last_clean":"2024-05-10T10:37:35.718378+0000","last_became_active":"2024-05-10T10:36:01.984758+0000","last_became_peered":"2024-05-10T10:36:01.984758+0000","last_unstale":"2024-05-10T10:37:35.718378+0000","last_undegraded":"2024-05-10T10:37:35.718378+0000","last_fullsized":"2024-05-10T10:37:35.718378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:52.733063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566564+0000","last_change":"2024-05-10T10:35:47.908645+0000","last_active":"2024-05-10T10:37:35.566564+0000","last_peered":"2024-05-10T10:37:35.566564+0000","last_clean":"2024-05-10T10:37:35.566564+0000","last_became_active":"2024-05-10T10:35:47.908123+0000","last_became_peered":"2024-05-10T10:35:47.908123+0000","last_unstale":"2024-05-10T10:37:35.566564+0000","last_undegraded":"2024-05-10T10:37:35.566564+0000","last_fullsized":"2024-05-10T10:37:35.566564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:46.181724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289675+0000","last_change":"2024-05-10T10:35:50.929215+0000","last_active":"2024-05-10T10:38:33.289675+0000","last_peered":"2024-05-10T10:38:33.289675+0000","last_clean":"2024-05-10T10:38:33.289675+0000","last_became_active":"2024-05-10T10:35:50.928907+0000","last_became_peered":"2024-05-10T10:35:50.928907+0000","last_unstale":"2024-05-10T10:38:33.289675+0000","last_undegraded":"2024-05-10T10:38:33.289675+0000","last_fullsized":"2024-05-10T10:38:33.289675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:57:17.340687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565667+0000","last_change":"2024-05-10T10:36:01.988548+0000","last_active":"2024-05-10T10:37:35.565667+0000","last_peered":"2024-05-10T10:37:35.565667+0000","last_clean":"2024-05-10T10:37:35.565667+0000","last_became_active":"2024-05-10T10:36:01.987221+0000","last_became_peered":"2024-05-10T10:36:01.987221+0000","last_unstale":"2024-05-10T10:37:35.565667+0000","last_undegraded":"2024-05-10T10:37:35.565667+0000","last_fullsized":"2024-05-10T10:37:35.565667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:55:22.434536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565063+0000","last_change":"2024-05-10T10:35:47.914022+0000","last_active":"2024-05-10T10:37:35.565063+0000","last_peered":"2024-05-10T10:37:35.565063+0000","last_clean":"2024-05-10T10:37:35.565063+0000","last_became_active":"2024-05-10T10:35:47.913711+0000","last_became_peered":"2024-05-10T10:35:47.913711+0000","last_unstale":"2024-05-10T10:37:35.565063+0000","last_undegraded":"2024-05-10T10:37:35.565063+0000","last_fullsized":"2024-05-10T10:37:35.565063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:13.291307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284901+0000","last_change":"2024-05-10T10:35:50.927452+0000","last_active":"2024-05-10T10:38:33.284901+0000","last_peered":"2024-05-10T10:38:33.284901+0000","last_clean":"2024-05-10T10:38:33.284901+0000","last_became_active":"2024-05-10T10:35:50.927102+0000","last_became_peered":"2024-05-10T10:35:50.927102+0000","last_unstale":"2024-05-10T10:38:33.284901+0000","last_undegraded":"2024-05-10T10:38:33.284901+0000","last_fullsized":"2024-05-10T10:38:33.284901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:07.704196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293319+0000","last_change":"2024-05-10T10:35:50.933072+0000","last_active":"2024-05-10T10:38:33.293319+0000","last_peered":"2024-05-10T10:38:33.293319+0000","last_clean":"2024-05-10T10:38:33.293319+0000","last_became_active":"2024-05-10T10:35:50.932724+0000","last_became_peered":"2024-05-10T10:35:50.932724+0000","last_unstale":"2024-05-10T10:38:33.293319+0000","last_undegraded":"2024-05-10T10:38:33.293319+0000","last_fullsized":"2024-05-10T10:38:33.293319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:08.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287846+0000","last_change":"2024-05-10T10:35:50.931635+0000","last_active":"2024-05-10T10:38:33.287846+0000","last_peered":"2024-05-10T10:38:33.287846+0000","last_clean":"2024-05-10T10:38:33.287846+0000","last_became_active":"2024-05-10T10:35:50.930158+0000","last_became_peered":"2024-05-10T10:35:50.930158+0000","last_unstale":"2024-05-10T10:38:33.287846+0000","last_undegraded":"2024-05-10T10:38:33.287846+0000","last_fullsized":"2024-05-10T10:38:33.287846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:22.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285276+0000","last_change":"2024-05-10T10:35:50.936947+0000","last_active":"2024-05-10T10:38:33.285276+0000","last_peered":"2024-05-10T10:38:33.285276+0000","last_clean":"2024-05-10T10:38:33.285276+0000","last_became_active":"2024-05-10T10:35:50.936156+0000","last_became_peered":"2024-05-10T10:35:50.936156+0000","last_unstale":"2024-05-10T10:38:33.285276+0000","last_undegraded":"2024-05-10T10:38:33.285276+0000","last_fullsized":"2024-05-10T10:38:33.285276+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:21.167787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294617+0000","last_change":"2024-05-10T10:35:50.935384+0000","last_active":"2024-05-10T10:38:33.294617+0000","last_peered":"2024-05-10T10:38:33.294617+0000","last_clean":"2024-05-10T10:38:33.294617+0000","last_became_active":"2024-05-10T10:35:50.932555+0000","last_became_peered":"2024-05-10T10:35:50.932555+0000","last_unstale":"2024-05-10T10:38:33.294617+0000","last_undegraded":"2024-05-10T10:38:33.294617+0000","last_fullsized":"2024-05-10T10:38:33.294617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:09:11.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295227+0000","last_change":"2024-05-10T10:35:50.929322+0000","last_active":"2024-05-10T10:38:33.295227+0000","last_peered":"2024-05-10T10:38:33.295227+0000","last_clean":"2024-05-10T10:38:33.295227+0000","last_became_active":"2024-05-10T10:35:50.928366+0000","last_became_peered":"2024-05-10T10:35:50.928366+0000","last_unstale":"2024-05-10T10:38:33.295227+0000","last_undegraded":"2024-05-10T10:38:33.295227+0000","last_fullsized":"2024-05-10T10:38:33.295227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:32:28.337962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292799+0000","last_change":"2024-05-10T10:36:35.110922+0000","last_active":"2024-05-10T10:38:33.292799+0000","last_peered":"2024-05-10T10:38:33.292799+0000","last_clean":"2024-05-10T10:38:33.292799+0000","last_became_active":"2024-05-10T10:36:35.110525+0000","last_became_peered":"2024-05-10T10:36:35.110525+0000","last_unstale":"2024-05-10T10:38:33.292799+0000","last_undegraded":"2024-05-10T10:38:33.292799+0000","last_fullsized":"2024-05-10T10:38:33.292799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:47.825168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287609+0000","last_change":"2024-05-10T10:35:50.927869+0000","last_active":"2024-05-10T10:38:33.287609+0000","last_peered":"2024-05-10T10:38:33.287609+0000","last_clean":"2024-05-10T10:38:33.287609+0000","last_became_active":"2024-05-10T10:35:50.927552+0000","last_became_peered":"2024-05-10T10:35:50.927552+0000","last_unstale":"2024-05-10T10:38:33.287609+0000","last_undegraded":"2024-05-10T10:38:33.287609+0000","last_fullsized":"2024-05-10T10:38:33.287609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:36.962316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285786+0000","last_change":"2024-05-10T10:35:50.933099+0000","last_active":"2024-05-10T10:38:33.285786+0000","last_peered":"2024-05-10T10:38:33.285786+0000","last_clean":"2024-05-10T10:38:33.285786+0000","last_became_active":"2024-05-10T10:35:50.932765+0000","last_became_peered":"2024-05-10T10:35:50.932765+0000","last_unstale":"2024-05-10T10:38:33.285786+0000","last_undegraded":"2024-05-10T10:38:33.285786+0000","last_fullsized":"2024-05-10T10:38:33.285786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:19.095217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294630+0000","last_change":"2024-05-10T10:35:50.937704+0000","last_active":"2024-05-10T10:38:33.294630+0000","last_peered":"2024-05-10T10:38:33.294630+0000","last_clean":"2024-05-10T10:38:33.294630+0000","last_became_active":"2024-05-10T10:35:50.937236+0000","last_became_peered":"2024-05-10T10:35:50.937236+0000","last_unstale":"2024-05-10T10:38:33.294630+0000","last_undegraded":"2024-05-10T10:38:33.294630+0000","last_fullsized":"2024-05-10T10:38:33.294630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:46.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292751+0000","last_change":"2024-05-10T10:35:50.935181+0000","last_active":"2024-05-10T10:38:33.292751+0000","last_peered":"2024-05-10T10:38:33.292751+0000","last_clean":"2024-05-10T10:38:33.292751+0000","last_became_active":"2024-05-10T10:35:50.933446+0000","last_became_peered":"2024-05-10T10:35:50.933446+0000","last_unstale":"2024-05-10T10:38:33.292751+0000","last_undegraded":"2024-05-10T10:38:33.292751+0000","last_fullsized":"2024-05-10T10:38:33.292751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:03.972410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285940+0000","last_change":"2024-05-10T10:35:50.931797+0000","last_active":"2024-05-10T10:38:33.285940+0000","last_peered":"2024-05-10T10:38:33.285940+0000","last_clean":"2024-05-10T10:38:33.285940+0000","last_became_active":"2024-05-10T10:35:50.931494+0000","last_became_peered":"2024-05-10T10:35:50.931494+0000","last_unstale":"2024-05-10T10:38:33.285940+0000","last_undegraded":"2024-05-10T10:38:33.285940+0000","last_fullsized":"2024-05-10T10:38:33.285940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:30.797462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296149+0000","last_change":"2024-05-10T10:35:50.931688+0000","last_active":"2024-05-10T10:38:33.296149+0000","last_peered":"2024-05-10T10:38:33.296149+0000","last_clean":"2024-05-10T10:38:33.296149+0000","last_became_active":"2024-05-10T10:35:50.931362+0000","last_became_peered":"2024-05-10T10:35:50.931362+0000","last_unstale":"2024-05-10T10:38:33.296149+0000","last_undegraded":"2024-05-10T10:38:33.296149+0000","last_fullsized":"2024-05-10T10:38:33.296149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:24.263752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292719+0000","last_change":"2024-05-10T10:35:50.930905+0000","last_active":"2024-05-10T10:38:33.292719+0000","last_peered":"2024-05-10T10:38:33.292719+0000","last_clean":"2024-05-10T10:38:33.292719+0000","last_became_active":"2024-05-10T10:35:50.930448+0000","last_became_peered":"2024-05-10T10:35:50.930448+0000","last_unstale":"2024-05-10T10:38:33.292719+0000","last_undegraded":"2024-05-10T10:38:33.292719+0000","last_fullsized":"2024-05-10T10:38:33.292719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:52:24.468600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74237,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1032,"num_read_kb":598,"num_write":255,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3701,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":222711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874295,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1175621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037821,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430280,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1117936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626509,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":516424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":584737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":517880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":517674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":516132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986262,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":588313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822738,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1112986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:38:44.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T10:38:44.677 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:45.061 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:44 smithi112 ceph-mon[30100]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:38:45.062 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:44 smithi112 ceph-mon[30100]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:38:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:44 smithi148 ceph-mon[33568]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:38:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:44 smithi148 ceph-mon[33568]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:38:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:44 smithi195 ceph-mon[33773]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:38:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:44 smithi195 ceph-mon[33773]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:38:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 -- 172.21.15.112:0/2212424702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc1076e0 msgr2=0x7f1cfc10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 --2- 172.21.15.112:0/2212424702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc1076e0 0x7f1cfc10f2b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400b0d0 tx=0x7f1cf400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 -- 172.21.15.112:0/2212424702 shutdown_connections 2024-05-10T10:38:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 --2- 172.21.15.112:0/2212424702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc1076e0 0x7f1cfc10f2b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 --2- 172.21.15.112:0/2212424702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cfc102690 0x7f1cfc102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 --2- 172.21.15.112:0/2212424702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cfc103040 0x7f1cfc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 -- 172.21.15.112:0/2212424702 >> 172.21.15.112:0/2212424702 conn(0x7f1cfc0783f0 msgr2=0x7f1cfc078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 -- 172.21.15.112:0/2212424702 shutdown_connections 2024-05-10T10:38:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.414+0000 7f1d0178c700 1 -- 172.21.15.112:0/2212424702 wait complete. 2024-05-10T10:38:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 Processor -- start 2024-05-10T10:38:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 -- start start 2024-05-10T10:38:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cfc102690 0x7f1cfc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc103040 0x7f1cfc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cfc1076e0 0x7f1cfc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc113580 con 0x7f1cfc103040 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc113400 con 0x7f1cfc102690 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1d0178c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc113700 con 0x7f1cfc1076e0 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1cfa7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc103040 0x7f1cfc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1cfa7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc103040 0x7f1cfc19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46776/0 (socket says 172.21.15.112:46776) 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1cfa7fc700 1 -- 172.21.15.112:0/972553560 learned_addr learned my addr 172.21.15.112:0/972553560 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.415+0000 7f1cfb7fe700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cfc1076e0 0x7f1cfc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1cfaffd700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cfc102690 0x7f1cfc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1cfa7fc700 1 -- 172.21.15.112:0/972553560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cfc1076e0 msgr2=0x7f1cfc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1cfa7fc700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cfc1076e0 0x7f1cfc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1cfa7fc700 1 -- 172.21.15.112:0/972553560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cfc102690 msgr2=0x7f1cfc19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1cfa7fc700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cfc102690 0x7f1cfc19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1cfa7fc700 1 -- 172.21.15.112:0/972553560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc199fc0 con 0x7f1cfc103040 2024-05-10T10:38:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1cfa7fc700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc103040 0x7f1cfc19fba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000d950 tx=0x7f1cf000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1ce3fff700 1 -- 172.21.15.112:0/972553560 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0017070 con 0x7f1cfc103040 2024-05-10T10:38:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc19a2b0 con 0x7f1cfc103040 2024-05-10T10:38:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1ce3fff700 1 -- 172.21.15.112:0/972553560 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cf0005020 con 0x7f1cfc103040 2024-05-10T10:38:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1ce3fff700 1 -- 172.21.15.112:0/972553560 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0009a20 con 0x7f1cfc103040 2024-05-10T10:38:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.416+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc1a64c0 con 0x7f1cfc103040 2024-05-10T10:38:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.417+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cfc103ae0 con 0x7f1cfc103040 2024-05-10T10:38:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.418+0000 7f1ce3fff700 1 -- 172.21.15.112:0/972553560 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1cf0009bc0 con 0x7f1cfc103040 2024-05-10T10:38:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.419+0000 7f1ce3fff700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ce40b2880 0x7f1ce40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.419+0000 7f1cfaffd700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ce40b2880 0x7f1ce40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.419+0000 7f1ce3fff700 1 -- 172.21.15.112:0/972553560 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1cf00e21b0 con 0x7f1cfc103040 2024-05-10T10:38:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.420+0000 7f1cfaffd700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ce40b2880 0x7f1ce40b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1cec005ea0 tx=0x7f1cec005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:46.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.422+0000 7f1ce3fff700 1 -- 172.21.15.112:0/972553560 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1cf00a9b40 con 0x7f1cfc103040 2024-05-10T10:38:46.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.622+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cfc0638e0 con 0x7f1ce40b2880 2024-05-10T10:38:46.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.631+0000 7f1ce3fff700 1 -- 172.21.15.112:0/972553560 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340535 (secure 0 0 0) 0x7f1cfc0638e0 con 0x7f1ce40b2880 2024-05-10T10:38:46.638 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:38:46.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ce40b2880 msgr2=0x7f1ce40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:46.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ce40b2880 0x7f1ce40b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1cec005ea0 tx=0x7f1cec005e30 comp rx=0 tx=0).stop 2024-05-10T10:38:46.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc103040 msgr2=0x7f1cfc19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:46.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc103040 0x7f1cfc19fba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000d950 tx=0x7f1cf000dc60 comp rx=0 tx=0).stop 2024-05-10T10:38:46.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 shutdown_connections 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1ce40b2880 0x7f1ce40b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cfc1076e0 0x7f1cfc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cfc102690 0x7f1cfc19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 --2- 172.21.15.112:0/972553560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cfc103040 0x7f1cfc19fba0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 >> 172.21.15.112:0/972553560 conn(0x7f1cfc0783f0 msgr2=0x7f1cfc100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 shutdown_connections 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:46.635+0000 7f1d0178c700 1 -- 172.21.15.112:0/972553560 wait complete. 2024-05-10T10:38:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-10T10:38:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:46 smithi148 ceph-mon[33568]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:46 smithi195 ceph-mon[33773]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:47.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:46 smithi112 ceph-mon[30100]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:38:47.438 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-05-10T10:38:45.311400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664839,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1143,"num_read_kb":682,"num_write":380,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2955541,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367708,"kb_used_data":14396,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574116,"statfs":{"total":1151940427776,"available":1151563894784,"internally_reserved":0,"allocated":14741504,"data_stored":8215007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":72,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003358"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295460+0000","last_change":"2024-05-10T10:35:50.924449+0000","last_active":"2024-05-10T10:38:33.295460+0000","last_peered":"2024-05-10T10:38:33.295460+0000","last_clean":"2024-05-10T10:38:33.295460+0000","last_became_active":"2024-05-10T10:35:50.924114+0000","last_became_peered":"2024-05-10T10:35:50.924114+0000","last_unstale":"2024-05-10T10:38:33.295460+0000","last_undegraded":"2024-05-10T10:38:33.295460+0000","last_fullsized":"2024-05-10T10:38:33.295460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:50.636501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292146+0000","last_change":"2024-05-10T10:35:50.928715+0000","last_active":"2024-05-10T10:38:33.292146+0000","last_peered":"2024-05-10T10:38:33.292146+0000","last_clean":"2024-05-10T10:38:33.292146+0000","last_became_active":"2024-05-10T10:35:50.928404+0000","last_became_peered":"2024-05-10T10:35:50.928404+0000","last_unstale":"2024-05-10T10:38:33.292146+0000","last_undegraded":"2024-05-10T10:38:33.292146+0000","last_fullsized":"2024-05-10T10:38:33.292146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:44.934022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294960+0000","last_change":"2024-05-10T10:35:50.925134+0000","last_active":"2024-05-10T10:38:33.294960+0000","last_peered":"2024-05-10T10:38:33.294960+0000","last_clean":"2024-05-10T10:38:33.294960+0000","last_became_active":"2024-05-10T10:35:50.924797+0000","last_became_peered":"2024-05-10T10:35:50.924797+0000","last_unstale":"2024-05-10T10:38:33.294960+0000","last_undegraded":"2024-05-10T10:38:33.294960+0000","last_fullsized":"2024-05-10T10:38:33.294960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:43:52.666519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.269216+0000","last_change":"2024-05-10T10:35:50.930916+0000","last_active":"2024-05-10T10:38:33.269216+0000","last_peered":"2024-05-10T10:38:33.269216+0000","last_clean":"2024-05-10T10:38:33.269216+0000","last_became_active":"2024-05-10T10:35:50.930378+0000","last_became_peered":"2024-05-10T10:35:50.930378+0000","last_unstale":"2024-05-10T10:38:33.269216+0000","last_undegraded":"2024-05-10T10:38:33.269216+0000","last_fullsized":"2024-05-10T10:38:33.269216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:07.165595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.269516+0000","last_change":"2024-05-10T10:35:50.925459+0000","last_active":"2024-05-10T10:38:33.269516+0000","last_peered":"2024-05-10T10:38:33.269516+0000","last_clean":"2024-05-10T10:38:33.269516+0000","last_became_active":"2024-05-10T10:35:50.925061+0000","last_became_peered":"2024-05-10T10:35:50.925061+0000","last_unstale":"2024-05-10T10:38:33.269516+0000","last_undegraded":"2024-05-10T10:38:33.269516+0000","last_fullsized":"2024-05-10T10:38:33.269516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:48:06.622369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290891+0000","last_change":"2024-05-10T10:35:50.932052+0000","last_active":"2024-05-10T10:38:33.290891+0000","last_peered":"2024-05-10T10:38:33.290891+0000","last_clean":"2024-05-10T10:38:33.290891+0000","last_became_active":"2024-05-10T10:35:50.931685+0000","last_became_peered":"2024-05-10T10:35:50.931685+0000","last_unstale":"2024-05-10T10:38:33.290891+0000","last_undegraded":"2024-05-10T10:38:33.290891+0000","last_fullsized":"2024-05-10T10:38:33.290891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:13:39.930917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289511+0000","last_change":"2024-05-10T10:35:50.930620+0000","last_active":"2024-05-10T10:38:33.289511+0000","last_peered":"2024-05-10T10:38:33.289511+0000","last_clean":"2024-05-10T10:38:33.289511+0000","last_became_active":"2024-05-10T10:35:50.929696+0000","last_became_peered":"2024-05-10T10:35:50.929696+0000","last_unstale":"2024-05-10T10:38:33.289511+0000","last_undegraded":"2024-05-10T10:38:33.289511+0000","last_fullsized":"2024-05-10T10:38:33.289511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:23.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296397+0000","last_change":"2024-05-10T10:35:50.937517+0000","last_active":"2024-05-10T10:38:33.296397+0000","last_peered":"2024-05-10T10:38:33.296397+0000","last_clean":"2024-05-10T10:38:33.296397+0000","last_became_active":"2024-05-10T10:35:50.937180+0000","last_became_peered":"2024-05-10T10:35:50.937180+0000","last_unstale":"2024-05-10T10:38:33.296397+0000","last_undegraded":"2024-05-10T10:38:33.296397+0000","last_fullsized":"2024-05-10T10:38:33.296397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.568509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293452+0000","last_change":"2024-05-10T10:35:50.937825+0000","last_active":"2024-05-10T10:38:33.293452+0000","last_peered":"2024-05-10T10:38:33.293452+0000","last_clean":"2024-05-10T10:38:33.293452+0000","last_became_active":"2024-05-10T10:35:50.935650+0000","last_became_peered":"2024-05-10T10:35:50.935650+0000","last_unstale":"2024-05-10T10:38:33.293452+0000","last_undegraded":"2024-05-10T10:38:33.293452+0000","last_fullsized":"2024-05-10T10:38:33.293452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:43:55.780266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285811+0000","last_change":"2024-05-10T10:35:50.933598+0000","last_active":"2024-05-10T10:38:33.285811+0000","last_peered":"2024-05-10T10:38:33.285811+0000","last_clean":"2024-05-10T10:38:33.285811+0000","last_became_active":"2024-05-10T10:35:50.932767+0000","last_became_peered":"2024-05-10T10:35:50.932767+0000","last_unstale":"2024-05-10T10:38:33.285811+0000","last_undegraded":"2024-05-10T10:38:33.285811+0000","last_fullsized":"2024-05-10T10:38:33.285811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:40.610376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287212+0000","last_change":"2024-05-10T10:35:50.934882+0000","last_active":"2024-05-10T10:38:33.287212+0000","last_peered":"2024-05-10T10:38:33.287212+0000","last_clean":"2024-05-10T10:38:33.287212+0000","last_became_active":"2024-05-10T10:35:50.934528+0000","last_became_peered":"2024-05-10T10:35:50.934528+0000","last_unstale":"2024-05-10T10:38:33.287212+0000","last_undegraded":"2024-05-10T10:38:33.287212+0000","last_fullsized":"2024-05-10T10:38:33.287212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:15.319765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289469+0000","last_change":"2024-05-10T10:36:35.110939+0000","last_active":"2024-05-10T10:38:33.289469+0000","last_peered":"2024-05-10T10:38:33.289469+0000","last_clean":"2024-05-10T10:38:33.289469+0000","last_became_active":"2024-05-10T10:36:35.110600+0000","last_became_peered":"2024-05-10T10:36:35.110600+0000","last_unstale":"2024-05-10T10:38:33.289469+0000","last_undegraded":"2024-05-10T10:38:33.289469+0000","last_fullsized":"2024-05-10T10:38:33.289469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:45:58.547301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.297102+0000","last_change":"2024-05-10T10:35:50.934161+0000","last_active":"2024-05-10T10:38:33.297102+0000","last_peered":"2024-05-10T10:38:33.297102+0000","last_clean":"2024-05-10T10:38:33.297102+0000","last_became_active":"2024-05-10T10:35:50.933831+0000","last_became_peered":"2024-05-10T10:35:50.933831+0000","last_unstale":"2024-05-10T10:38:33.297102+0000","last_undegraded":"2024-05-10T10:38:33.297102+0000","last_fullsized":"2024-05-10T10:38:33.297102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:09.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286140+0000","last_change":"2024-05-10T10:35:50.927404+0000","last_active":"2024-05-10T10:38:33.286140+0000","last_peered":"2024-05-10T10:38:33.286140+0000","last_clean":"2024-05-10T10:38:33.286140+0000","last_became_active":"2024-05-10T10:35:50.927094+0000","last_became_peered":"2024-05-10T10:35:50.927094+0000","last_unstale":"2024-05-10T10:38:33.286140+0000","last_undegraded":"2024-05-10T10:38:33.286140+0000","last_fullsized":"2024-05-10T10:38:33.286140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:25.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293207+0000","last_change":"2024-05-10T10:36:35.142063+0000","last_active":"2024-05-10T10:38:33.293207+0000","last_peered":"2024-05-10T10:38:33.293207+0000","last_clean":"2024-05-10T10:38:33.293207+0000","last_became_active":"2024-05-10T10:36:35.141685+0000","last_became_peered":"2024-05-10T10:36:35.141685+0000","last_unstale":"2024-05-10T10:38:33.293207+0000","last_undegraded":"2024-05-10T10:38:33.293207+0000","last_fullsized":"2024-05-10T10:38:33.293207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:47.284288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294424+0000","last_change":"2024-05-10T10:35:50.933172+0000","last_active":"2024-05-10T10:38:33.294424+0000","last_peered":"2024-05-10T10:38:33.294424+0000","last_clean":"2024-05-10T10:38:33.294424+0000","last_became_active":"2024-05-10T10:35:50.931864+0000","last_became_peered":"2024-05-10T10:35:50.931864+0000","last_unstale":"2024-05-10T10:38:33.294424+0000","last_undegraded":"2024-05-10T10:38:33.294424+0000","last_fullsized":"2024-05-10T10:38:33.294424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:50.620315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295591+0000","last_change":"2024-05-10T10:35:50.925952+0000","last_active":"2024-05-10T10:38:33.295591+0000","last_peered":"2024-05-10T10:38:33.295591+0000","last_clean":"2024-05-10T10:38:33.295591+0000","last_became_active":"2024-05-10T10:35:50.925610+0000","last_became_peered":"2024-05-10T10:35:50.925610+0000","last_unstale":"2024-05-10T10:38:33.295591+0000","last_undegraded":"2024-05-10T10:38:33.295591+0000","last_fullsized":"2024-05-10T10:38:33.295591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:38.287235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291553+0000","last_change":"2024-05-10T10:35:50.929379+0000","last_active":"2024-05-10T10:38:33.291553+0000","last_peered":"2024-05-10T10:38:33.291553+0000","last_clean":"2024-05-10T10:38:33.291553+0000","last_became_active":"2024-05-10T10:35:50.929082+0000","last_became_peered":"2024-05-10T10:35:50.929082+0000","last_unstale":"2024-05-10T10:38:33.291553+0000","last_undegraded":"2024-05-10T10:38:33.291553+0000","last_fullsized":"2024-05-10T10:38:33.291553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:54.097855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.283992+0000","last_change":"2024-05-10T10:35:50.937754+0000","last_active":"2024-05-10T10:38:33.283992+0000","last_peered":"2024-05-10T10:38:33.283992+0000","last_clean":"2024-05-10T10:38:33.283992+0000","last_became_active":"2024-05-10T10:35:50.935662+0000","last_became_peered":"2024-05-10T10:35:50.935662+0000","last_unstale":"2024-05-10T10:38:33.283992+0000","last_undegraded":"2024-05-10T10:38:33.283992+0000","last_fullsized":"2024-05-10T10:38:33.283992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:52.692279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571937+0000","last_change":"2024-05-10T10:36:01.989463+0000","last_active":"2024-05-10T10:37:35.571937+0000","last_peered":"2024-05-10T10:37:35.571937+0000","last_clean":"2024-05-10T10:37:35.571937+0000","last_became_active":"2024-05-10T10:36:01.988949+0000","last_became_peered":"2024-05-10T10:36:01.988949+0000","last_unstale":"2024-05-10T10:37:35.571937+0000","last_undegraded":"2024-05-10T10:37:35.571937+0000","last_fullsized":"2024-05-10T10:37:35.571937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:47.529845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:41.539200+0000","last_change":"2024-05-10T10:35:47.909033+0000","last_active":"2024-05-10T10:38:41.539200+0000","last_peered":"2024-05-10T10:38:41.539200+0000","last_clean":"2024-05-10T10:38:41.539200+0000","last_became_active":"2024-05-10T10:35:47.908680+0000","last_became_peered":"2024-05-10T10:35:47.908680+0000","last_unstale":"2024-05-10T10:38:41.539200+0000","last_undegraded":"2024-05-10T10:38:41.539200+0000","last_fullsized":"2024-05-10T10:38:41.539200+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:46.642113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294517+0000","last_change":"2024-05-10T10:35:50.939233+0000","last_active":"2024-05-10T10:38:33.294517+0000","last_peered":"2024-05-10T10:38:33.294517+0000","last_clean":"2024-05-10T10:38:33.294517+0000","last_became_active":"2024-05-10T10:35:50.938949+0000","last_became_peered":"2024-05-10T10:35:50.938949+0000","last_unstale":"2024-05-10T10:38:33.294517+0000","last_undegraded":"2024-05-10T10:38:33.294517+0000","last_fullsized":"2024-05-10T10:38:33.294517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:06.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718685+0000","last_change":"2024-05-10T10:36:01.990982+0000","last_active":"2024-05-10T10:37:35.718685+0000","last_peered":"2024-05-10T10:37:35.718685+0000","last_clean":"2024-05-10T10:37:35.718685+0000","last_became_active":"2024-05-10T10:36:01.990325+0000","last_became_peered":"2024-05-10T10:36:01.990325+0000","last_unstale":"2024-05-10T10:37:35.718685+0000","last_undegraded":"2024-05-10T10:37:35.718685+0000","last_fullsized":"2024-05-10T10:37:35.718685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:18.424130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569481+0000","last_change":"2024-05-10T10:35:47.911239+0000","last_active":"2024-05-10T10:37:35.569481+0000","last_peered":"2024-05-10T10:37:35.569481+0000","last_clean":"2024-05-10T10:37:35.569481+0000","last_became_active":"2024-05-10T10:35:47.910945+0000","last_became_peered":"2024-05-10T10:35:47.910945+0000","last_unstale":"2024-05-10T10:37:35.569481+0000","last_undegraded":"2024-05-10T10:37:35.569481+0000","last_fullsized":"2024-05-10T10:37:35.569481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:08:52.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294558+0000","last_change":"2024-05-10T10:35:50.923694+0000","last_active":"2024-05-10T10:38:33.294558+0000","last_peered":"2024-05-10T10:38:33.294558+0000","last_clean":"2024-05-10T10:38:33.294558+0000","last_became_active":"2024-05-10T10:35:50.923381+0000","last_became_peered":"2024-05-10T10:35:50.923381+0000","last_unstale":"2024-05-10T10:38:33.294558+0000","last_undegraded":"2024-05-10T10:38:33.294558+0000","last_fullsized":"2024-05-10T10:38:33.294558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:59.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718663+0000","last_change":"2024-05-10T10:36:01.989868+0000","last_active":"2024-05-10T10:37:35.718663+0000","last_peered":"2024-05-10T10:37:35.718663+0000","last_clean":"2024-05-10T10:37:35.718663+0000","last_became_active":"2024-05-10T10:36:01.989283+0000","last_became_peered":"2024-05-10T10:36:01.989283+0000","last_unstale":"2024-05-10T10:37:35.718663+0000","last_undegraded":"2024-05-10T10:37:35.718663+0000","last_fullsized":"2024-05-10T10:37:35.718663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:35.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570220+0000","last_change":"2024-05-10T10:35:47.915555+0000","last_active":"2024-05-10T10:37:35.570220+0000","last_peered":"2024-05-10T10:37:35.570220+0000","last_clean":"2024-05-10T10:37:35.570220+0000","last_became_active":"2024-05-10T10:35:47.912852+0000","last_became_peered":"2024-05-10T10:35:47.912852+0000","last_unstale":"2024-05-10T10:37:35.570220+0000","last_undegraded":"2024-05-10T10:37:35.570220+0000","last_fullsized":"2024-05-10T10:37:35.570220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:56:45.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291583+0000","last_change":"2024-05-10T10:35:50.934010+0000","last_active":"2024-05-10T10:38:33.291583+0000","last_peered":"2024-05-10T10:38:33.291583+0000","last_clean":"2024-05-10T10:38:33.291583+0000","last_became_active":"2024-05-10T10:35:50.933653+0000","last_became_peered":"2024-05-10T10:35:50.933653+0000","last_unstale":"2024-05-10T10:38:33.291583+0000","last_undegraded":"2024-05-10T10:38:33.291583+0000","last_fullsized":"2024-05-10T10:38:33.291583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:34.862734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718657+0000","last_change":"2024-05-10T10:36:01.990524+0000","last_active":"2024-05-10T10:37:35.718657+0000","last_peered":"2024-05-10T10:37:35.718657+0000","last_clean":"2024-05-10T10:37:35.718657+0000","last_became_active":"2024-05-10T10:36:01.990011+0000","last_became_peered":"2024-05-10T10:36:01.990011+0000","last_unstale":"2024-05-10T10:37:35.718657+0000","last_undegraded":"2024-05-10T10:37:35.718657+0000","last_fullsized":"2024-05-10T10:37:35.718657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:06.123095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570506+0000","last_change":"2024-05-10T10:35:47.910977+0000","last_active":"2024-05-10T10:37:35.570506+0000","last_peered":"2024-05-10T10:37:35.570506+0000","last_clean":"2024-05-10T10:37:35.570506+0000","last_became_active":"2024-05-10T10:35:47.910623+0000","last_became_peered":"2024-05-10T10:35:47.910623+0000","last_unstale":"2024-05-10T10:37:35.570506+0000","last_undegraded":"2024-05-10T10:37:35.570506+0000","last_fullsized":"2024-05-10T10:37:35.570506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:37.760666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296932+0000","last_change":"2024-05-10T10:35:50.937238+0000","last_active":"2024-05-10T10:38:33.296932+0000","last_peered":"2024-05-10T10:38:33.296932+0000","last_clean":"2024-05-10T10:38:33.296932+0000","last_became_active":"2024-05-10T10:35:50.936298+0000","last_became_peered":"2024-05-10T10:35:50.936298+0000","last_unstale":"2024-05-10T10:38:33.296932+0000","last_undegraded":"2024-05-10T10:38:33.296932+0000","last_fullsized":"2024-05-10T10:38:33.296932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:54.724452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571851+0000","last_change":"2024-05-10T10:36:01.987769+0000","last_active":"2024-05-10T10:37:35.571851+0000","last_peered":"2024-05-10T10:37:35.571851+0000","last_clean":"2024-05-10T10:37:35.571851+0000","last_became_active":"2024-05-10T10:36:01.987367+0000","last_became_peered":"2024-05-10T10:36:01.987367+0000","last_unstale":"2024-05-10T10:37:35.571851+0000","last_undegraded":"2024-05-10T10:37:35.571851+0000","last_fullsized":"2024-05-10T10:37:35.571851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:36.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:43.544011+0000","last_change":"2024-05-10T10:35:47.915082+0000","last_active":"2024-05-10T10:38:43.544011+0000","last_peered":"2024-05-10T10:38:43.544011+0000","last_clean":"2024-05-10T10:38:43.544011+0000","last_became_active":"2024-05-10T10:35:47.914722+0000","last_became_peered":"2024-05-10T10:35:47.914722+0000","last_unstale":"2024-05-10T10:38:43.544011+0000","last_undegraded":"2024-05-10T10:38:43.544011+0000","last_fullsized":"2024-05-10T10:38:43.544011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:31.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285496+0000","last_change":"2024-05-10T10:35:50.926181+0000","last_active":"2024-05-10T10:38:33.285496+0000","last_peered":"2024-05-10T10:38:33.285496+0000","last_clean":"2024-05-10T10:38:33.285496+0000","last_became_active":"2024-05-10T10:35:50.925865+0000","last_became_peered":"2024-05-10T10:35:50.925865+0000","last_unstale":"2024-05-10T10:38:33.285496+0000","last_undegraded":"2024-05-10T10:38:33.285496+0000","last_fullsized":"2024-05-10T10:38:33.285496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:04.976908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.572140+0000","last_change":"2024-05-10T10:36:35.142412+0000","last_active":"2024-05-10T10:37:35.572140+0000","last_peered":"2024-05-10T10:37:35.572140+0000","last_clean":"2024-05-10T10:37:35.572140+0000","last_became_active":"2024-05-10T10:36:35.141959+0000","last_became_peered":"2024-05-10T10:36:35.141959+0000","last_unstale":"2024-05-10T10:37:35.572140+0000","last_undegraded":"2024-05-10T10:37:35.572140+0000","last_fullsized":"2024-05-10T10:37:35.572140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:20:34.133261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720063+0000","last_change":"2024-05-10T10:35:47.912826+0000","last_active":"2024-05-10T10:37:35.720063+0000","last_peered":"2024-05-10T10:37:35.720063+0000","last_clean":"2024-05-10T10:37:35.720063+0000","last_became_active":"2024-05-10T10:35:47.912468+0000","last_became_peered":"2024-05-10T10:35:47.912468+0000","last_unstale":"2024-05-10T10:37:35.720063+0000","last_undegraded":"2024-05-10T10:37:35.720063+0000","last_fullsized":"2024-05-10T10:37:35.720063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:56.482104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293989+0000","last_change":"2024-05-10T10:36:35.114920+0000","last_active":"2024-05-10T10:38:33.293989+0000","last_peered":"2024-05-10T10:38:33.293989+0000","last_clean":"2024-05-10T10:38:33.293989+0000","last_became_active":"2024-05-10T10:36:35.114546+0000","last_became_peered":"2024-05-10T10:36:35.114546+0000","last_unstale":"2024-05-10T10:38:33.293989+0000","last_undegraded":"2024-05-10T10:38:33.293989+0000","last_fullsized":"2024-05-10T10:38:33.293989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:52.454111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570005+0000","last_change":"2024-05-10T10:36:01.990086+0000","last_active":"2024-05-10T10:37:35.570005+0000","last_peered":"2024-05-10T10:37:35.570005+0000","last_clean":"2024-05-10T10:37:35.570005+0000","last_became_active":"2024-05-10T10:36:01.989624+0000","last_became_peered":"2024-05-10T10:36:01.989624+0000","last_unstale":"2024-05-10T10:37:35.570005+0000","last_undegraded":"2024-05-10T10:37:35.570005+0000","last_fullsized":"2024-05-10T10:37:35.570005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:33.708069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564634+0000","last_change":"2024-05-10T10:35:47.905182+0000","last_active":"2024-05-10T10:37:35.564634+0000","last_peered":"2024-05-10T10:37:35.564634+0000","last_clean":"2024-05-10T10:37:35.564634+0000","last_became_active":"2024-05-10T10:35:47.904855+0000","last_became_peered":"2024-05-10T10:35:47.904855+0000","last_unstale":"2024-05-10T10:37:35.564634+0000","last_undegraded":"2024-05-10T10:37:35.564634+0000","last_fullsized":"2024-05-10T10:37:35.564634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:17:44.844156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292710+0000","last_change":"2024-05-10T10:35:50.935478+0000","last_active":"2024-05-10T10:38:33.292710+0000","last_peered":"2024-05-10T10:38:33.292710+0000","last_clean":"2024-05-10T10:38:33.292710+0000","last_became_active":"2024-05-10T10:35:50.933487+0000","last_became_peered":"2024-05-10T10:35:50.933487+0000","last_unstale":"2024-05-10T10:38:33.292710+0000","last_undegraded":"2024-05-10T10:38:33.292710+0000","last_fullsized":"2024-05-10T10:38:33.292710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:02.433423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718440+0000","last_change":"2024-05-10T10:36:01.984285+0000","last_active":"2024-05-10T10:37:35.718440+0000","last_peered":"2024-05-10T10:37:35.718440+0000","last_clean":"2024-05-10T10:37:35.718440+0000","last_became_active":"2024-05-10T10:36:01.983900+0000","last_became_peered":"2024-05-10T10:36:01.983900+0000","last_unstale":"2024-05-10T10:37:35.718440+0000","last_undegraded":"2024-05-10T10:37:35.718440+0000","last_fullsized":"2024-05-10T10:37:35.718440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:53.406106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565788+0000","last_change":"2024-05-10T10:35:47.911123+0000","last_active":"2024-05-10T10:37:35.565788+0000","last_peered":"2024-05-10T10:37:35.565788+0000","last_clean":"2024-05-10T10:37:35.565788+0000","last_became_active":"2024-05-10T10:35:47.910762+0000","last_became_peered":"2024-05-10T10:35:47.910762+0000","last_unstale":"2024-05-10T10:37:35.565788+0000","last_undegraded":"2024-05-10T10:37:35.565788+0000","last_fullsized":"2024-05-10T10:37:35.565788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:07.045586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288933+0000","last_change":"2024-05-10T10:35:50.928093+0000","last_active":"2024-05-10T10:38:33.288933+0000","last_peered":"2024-05-10T10:38:33.288933+0000","last_clean":"2024-05-10T10:38:33.288933+0000","last_became_active":"2024-05-10T10:35:50.926945+0000","last_became_peered":"2024-05-10T10:35:50.926945+0000","last_unstale":"2024-05-10T10:38:33.288933+0000","last_undegraded":"2024-05-10T10:38:33.288933+0000","last_fullsized":"2024-05-10T10:38:33.288933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:11.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569138+0000","last_change":"2024-05-10T10:36:35.143043+0000","last_active":"2024-05-10T10:37:35.569138+0000","last_peered":"2024-05-10T10:37:35.569138+0000","last_clean":"2024-05-10T10:37:35.569138+0000","last_became_active":"2024-05-10T10:36:35.142739+0000","last_became_peered":"2024-05-10T10:36:35.142739+0000","last_unstale":"2024-05-10T10:37:35.569138+0000","last_undegraded":"2024-05-10T10:37:35.569138+0000","last_fullsized":"2024-05-10T10:37:35.569138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:10.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":255,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:43.543285+0000","last_change":"2024-05-10T10:35:47.908687+0000","last_active":"2024-05-10T10:38:43.543285+0000","last_peered":"2024-05-10T10:38:43.543285+0000","last_clean":"2024-05-10T10:38:43.543285+0000","last_became_active":"2024-05-10T10:35:47.908349+0000","last_became_peered":"2024-05-10T10:35:47.908349+0000","last_unstale":"2024-05-10T10:38:43.543285+0000","last_undegraded":"2024-05-10T10:38:43.543285+0000","last_fullsized":"2024-05-10T10:38:43.543285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:48.696104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291371+0000","last_change":"2024-05-10T10:35:50.933729+0000","last_active":"2024-05-10T10:38:33.291371+0000","last_peered":"2024-05-10T10:38:33.291371+0000","last_clean":"2024-05-10T10:38:33.291371+0000","last_became_active":"2024-05-10T10:35:50.933109+0000","last_became_peered":"2024-05-10T10:35:50.933109+0000","last_unstale":"2024-05-10T10:38:33.291371+0000","last_undegraded":"2024-05-10T10:38:33.291371+0000","last_fullsized":"2024-05-10T10:38:33.291371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:10:38.305592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567135+0000","last_change":"2024-05-10T10:36:01.991969+0000","last_active":"2024-05-10T10:37:35.567135+0000","last_peered":"2024-05-10T10:37:35.567135+0000","last_clean":"2024-05-10T10:37:35.567135+0000","last_became_active":"2024-05-10T10:36:01.990278+0000","last_became_peered":"2024-05-10T10:36:01.990278+0000","last_unstale":"2024-05-10T10:37:35.567135+0000","last_undegraded":"2024-05-10T10:37:35.567135+0000","last_fullsized":"2024-05-10T10:37:35.567135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:33.070520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719303+0000","last_change":"2024-05-10T10:35:47.911682+0000","last_active":"2024-05-10T10:37:35.719303+0000","last_peered":"2024-05-10T10:37:35.719303+0000","last_clean":"2024-05-10T10:37:35.719303+0000","last_became_active":"2024-05-10T10:35:47.911328+0000","last_became_peered":"2024-05-10T10:35:47.911328+0000","last_unstale":"2024-05-10T10:37:35.719303+0000","last_undegraded":"2024-05-10T10:37:35.719303+0000","last_fullsized":"2024-05-10T10:37:35.719303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.361340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285082+0000","last_change":"2024-05-10T10:35:50.931541+0000","last_active":"2024-05-10T10:38:33.285082+0000","last_peered":"2024-05-10T10:38:33.285082+0000","last_clean":"2024-05-10T10:38:33.285082+0000","last_became_active":"2024-05-10T10:35:50.931244+0000","last_became_peered":"2024-05-10T10:35:50.931244+0000","last_unstale":"2024-05-10T10:38:33.285082+0000","last_undegraded":"2024-05-10T10:38:33.285082+0000","last_fullsized":"2024-05-10T10:38:33.285082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:38.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720689+0000","last_change":"2024-05-10T10:36:01.990390+0000","last_active":"2024-05-10T10:37:35.720689+0000","last_peered":"2024-05-10T10:37:35.720689+0000","last_clean":"2024-05-10T10:37:35.720689+0000","last_became_active":"2024-05-10T10:36:01.989434+0000","last_became_peered":"2024-05-10T10:36:01.989434+0000","last_unstale":"2024-05-10T10:37:35.720689+0000","last_undegraded":"2024-05-10T10:37:35.720689+0000","last_fullsized":"2024-05-10T10:37:35.720689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:53.129212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566392+0000","last_change":"2024-05-10T10:35:47.911590+0000","last_active":"2024-05-10T10:37:35.566392+0000","last_peered":"2024-05-10T10:37:35.566392+0000","last_clean":"2024-05-10T10:37:35.566392+0000","last_became_active":"2024-05-10T10:35:47.910950+0000","last_became_peered":"2024-05-10T10:35:47.910950+0000","last_unstale":"2024-05-10T10:37:35.566392+0000","last_undegraded":"2024-05-10T10:37:35.566392+0000","last_fullsized":"2024-05-10T10:37:35.566392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:18.844774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292200+0000","last_change":"2024-05-10T10:35:50.938174+0000","last_active":"2024-05-10T10:38:33.292200+0000","last_peered":"2024-05-10T10:38:33.292200+0000","last_clean":"2024-05-10T10:38:33.292200+0000","last_became_active":"2024-05-10T10:35:50.937618+0000","last_became_peered":"2024-05-10T10:35:50.937618+0000","last_unstale":"2024-05-10T10:38:33.292200+0000","last_undegraded":"2024-05-10T10:38:33.292200+0000","last_fullsized":"2024-05-10T10:38:33.292200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:26:48.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567141+0000","last_change":"2024-05-10T10:36:01.979226+0000","last_active":"2024-05-10T10:37:35.567141+0000","last_peered":"2024-05-10T10:37:35.567141+0000","last_clean":"2024-05-10T10:37:35.567141+0000","last_became_active":"2024-05-10T10:36:01.978847+0000","last_became_peered":"2024-05-10T10:36:01.978847+0000","last_unstale":"2024-05-10T10:37:35.567141+0000","last_undegraded":"2024-05-10T10:37:35.567141+0000","last_fullsized":"2024-05-10T10:37:35.567141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:10.629813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":317,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:43.543452+0000","last_change":"2024-05-10T10:36:35.150814+0000","last_active":"2024-05-10T10:38:43.543452+0000","last_peered":"2024-05-10T10:38:43.543452+0000","last_clean":"2024-05-10T10:38:43.543452+0000","last_became_active":"2024-05-10T10:36:35.142980+0000","last_became_peered":"2024-05-10T10:36:35.142980+0000","last_unstale":"2024-05-10T10:38:43.543452+0000","last_undegraded":"2024-05-10T10:38:43.543452+0000","last_fullsized":"2024-05-10T10:38:43.543452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:17.907275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67283,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":174,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290064+0000","last_change":"2024-05-10T10:35:50.925584+0000","last_active":"2024-05-10T10:38:33.290064+0000","last_peered":"2024-05-10T10:38:33.290064+0000","last_clean":"2024-05-10T10:38:33.290064+0000","last_became_active":"2024-05-10T10:35:50.925215+0000","last_became_peered":"2024-05-10T10:35:50.925215+0000","last_unstale":"2024-05-10T10:38:33.290064+0000","last_undegraded":"2024-05-10T10:38:33.290064+0000","last_fullsized":"2024-05-10T10:38:33.290064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:03.377110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566636+0000","last_change":"2024-05-10T10:36:01.985090+0000","last_active":"2024-05-10T10:37:35.566636+0000","last_peered":"2024-05-10T10:37:35.566636+0000","last_clean":"2024-05-10T10:37:35.566636+0000","last_became_active":"2024-05-10T10:36:01.984059+0000","last_became_peered":"2024-05-10T10:36:01.984059+0000","last_unstale":"2024-05-10T10:37:35.566636+0000","last_undegraded":"2024-05-10T10:37:35.566636+0000","last_fullsized":"2024-05-10T10:37:35.566636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:41.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571763+0000","last_change":"2024-05-10T10:35:47.914377+0000","last_active":"2024-05-10T10:37:35.571763+0000","last_peered":"2024-05-10T10:37:35.571763+0000","last_clean":"2024-05-10T10:37:35.571763+0000","last_became_active":"2024-05-10T10:35:47.913830+0000","last_became_peered":"2024-05-10T10:35:47.913830+0000","last_unstale":"2024-05-10T10:37:35.571763+0000","last_undegraded":"2024-05-10T10:37:35.571763+0000","last_fullsized":"2024-05-10T10:37:35.571763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:54:18.540952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284127+0000","last_change":"2024-05-10T10:35:50.932721+0000","last_active":"2024-05-10T10:38:33.284127+0000","last_peered":"2024-05-10T10:38:33.284127+0000","last_clean":"2024-05-10T10:38:33.284127+0000","last_became_active":"2024-05-10T10:35:50.932410+0000","last_became_peered":"2024-05-10T10:35:50.932410+0000","last_unstale":"2024-05-10T10:38:33.284127+0000","last_undegraded":"2024-05-10T10:38:33.284127+0000","last_fullsized":"2024-05-10T10:38:33.284127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:42.255534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569383+0000","last_change":"2024-05-10T10:36:35.143581+0000","last_active":"2024-05-10T10:37:35.569383+0000","last_peered":"2024-05-10T10:37:35.569383+0000","last_clean":"2024-05-10T10:37:35.569383+0000","last_became_active":"2024-05-10T10:36:35.142957+0000","last_became_peered":"2024-05-10T10:36:35.142957+0000","last_unstale":"2024-05-10T10:37:35.569383+0000","last_undegraded":"2024-05-10T10:37:35.569383+0000","last_fullsized":"2024-05-10T10:37:35.569383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:14:39.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566100+0000","last_change":"2024-05-10T10:35:47.908874+0000","last_active":"2024-05-10T10:37:35.566100+0000","last_peered":"2024-05-10T10:37:35.566100+0000","last_clean":"2024-05-10T10:37:35.566100+0000","last_became_active":"2024-05-10T10:35:47.908478+0000","last_became_peered":"2024-05-10T10:35:47.908478+0000","last_unstale":"2024-05-10T10:37:35.566100+0000","last_undegraded":"2024-05-10T10:37:35.566100+0000","last_fullsized":"2024-05-10T10:37:35.566100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:13.147762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285463+0000","last_change":"2024-05-10T10:35:50.925168+0000","last_active":"2024-05-10T10:38:33.285463+0000","last_peered":"2024-05-10T10:38:33.285463+0000","last_clean":"2024-05-10T10:38:33.285463+0000","last_became_active":"2024-05-10T10:35:50.924509+0000","last_became_peered":"2024-05-10T10:35:50.924509+0000","last_unstale":"2024-05-10T10:38:33.285463+0000","last_undegraded":"2024-05-10T10:38:33.285463+0000","last_fullsized":"2024-05-10T10:38:33.285463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:08.458195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569666+0000","last_change":"2024-05-10T10:36:35.144575+0000","last_active":"2024-05-10T10:37:35.569666+0000","last_peered":"2024-05-10T10:37:35.569666+0000","last_clean":"2024-05-10T10:37:35.569666+0000","last_became_active":"2024-05-10T10:36:35.144165+0000","last_became_peered":"2024-05-10T10:36:35.144165+0000","last_unstale":"2024-05-10T10:37:35.569666+0000","last_undegraded":"2024-05-10T10:37:35.569666+0000","last_fullsized":"2024-05-10T10:37:35.569666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:45.179163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:43.543277+0000","last_change":"2024-05-10T10:35:47.911208+0000","last_active":"2024-05-10T10:38:43.543277+0000","last_peered":"2024-05-10T10:38:43.543277+0000","last_clean":"2024-05-10T10:38:43.543277+0000","last_became_active":"2024-05-10T10:35:47.910820+0000","last_became_peered":"2024-05-10T10:35:47.910820+0000","last_unstale":"2024-05-10T10:38:43.543277+0000","last_undegraded":"2024-05-10T10:38:43.543277+0000","last_fullsized":"2024-05-10T10:38:43.543277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:04.804203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289463+0000","last_change":"2024-05-10T10:35:50.928195+0000","last_active":"2024-05-10T10:38:33.289463+0000","last_peered":"2024-05-10T10:38:33.289463+0000","last_clean":"2024-05-10T10:38:33.289463+0000","last_became_active":"2024-05-10T10:35:50.927832+0000","last_became_peered":"2024-05-10T10:35:50.927832+0000","last_unstale":"2024-05-10T10:38:33.289463+0000","last_undegraded":"2024-05-10T10:38:33.289463+0000","last_fullsized":"2024-05-10T10:38:33.289463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:03.071525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565054+0000","last_change":"2024-05-10T10:36:01.982483+0000","last_active":"2024-05-10T10:37:35.565054+0000","last_peered":"2024-05-10T10:37:35.565054+0000","last_clean":"2024-05-10T10:37:35.565054+0000","last_became_active":"2024-05-10T10:36:01.982117+0000","last_became_peered":"2024-05-10T10:36:01.982117+0000","last_unstale":"2024-05-10T10:37:35.565054+0000","last_undegraded":"2024-05-10T10:37:35.565054+0000","last_fullsized":"2024-05-10T10:37:35.565054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:08.145035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569474+0000","last_change":"2024-05-10T10:35:47.912929+0000","last_active":"2024-05-10T10:37:35.569474+0000","last_peered":"2024-05-10T10:37:35.569474+0000","last_clean":"2024-05-10T10:37:35.569474+0000","last_became_active":"2024-05-10T10:35:47.912575+0000","last_became_peered":"2024-05-10T10:35:47.912575+0000","last_unstale":"2024-05-10T10:37:35.569474+0000","last_undegraded":"2024-05-10T10:37:35.569474+0000","last_fullsized":"2024-05-10T10:37:35.569474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:03.235579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296070+0000","last_change":"2024-05-10T10:35:50.935967+0000","last_active":"2024-05-10T10:38:33.296070+0000","last_peered":"2024-05-10T10:38:33.296070+0000","last_clean":"2024-05-10T10:38:33.296070+0000","last_became_active":"2024-05-10T10:35:50.935635+0000","last_became_peered":"2024-05-10T10:35:50.935635+0000","last_unstale":"2024-05-10T10:38:33.296070+0000","last_undegraded":"2024-05-10T10:38:33.296070+0000","last_fullsized":"2024-05-10T10:38:33.296070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:02:53.681522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565799+0000","last_change":"2024-05-10T10:36:01.979488+0000","last_active":"2024-05-10T10:37:35.565799+0000","last_peered":"2024-05-10T10:37:35.565799+0000","last_clean":"2024-05-10T10:37:35.565799+0000","last_became_active":"2024-05-10T10:36:01.978839+0000","last_became_peered":"2024-05-10T10:36:01.978839+0000","last_unstale":"2024-05-10T10:37:35.565799+0000","last_undegraded":"2024-05-10T10:37:35.565799+0000","last_fullsized":"2024-05-10T10:37:35.565799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:20.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570423+0000","last_change":"2024-05-10T10:35:47.913235+0000","last_active":"2024-05-10T10:37:35.570423+0000","last_peered":"2024-05-10T10:37:35.570423+0000","last_clean":"2024-05-10T10:37:35.570423+0000","last_became_active":"2024-05-10T10:35:47.912788+0000","last_became_peered":"2024-05-10T10:35:47.912788+0000","last_unstale":"2024-05-10T10:37:35.570423+0000","last_undegraded":"2024-05-10T10:37:35.570423+0000","last_fullsized":"2024-05-10T10:37:35.570423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:56:10.275488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290951+0000","last_change":"2024-05-10T10:35:50.930269+0000","last_active":"2024-05-10T10:38:33.290951+0000","last_peered":"2024-05-10T10:38:33.290951+0000","last_clean":"2024-05-10T10:38:33.290951+0000","last_became_active":"2024-05-10T10:35:50.929918+0000","last_became_peered":"2024-05-10T10:35:50.929918+0000","last_unstale":"2024-05-10T10:38:33.290951+0000","last_undegraded":"2024-05-10T10:38:33.290951+0000","last_fullsized":"2024-05-10T10:38:33.290951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:31.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569677+0000","last_change":"2024-05-10T10:36:02.018506+0000","last_active":"2024-05-10T10:37:35.569677+0000","last_peered":"2024-05-10T10:37:35.569677+0000","last_clean":"2024-05-10T10:37:35.569677+0000","last_became_active":"2024-05-10T10:36:02.018004+0000","last_became_peered":"2024-05-10T10:36:02.018004+0000","last_unstale":"2024-05-10T10:37:35.569677+0000","last_undegraded":"2024-05-10T10:37:35.569677+0000","last_fullsized":"2024-05-10T10:37:35.569677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:37.404970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567545+0000","last_change":"2024-05-10T10:35:47.910654+0000","last_active":"2024-05-10T10:37:35.567545+0000","last_peered":"2024-05-10T10:37:35.567545+0000","last_clean":"2024-05-10T10:37:35.567545+0000","last_became_active":"2024-05-10T10:35:47.910338+0000","last_became_peered":"2024-05-10T10:35:47.910338+0000","last_unstale":"2024-05-10T10:37:35.567545+0000","last_undegraded":"2024-05-10T10:37:35.567545+0000","last_fullsized":"2024-05-10T10:37:35.567545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:07.127574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288361+0000","last_change":"2024-05-10T10:35:50.940950+0000","last_active":"2024-05-10T10:38:33.288361+0000","last_peered":"2024-05-10T10:38:33.288361+0000","last_clean":"2024-05-10T10:38:33.288361+0000","last_became_active":"2024-05-10T10:35:50.940562+0000","last_became_peered":"2024-05-10T10:35:50.940562+0000","last_unstale":"2024-05-10T10:38:33.288361+0000","last_undegraded":"2024-05-10T10:38:33.288361+0000","last_fullsized":"2024-05-10T10:38:33.288361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:24.825900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564778+0000","last_change":"2024-05-10T10:36:01.989277+0000","last_active":"2024-05-10T10:37:35.564778+0000","last_peered":"2024-05-10T10:37:35.564778+0000","last_clean":"2024-05-10T10:37:35.564778+0000","last_became_active":"2024-05-10T10:36:01.988866+0000","last_became_peered":"2024-05-10T10:36:01.988866+0000","last_unstale":"2024-05-10T10:37:35.564778+0000","last_undegraded":"2024-05-10T10:37:35.564778+0000","last_fullsized":"2024-05-10T10:37:35.564778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:08:29.740044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:39.537275+0000","last_change":"2024-05-10T10:35:47.908332+0000","last_active":"2024-05-10T10:38:39.537275+0000","last_peered":"2024-05-10T10:38:39.537275+0000","last_clean":"2024-05-10T10:38:39.537275+0000","last_became_active":"2024-05-10T10:35:47.907933+0000","last_became_peered":"2024-05-10T10:35:47.907933+0000","last_unstale":"2024-05-10T10:38:39.537275+0000","last_undegraded":"2024-05-10T10:38:39.537275+0000","last_fullsized":"2024-05-10T10:38:39.537275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:47:48.702338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288662+0000","last_change":"2024-05-10T10:35:50.934638+0000","last_active":"2024-05-10T10:38:33.288662+0000","last_peered":"2024-05-10T10:38:33.288662+0000","last_clean":"2024-05-10T10:38:33.288662+0000","last_became_active":"2024-05-10T10:35:50.934285+0000","last_became_peered":"2024-05-10T10:35:50.934285+0000","last_unstale":"2024-05-10T10:38:33.288662+0000","last_undegraded":"2024-05-10T10:38:33.288662+0000","last_fullsized":"2024-05-10T10:38:33.288662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:24:31.179005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565003+0000","last_change":"2024-05-10T10:36:02.018468+0000","last_active":"2024-05-10T10:37:35.565003+0000","last_peered":"2024-05-10T10:37:35.565003+0000","last_clean":"2024-05-10T10:37:35.565003+0000","last_became_active":"2024-05-10T10:36:02.018087+0000","last_became_peered":"2024-05-10T10:36:02.018087+0000","last_unstale":"2024-05-10T10:37:35.565003+0000","last_undegraded":"2024-05-10T10:37:35.565003+0000","last_fullsized":"2024-05-10T10:37:35.565003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:55.712012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295331+0000","last_change":"2024-05-10T10:35:50.933660+0000","last_active":"2024-05-10T10:38:33.295331+0000","last_peered":"2024-05-10T10:38:33.295331+0000","last_clean":"2024-05-10T10:38:33.295331+0000","last_became_active":"2024-05-10T10:35:50.933230+0000","last_became_peered":"2024-05-10T10:35:50.933230+0000","last_unstale":"2024-05-10T10:38:33.295331+0000","last_undegraded":"2024-05-10T10:38:33.295331+0000","last_fullsized":"2024-05-10T10:38:33.295331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:34:54.272278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:43.542488+0000","last_change":"2024-05-10T10:35:47.910161+0000","last_active":"2024-05-10T10:38:43.542488+0000","last_peered":"2024-05-10T10:38:43.542488+0000","last_clean":"2024-05-10T10:38:43.542488+0000","last_became_active":"2024-05-10T10:35:47.909875+0000","last_became_peered":"2024-05-10T10:35:47.909875+0000","last_unstale":"2024-05-10T10:38:43.542488+0000","last_undegraded":"2024-05-10T10:38:43.542488+0000","last_fullsized":"2024-05-10T10:38:43.542488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:21.620921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719393+0000","last_change":"2024-05-10T10:36:01.986755+0000","last_active":"2024-05-10T10:37:35.719393+0000","last_peered":"2024-05-10T10:37:35.719393+0000","last_clean":"2024-05-10T10:37:35.719393+0000","last_became_active":"2024-05-10T10:36:01.986340+0000","last_became_peered":"2024-05-10T10:36:01.986340+0000","last_unstale":"2024-05-10T10:37:35.719393+0000","last_undegraded":"2024-05-10T10:37:35.719393+0000","last_fullsized":"2024-05-10T10:37:35.719393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:38.434323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293573+0000","last_change":"2024-05-10T10:36:35.114218+0000","last_active":"2024-05-10T10:38:33.293573+0000","last_peered":"2024-05-10T10:38:33.293573+0000","last_clean":"2024-05-10T10:38:33.293573+0000","last_became_active":"2024-05-10T10:36:35.113876+0000","last_became_peered":"2024-05-10T10:36:35.113876+0000","last_unstale":"2024-05-10T10:38:33.293573+0000","last_undegraded":"2024-05-10T10:38:33.293573+0000","last_fullsized":"2024-05-10T10:38:33.293573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:21.415678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718798+0000","last_change":"2024-05-10T10:35:47.909161+0000","last_active":"2024-05-10T10:37:35.718798+0000","last_peered":"2024-05-10T10:37:35.718798+0000","last_clean":"2024-05-10T10:37:35.718798+0000","last_became_active":"2024-05-10T10:35:47.908546+0000","last_became_peered":"2024-05-10T10:35:47.908546+0000","last_unstale":"2024-05-10T10:37:35.718798+0000","last_undegraded":"2024-05-10T10:37:35.718798+0000","last_fullsized":"2024-05-10T10:37:35.718798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:52.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719177+0000","last_change":"2024-05-10T10:34:20.980948+0000","last_active":"2024-05-10T10:37:35.719177+0000","last_peered":"2024-05-10T10:37:35.719177+0000","last_clean":"2024-05-10T10:37:35.719177+0000","last_became_active":"2024-05-10T10:34:20.975662+0000","last_became_peered":"2024-05-10T10:34:20.975662+0000","last_unstale":"2024-05-10T10:37:35.719177+0000","last_undegraded":"2024-05-10T10:37:35.719177+0000","last_fullsized":"2024-05-10T10:37:35.719177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_clean_scrub_stamp":"2024-05-10T10:30:55.419211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:33.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566301+0000","last_change":"2024-05-10T10:36:02.018578+0000","last_active":"2024-05-10T10:37:35.566301+0000","last_peered":"2024-05-10T10:37:35.566301+0000","last_clean":"2024-05-10T10:37:35.566301+0000","last_became_active":"2024-05-10T10:36:02.018126+0000","last_became_peered":"2024-05-10T10:36:02.018126+0000","last_unstale":"2024-05-10T10:37:35.566301+0000","last_undegraded":"2024-05-10T10:37:35.566301+0000","last_fullsized":"2024-05-10T10:37:35.566301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:18:17.112432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285100+0000","last_change":"2024-05-10T10:35:50.927472+0000","last_active":"2024-05-10T10:38:33.285100+0000","last_peered":"2024-05-10T10:38:33.285100+0000","last_clean":"2024-05-10T10:38:33.285100+0000","last_became_active":"2024-05-10T10:35:50.927030+0000","last_became_peered":"2024-05-10T10:35:50.927030+0000","last_unstale":"2024-05-10T10:38:33.285100+0000","last_undegraded":"2024-05-10T10:38:33.285100+0000","last_fullsized":"2024-05-10T10:38:33.285100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:15:03.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719958+0000","last_change":"2024-05-10T10:35:47.912675+0000","last_active":"2024-05-10T10:37:35.719958+0000","last_peered":"2024-05-10T10:37:35.719958+0000","last_clean":"2024-05-10T10:37:35.719958+0000","last_became_active":"2024-05-10T10:35:47.912297+0000","last_became_peered":"2024-05-10T10:35:47.912297+0000","last_unstale":"2024-05-10T10:37:35.719958+0000","last_undegraded":"2024-05-10T10:37:35.719958+0000","last_fullsized":"2024-05-10T10:37:35.719958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:46.178221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569247+0000","last_change":"2024-05-10T10:36:01.981501+0000","last_active":"2024-05-10T10:37:35.569247+0000","last_peered":"2024-05-10T10:37:35.569247+0000","last_clean":"2024-05-10T10:37:35.569247+0000","last_became_active":"2024-05-10T10:36:01.981029+0000","last_became_peered":"2024-05-10T10:36:01.981029+0000","last_unstale":"2024-05-10T10:37:35.569247+0000","last_undegraded":"2024-05-10T10:37:35.569247+0000","last_fullsized":"2024-05-10T10:37:35.569247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:18.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570023+0000","last_change":"2024-05-10T10:35:47.910701+0000","last_active":"2024-05-10T10:37:35.570023+0000","last_peered":"2024-05-10T10:37:35.570023+0000","last_clean":"2024-05-10T10:37:35.570023+0000","last_became_active":"2024-05-10T10:35:47.910286+0000","last_became_peered":"2024-05-10T10:35:47.910286+0000","last_unstale":"2024-05-10T10:37:35.570023+0000","last_undegraded":"2024-05-10T10:37:35.570023+0000","last_fullsized":"2024-05-10T10:37:35.570023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:07.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286038+0000","last_change":"2024-05-10T10:35:50.926026+0000","last_active":"2024-05-10T10:38:33.286038+0000","last_peered":"2024-05-10T10:38:33.286038+0000","last_clean":"2024-05-10T10:38:33.286038+0000","last_became_active":"2024-05-10T10:35:50.925624+0000","last_became_peered":"2024-05-10T10:35:50.925624+0000","last_unstale":"2024-05-10T10:38:33.286038+0000","last_undegraded":"2024-05-10T10:38:33.286038+0000","last_fullsized":"2024-05-10T10:38:33.286038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:22:02.828681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569074+0000","last_change":"2024-05-10T10:36:02.018403+0000","last_active":"2024-05-10T10:37:35.569074+0000","last_peered":"2024-05-10T10:37:35.569074+0000","last_clean":"2024-05-10T10:37:35.569074+0000","last_became_active":"2024-05-10T10:36:02.017980+0000","last_became_peered":"2024-05-10T10:36:02.017980+0000","last_unstale":"2024-05-10T10:37:35.569074+0000","last_undegraded":"2024-05-10T10:37:35.569074+0000","last_fullsized":"2024-05-10T10:37:35.569074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:07.264730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:57.730122+0000","last_change":"2024-05-10T10:37:35.577657+0000","last_active":"2024-05-10T10:37:57.730122+0000","last_peered":"2024-05-10T10:37:57.730122+0000","last_clean":"2024-05-10T10:37:57.730122+0000","last_became_active":"2024-05-10T10:37:35.570901+0000","last_became_peered":"2024-05-10T10:37:35.570901+0000","last_unstale":"2024-05-10T10:37:57.730122+0000","last_undegraded":"2024-05-10T10:37:57.730122+0000","last_fullsized":"2024-05-10T10:37:57.730122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:30.088160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293071+0000","last_change":"2024-05-10T10:35:50.935817+0000","last_active":"2024-05-10T10:38:33.293071+0000","last_peered":"2024-05-10T10:38:33.293071+0000","last_clean":"2024-05-10T10:38:33.293071+0000","last_became_active":"2024-05-10T10:35:50.935515+0000","last_became_peered":"2024-05-10T10:35:50.935515+0000","last_unstale":"2024-05-10T10:38:33.293071+0000","last_undegraded":"2024-05-10T10:38:33.293071+0000","last_fullsized":"2024-05-10T10:38:33.293071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:49.042193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571516+0000","last_change":"2024-05-10T10:36:01.986002+0000","last_active":"2024-05-10T10:37:35.571516+0000","last_peered":"2024-05-10T10:37:35.571516+0000","last_clean":"2024-05-10T10:37:35.571516+0000","last_became_active":"2024-05-10T10:36:01.985558+0000","last_became_peered":"2024-05-10T10:36:01.985558+0000","last_unstale":"2024-05-10T10:37:35.571516+0000","last_undegraded":"2024-05-10T10:37:35.571516+0000","last_fullsized":"2024-05-10T10:37:35.571516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:27.732316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720203+0000","last_change":"2024-05-10T10:35:47.909254+0000","last_active":"2024-05-10T10:37:35.720203+0000","last_peered":"2024-05-10T10:37:35.720203+0000","last_clean":"2024-05-10T10:37:35.720203+0000","last_became_active":"2024-05-10T10:35:47.908370+0000","last_became_peered":"2024-05-10T10:35:47.908370+0000","last_unstale":"2024-05-10T10:37:35.720203+0000","last_undegraded":"2024-05-10T10:37:35.720203+0000","last_fullsized":"2024-05-10T10:37:35.720203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:44.643592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287742+0000","last_change":"2024-05-10T10:35:50.933001+0000","last_active":"2024-05-10T10:38:33.287742+0000","last_peered":"2024-05-10T10:38:33.287742+0000","last_clean":"2024-05-10T10:38:33.287742+0000","last_became_active":"2024-05-10T10:35:50.932010+0000","last_became_peered":"2024-05-10T10:35:50.932010+0000","last_unstale":"2024-05-10T10:38:33.287742+0000","last_undegraded":"2024-05-10T10:38:33.287742+0000","last_fullsized":"2024-05-10T10:38:33.287742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:01.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720023+0000","last_change":"2024-05-10T10:36:02.018650+0000","last_active":"2024-05-10T10:37:35.720023+0000","last_peered":"2024-05-10T10:37:35.720023+0000","last_clean":"2024-05-10T10:37:35.720023+0000","last_became_active":"2024-05-10T10:36:02.018258+0000","last_became_peered":"2024-05-10T10:36:02.018258+0000","last_unstale":"2024-05-10T10:37:35.720023+0000","last_undegraded":"2024-05-10T10:37:35.720023+0000","last_fullsized":"2024-05-10T10:37:35.720023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:33.590752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565584+0000","last_change":"2024-05-10T10:35:47.910419+0000","last_active":"2024-05-10T10:37:35.565584+0000","last_peered":"2024-05-10T10:37:35.565584+0000","last_clean":"2024-05-10T10:37:35.565584+0000","last_became_active":"2024-05-10T10:35:47.909930+0000","last_became_peered":"2024-05-10T10:35:47.909930+0000","last_unstale":"2024-05-10T10:37:35.565584+0000","last_undegraded":"2024-05-10T10:37:35.565584+0000","last_fullsized":"2024-05-10T10:37:35.565584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:32.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.250953+0000","last_change":"2024-05-10T10:35:50.923207+0000","last_active":"2024-05-10T10:38:33.250953+0000","last_peered":"2024-05-10T10:38:33.250953+0000","last_clean":"2024-05-10T10:38:33.250953+0000","last_became_active":"2024-05-10T10:35:50.922866+0000","last_became_peered":"2024-05-10T10:35:50.922866+0000","last_unstale":"2024-05-10T10:38:33.250953+0000","last_undegraded":"2024-05-10T10:38:33.250953+0000","last_fullsized":"2024-05-10T10:38:33.250953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:48.096498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567305+0000","last_change":"2024-05-10T10:36:01.985399+0000","last_active":"2024-05-10T10:37:35.567305+0000","last_peered":"2024-05-10T10:37:35.567305+0000","last_clean":"2024-05-10T10:37:35.567305+0000","last_became_active":"2024-05-10T10:36:01.985016+0000","last_became_peered":"2024-05-10T10:36:01.985016+0000","last_unstale":"2024-05-10T10:37:35.567305+0000","last_undegraded":"2024-05-10T10:37:35.567305+0000","last_fullsized":"2024-05-10T10:37:35.567305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:12.151693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569591+0000","last_change":"2024-05-10T10:35:47.908594+0000","last_active":"2024-05-10T10:37:35.569591+0000","last_peered":"2024-05-10T10:37:35.569591+0000","last_clean":"2024-05-10T10:37:35.569591+0000","last_became_active":"2024-05-10T10:35:47.908278+0000","last_became_peered":"2024-05-10T10:35:47.908278+0000","last_unstale":"2024-05-10T10:37:35.569591+0000","last_undegraded":"2024-05-10T10:37:35.569591+0000","last_fullsized":"2024-05-10T10:37:35.569591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:24.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294756+0000","last_change":"2024-05-10T10:35:50.933089+0000","last_active":"2024-05-10T10:38:33.294756+0000","last_peered":"2024-05-10T10:38:33.294756+0000","last_clean":"2024-05-10T10:38:33.294756+0000","last_became_active":"2024-05-10T10:35:50.932739+0000","last_became_peered":"2024-05-10T10:35:50.932739+0000","last_unstale":"2024-05-10T10:38:33.294756+0000","last_undegraded":"2024-05-10T10:38:33.294756+0000","last_fullsized":"2024-05-10T10:38:33.294756+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:15.302832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720006+0000","last_change":"2024-05-10T10:36:01.988561+0000","last_active":"2024-05-10T10:37:35.720006+0000","last_peered":"2024-05-10T10:37:35.720006+0000","last_clean":"2024-05-10T10:37:35.720006+0000","last_became_active":"2024-05-10T10:36:01.988120+0000","last_became_peered":"2024-05-10T10:36:01.988120+0000","last_unstale":"2024-05-10T10:37:35.720006+0000","last_undegraded":"2024-05-10T10:37:35.720006+0000","last_fullsized":"2024-05-10T10:37:35.720006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:35:37.120469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.581302+0000","last_change":"2024-05-10T10:37:35.580875+0000","last_active":"2024-05-10T10:37:35.581302+0000","last_peered":"2024-05-10T10:37:35.581302+0000","last_clean":"2024-05-10T10:37:35.581302+0000","last_became_active":"2024-05-10T10:37:35.572187+0000","last_became_peered":"2024-05-10T10:37:35.572187+0000","last_unstale":"2024-05-10T10:37:35.581302+0000","last_undegraded":"2024-05-10T10:37:35.581302+0000","last_fullsized":"2024-05-10T10:37:35.581302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:06.330404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290565+0000","last_change":"2024-05-10T10:35:50.938466+0000","last_active":"2024-05-10T10:38:33.290565+0000","last_peered":"2024-05-10T10:38:33.290565+0000","last_clean":"2024-05-10T10:38:33.290565+0000","last_became_active":"2024-05-10T10:35:50.938167+0000","last_became_peered":"2024-05-10T10:35:50.938167+0000","last_unstale":"2024-05-10T10:38:33.290565+0000","last_undegraded":"2024-05-10T10:38:33.290565+0000","last_fullsized":"2024-05-10T10:38:33.290565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:18.014708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719597+0000","last_change":"2024-05-10T10:36:02.018661+0000","last_active":"2024-05-10T10:37:35.719597+0000","last_peered":"2024-05-10T10:37:35.719597+0000","last_clean":"2024-05-10T10:37:35.719597+0000","last_became_active":"2024-05-10T10:36:02.018196+0000","last_became_peered":"2024-05-10T10:36:02.018196+0000","last_unstale":"2024-05-10T10:37:35.719597+0000","last_undegraded":"2024-05-10T10:37:35.719597+0000","last_fullsized":"2024-05-10T10:37:35.719597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:11.605012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570017+0000","last_change":"2024-05-10T10:35:47.917009+0000","last_active":"2024-05-10T10:37:35.570017+0000","last_peered":"2024-05-10T10:37:35.570017+0000","last_clean":"2024-05-10T10:37:35.570017+0000","last_became_active":"2024-05-10T10:35:47.916536+0000","last_became_peered":"2024-05-10T10:35:47.916536+0000","last_unstale":"2024-05-10T10:37:35.570017+0000","last_undegraded":"2024-05-10T10:37:35.570017+0000","last_fullsized":"2024-05-10T10:37:35.570017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:17.783027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287009+0000","last_change":"2024-05-10T10:35:50.926173+0000","last_active":"2024-05-10T10:38:33.287009+0000","last_peered":"2024-05-10T10:38:33.287009+0000","last_clean":"2024-05-10T10:38:33.287009+0000","last_became_active":"2024-05-10T10:35:50.925846+0000","last_became_peered":"2024-05-10T10:35:50.925846+0000","last_unstale":"2024-05-10T10:38:33.287009+0000","last_undegraded":"2024-05-10T10:38:33.287009+0000","last_fullsized":"2024-05-10T10:38:33.287009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:29:02.587667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565679+0000","last_change":"2024-05-10T10:36:01.981312+0000","last_active":"2024-05-10T10:37:35.565679+0000","last_peered":"2024-05-10T10:37:35.565679+0000","last_clean":"2024-05-10T10:37:35.565679+0000","last_became_active":"2024-05-10T10:36:01.980861+0000","last_became_peered":"2024-05-10T10:36:01.980861+0000","last_unstale":"2024-05-10T10:37:35.565679+0000","last_undegraded":"2024-05-10T10:37:35.565679+0000","last_fullsized":"2024-05-10T10:37:35.565679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:29.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:38.536724+0000","last_change":"2024-05-10T10:35:47.911120+0000","last_active":"2024-05-10T10:38:38.536724+0000","last_peered":"2024-05-10T10:38:38.536724+0000","last_clean":"2024-05-10T10:38:38.536724+0000","last_became_active":"2024-05-10T10:35:47.909945+0000","last_became_peered":"2024-05-10T10:35:47.909945+0000","last_unstale":"2024-05-10T10:38:38.536724+0000","last_undegraded":"2024-05-10T10:38:38.536724+0000","last_fullsized":"2024-05-10T10:38:38.536724+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:40.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287370+0000","last_change":"2024-05-10T10:35:50.933855+0000","last_active":"2024-05-10T10:38:33.287370+0000","last_peered":"2024-05-10T10:38:33.287370+0000","last_clean":"2024-05-10T10:38:33.287370+0000","last_became_active":"2024-05-10T10:35:50.933502+0000","last_became_peered":"2024-05-10T10:35:50.933502+0000","last_unstale":"2024-05-10T10:38:33.287370+0000","last_undegraded":"2024-05-10T10:38:33.287370+0000","last_fullsized":"2024-05-10T10:38:33.287370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:36.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718378+0000","last_change":"2024-05-10T10:36:01.985133+0000","last_active":"2024-05-10T10:37:35.718378+0000","last_peered":"2024-05-10T10:37:35.718378+0000","last_clean":"2024-05-10T10:37:35.718378+0000","last_became_active":"2024-05-10T10:36:01.984758+0000","last_became_peered":"2024-05-10T10:36:01.984758+0000","last_unstale":"2024-05-10T10:37:35.718378+0000","last_undegraded":"2024-05-10T10:37:35.718378+0000","last_fullsized":"2024-05-10T10:37:35.718378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:52.733063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566564+0000","last_change":"2024-05-10T10:35:47.908645+0000","last_active":"2024-05-10T10:37:35.566564+0000","last_peered":"2024-05-10T10:37:35.566564+0000","last_clean":"2024-05-10T10:37:35.566564+0000","last_became_active":"2024-05-10T10:35:47.908123+0000","last_became_peered":"2024-05-10T10:35:47.908123+0000","last_unstale":"2024-05-10T10:37:35.566564+0000","last_undegraded":"2024-05-10T10:37:35.566564+0000","last_fullsized":"2024-05-10T10:37:35.566564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:46.181724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289675+0000","last_change":"2024-05-10T10:35:50.929215+0000","last_active":"2024-05-10T10:38:33.289675+0000","last_peered":"2024-05-10T10:38:33.289675+0000","last_clean":"2024-05-10T10:38:33.289675+0000","last_became_active":"2024-05-10T10:35:50.928907+0000","last_became_peered":"2024-05-10T10:35:50.928907+0000","last_unstale":"2024-05-10T10:38:33.289675+0000","last_undegraded":"2024-05-10T10:38:33.289675+0000","last_fullsized":"2024-05-10T10:38:33.289675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:57:17.340687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565667+0000","last_change":"2024-05-10T10:36:01.988548+0000","last_active":"2024-05-10T10:37:35.565667+0000","last_peered":"2024-05-10T10:37:35.565667+0000","last_clean":"2024-05-10T10:37:35.565667+0000","last_became_active":"2024-05-10T10:36:01.987221+0000","last_became_peered":"2024-05-10T10:36:01.987221+0000","last_unstale":"2024-05-10T10:37:35.565667+0000","last_undegraded":"2024-05-10T10:37:35.565667+0000","last_fullsized":"2024-05-10T10:37:35.565667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:55:22.434536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565063+0000","last_change":"2024-05-10T10:35:47.914022+0000","last_active":"2024-05-10T10:37:35.565063+0000","last_peered":"2024-05-10T10:37:35.565063+0000","last_clean":"2024-05-10T10:37:35.565063+0000","last_became_active":"2024-05-10T10:35:47.913711+0000","last_became_peered":"2024-05-10T10:35:47.913711+0000","last_unstale":"2024-05-10T10:37:35.565063+0000","last_undegraded":"2024-05-10T10:37:35.565063+0000","last_fullsized":"2024-05-10T10:37:35.565063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:13.291307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284901+0000","last_change":"2024-05-10T10:35:50.927452+0000","last_active":"2024-05-10T10:38:33.284901+0000","last_peered":"2024-05-10T10:38:33.284901+0000","last_clean":"2024-05-10T10:38:33.284901+0000","last_became_active":"2024-05-10T10:35:50.927102+0000","last_became_peered":"2024-05-10T10:35:50.927102+0000","last_unstale":"2024-05-10T10:38:33.284901+0000","last_undegraded":"2024-05-10T10:38:33.284901+0000","last_fullsized":"2024-05-10T10:38:33.284901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:07.704196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293319+0000","last_change":"2024-05-10T10:35:50.933072+0000","last_active":"2024-05-10T10:38:33.293319+0000","last_peered":"2024-05-10T10:38:33.293319+0000","last_clean":"2024-05-10T10:38:33.293319+0000","last_became_active":"2024-05-10T10:35:50.932724+0000","last_became_peered":"2024-05-10T10:35:50.932724+0000","last_unstale":"2024-05-10T10:38:33.293319+0000","last_undegraded":"2024-05-10T10:38:33.293319+0000","last_fullsized":"2024-05-10T10:38:33.293319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:08.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287846+0000","last_change":"2024-05-10T10:35:50.931635+0000","last_active":"2024-05-10T10:38:33.287846+0000","last_peered":"2024-05-10T10:38:33.287846+0000","last_clean":"2024-05-10T10:38:33.287846+0000","last_became_active":"2024-05-10T10:35:50.930158+0000","last_became_peered":"2024-05-10T10:35:50.930158+0000","last_unstale":"2024-05-10T10:38:33.287846+0000","last_undegraded":"2024-05-10T10:38:33.287846+0000","last_fullsized":"2024-05-10T10:38:33.287846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:22.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285276+0000","last_change":"2024-05-10T10:35:50.936947+0000","last_active":"2024-05-10T10:38:33.285276+0000","last_peered":"2024-05-10T10:38:33.285276+0000","last_clean":"2024-05-10T10:38:33.285276+0000","last_became_active":"2024-05-10T10:35:50.936156+0000","last_became_peered":"2024-05-10T10:35:50.936156+0000","last_unstale":"2024-05-10T10:38:33.285276+0000","last_undegraded":"2024-05-10T10:38:33.285276+0000","last_fullsized":"2024-05-10T10:38:33.285276+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:21.167787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294617+0000","last_change":"2024-05-10T10:35:50.935384+0000","last_active":"2024-05-10T10:38:33.294617+0000","last_peered":"2024-05-10T10:38:33.294617+0000","last_clean":"2024-05-10T10:38:33.294617+0000","last_became_active":"2024-05-10T10:35:50.932555+0000","last_became_peered":"2024-05-10T10:35:50.932555+0000","last_unstale":"2024-05-10T10:38:33.294617+0000","last_undegraded":"2024-05-10T10:38:33.294617+0000","last_fullsized":"2024-05-10T10:38:33.294617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:09:11.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295227+0000","last_change":"2024-05-10T10:35:50.929322+0000","last_active":"2024-05-10T10:38:33.295227+0000","last_peered":"2024-05-10T10:38:33.295227+0000","last_clean":"2024-05-10T10:38:33.295227+0000","last_became_active":"2024-05-10T10:35:50.928366+0000","last_became_peered":"2024-05-10T10:35:50.928366+0000","last_unstale":"2024-05-10T10:38:33.295227+0000","last_undegraded":"2024-05-10T10:38:33.295227+0000","last_fullsized":"2024-05-10T10:38:33.295227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:32:28.337962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292799+0000","last_change":"2024-05-10T10:36:35.110922+0000","last_active":"2024-05-10T10:38:33.292799+0000","last_peered":"2024-05-10T10:38:33.292799+0000","last_clean":"2024-05-10T10:38:33.292799+0000","last_became_active":"2024-05-10T10:36:35.110525+0000","last_became_peered":"2024-05-10T10:36:35.110525+0000","last_unstale":"2024-05-10T10:38:33.292799+0000","last_undegraded":"2024-05-10T10:38:33.292799+0000","last_fullsized":"2024-05-10T10:38:33.292799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:47.825168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287609+0000","last_change":"2024-05-10T10:35:50.927869+0000","last_active":"2024-05-10T10:38:33.287609+0000","last_peered":"2024-05-10T10:38:33.287609+0000","last_clean":"2024-05-10T10:38:33.287609+0000","last_became_active":"2024-05-10T10:35:50.927552+0000","last_became_peered":"2024-05-10T10:35:50.927552+0000","last_unstale":"2024-05-10T10:38:33.287609+0000","last_undegraded":"2024-05-10T10:38:33.287609+0000","last_fullsized":"2024-05-10T10:38:33.287609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:36.962316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285786+0000","last_change":"2024-05-10T10:35:50.933099+0000","last_active":"2024-05-10T10:38:33.285786+0000","last_peered":"2024-05-10T10:38:33.285786+0000","last_clean":"2024-05-10T10:38:33.285786+0000","last_became_active":"2024-05-10T10:35:50.932765+0000","last_became_peered":"2024-05-10T10:35:50.932765+0000","last_unstale":"2024-05-10T10:38:33.285786+0000","last_undegraded":"2024-05-10T10:38:33.285786+0000","last_fullsized":"2024-05-10T10:38:33.285786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:19.095217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294630+0000","last_change":"2024-05-10T10:35:50.937704+0000","last_active":"2024-05-10T10:38:33.294630+0000","last_peered":"2024-05-10T10:38:33.294630+0000","last_clean":"2024-05-10T10:38:33.294630+0000","last_became_active":"2024-05-10T10:35:50.937236+0000","last_became_peered":"2024-05-10T10:35:50.937236+0000","last_unstale":"2024-05-10T10:38:33.294630+0000","last_undegraded":"2024-05-10T10:38:33.294630+0000","last_fullsized":"2024-05-10T10:38:33.294630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:46.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292751+0000","last_change":"2024-05-10T10:35:50.935181+0000","last_active":"2024-05-10T10:38:33.292751+0000","last_peered":"2024-05-10T10:38:33.292751+0000","last_clean":"2024-05-10T10:38:33.292751+0000","last_became_active":"2024-05-10T10:35:50.933446+0000","last_became_peered":"2024-05-10T10:35:50.933446+0000","last_unstale":"2024-05-10T10:38:33.292751+0000","last_undegraded":"2024-05-10T10:38:33.292751+0000","last_fullsized":"2024-05-10T10:38:33.292751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:03.972410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285940+0000","last_change":"2024-05-10T10:35:50.931797+0000","last_active":"2024-05-10T10:38:33.285940+0000","last_peered":"2024-05-10T10:38:33.285940+0000","last_clean":"2024-05-10T10:38:33.285940+0000","last_became_active":"2024-05-10T10:35:50.931494+0000","last_became_peered":"2024-05-10T10:35:50.931494+0000","last_unstale":"2024-05-10T10:38:33.285940+0000","last_undegraded":"2024-05-10T10:38:33.285940+0000","last_fullsized":"2024-05-10T10:38:33.285940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:30.797462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296149+0000","last_change":"2024-05-10T10:35:50.931688+0000","last_active":"2024-05-10T10:38:33.296149+0000","last_peered":"2024-05-10T10:38:33.296149+0000","last_clean":"2024-05-10T10:38:33.296149+0000","last_became_active":"2024-05-10T10:35:50.931362+0000","last_became_peered":"2024-05-10T10:35:50.931362+0000","last_unstale":"2024-05-10T10:38:33.296149+0000","last_undegraded":"2024-05-10T10:38:33.296149+0000","last_fullsized":"2024-05-10T10:38:33.296149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:24.263752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292719+0000","last_change":"2024-05-10T10:35:50.930905+0000","last_active":"2024-05-10T10:38:33.292719+0000","last_peered":"2024-05-10T10:38:33.292719+0000","last_clean":"2024-05-10T10:38:33.292719+0000","last_became_active":"2024-05-10T10:35:50.930448+0000","last_became_peered":"2024-05-10T10:35:50.930448+0000","last_unstale":"2024-05-10T10:38:33.292719+0000","last_undegraded":"2024-05-10T10:38:33.292719+0000","last_fullsized":"2024-05-10T10:38:33.292719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:52:24.468600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74237,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1067,"num_read_kb":618,"num_write":255,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3701,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":222711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":520990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1175621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037821,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430280,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1117936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":516424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":584737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":517880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":517674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":516132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986263,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":588313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822739,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1112986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:38:47.443 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T10:38:47.443 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T10:38:47.443 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-10T10:38:47.443 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-10T10:38:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:48.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:47 smithi112 ceph-mon[30100]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:38:48.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:47 smithi148 ceph-mon[33568]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:38:48.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:47 smithi195 ceph-mon[33773]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:38:49.164 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:49 smithi112 ceph-mon[30100]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:38:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:49 smithi148 ceph-mon[33568]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:38:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:49 smithi195 ceph-mon[33773]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.557+0000 7fe959f01700 1 -- 172.21.15.112:0/4098832007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9541083c0 msgr2=0x7fe95406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.557+0000 7fe959f01700 1 --2- 172.21.15.112:0/4098832007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9541083c0 0x7fe95406bbf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe944009990 tx=0x7fe944009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.557+0000 7fe959f01700 1 -- 172.21.15.112:0/4098832007 shutdown_connections 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.557+0000 7fe959f01700 1 --2- 172.21.15.112:0/4098832007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe954105ed0 0x7fe954106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.557+0000 7fe959f01700 1 --2- 172.21.15.112:0/4098832007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9541083c0 0x7fe95406bbf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.557+0000 7fe959f01700 1 --2- 172.21.15.112:0/4098832007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe95406c130 0x7fe95406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.557+0000 7fe959f01700 1 -- 172.21.15.112:0/4098832007 >> 172.21.15.112:0/4098832007 conn(0x7fe954079340 msgr2=0x7fe954079750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.558+0000 7fe959f01700 1 -- 172.21.15.112:0/4098832007 shutdown_connections 2024-05-10T10:38:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.558+0000 7fe959f01700 1 -- 172.21.15.112:0/4098832007 wait complete. 2024-05-10T10:38:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.558+0000 7fe959f01700 1 Processor -- start 2024-05-10T10:38:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe959f01700 1 -- start start 2024-05-10T10:38:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe959f01700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95406c130 0x7fe9541a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe959f01700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe954105ed0 0x7fe9541a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe959f01700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9541083c0 0x7fe9541a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe959f01700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe954119c00 con 0x7fe95406c130 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe959f01700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe954119a80 con 0x7fe9541083c0 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe959f01700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe954119d80 con 0x7fe954105ed0 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95406c130 0x7fe9541a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95406c130 0x7fe9541a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46796/0 (socket says 172.21.15.112:46796) 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 -- 172.21.15.112:0/12313083 learned_addr learned my addr 172.21.15.112:0/12313083 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe952ffd700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe954105ed0 0x7fe9541a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe953fff700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9541083c0 0x7fe9541a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 -- 172.21.15.112:0/12313083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe954105ed0 msgr2=0x7fe9541a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe954105ed0 0x7fe9541a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 -- 172.21.15.112:0/12313083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9541083c0 msgr2=0x7fe9541a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9541083c0 0x7fe9541a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 -- 172.21.15.112:0/12313083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9541a52d0 con 0x7fe95406c130 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe953fff700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9541083c0 0x7fe9541a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.559+0000 7fe9537fe700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95406c130 0x7fe9541a02c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe9440036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.560+0000 7fe950ff9700 1 -- 172.21.15.112:0/12313083 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944027070 con 0x7fe95406c130 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.560+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9541a5560 con 0x7fe95406c130 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.560+0000 7fe950ff9700 1 -- 172.21.15.112:0/12313083 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe94400fc80 con 0x7fe95406c130 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.560+0000 7fe950ff9700 1 -- 172.21.15.112:0/12313083 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944022d60 con 0x7fe95406c130 2024-05-10T10:38:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.560+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541ace40 con 0x7fe95406c130 2024-05-10T10:38:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.561+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9541029b0 con 0x7fe95406c130 2024-05-10T10:38:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.562+0000 7fe950ff9700 1 -- 172.21.15.112:0/12313083 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe94402a070 con 0x7fe95406c130 2024-05-10T10:38:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.563+0000 7fe950ff9700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe93c0b2900 0x7fe93c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.563+0000 7fe952ffd700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe93c0b2900 0x7fe93c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.563+0000 7fe950ff9700 1 -- 172.21.15.112:0/12313083 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe9440e3470 con 0x7fe95406c130 2024-05-10T10:38:49.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.563+0000 7fe952ffd700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe93c0b2900 0x7fe93c0b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe948006fd0 tx=0x7fe948008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.567+0000 7fe950ff9700 1 -- 172.21.15.112:0/12313083 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe944013070 con 0x7fe95406c130 2024-05-10T10:38:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.815+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe95406a190 con 0x7fe95406c130 2024-05-10T10:38:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.815+0000 7fe950ff9700 1 -- 172.21.15.112:0/12313083 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe9440aae90 con 0x7fe95406c130 2024-05-10T10:38:49.817 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:38:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T10:38:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe93c0b2900 msgr2=0x7fe93c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe93c0b2900 0x7fe93c0b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe948006fd0 tx=0x7fe948008040 comp rx=0 tx=0).stop 2024-05-10T10:38:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95406c130 msgr2=0x7fe9541a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95406c130 0x7fe9541a02c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe9440036f0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 shutdown_connections 2024-05-10T10:38:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9541083c0 0x7fe9541a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95406c130 0x7fe9541a02c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe93c0b2900 0x7fe93c0b4dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 --2- 172.21.15.112:0/12313083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe954105ed0 0x7fe9541a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 >> 172.21.15.112:0/12313083 conn(0x7fe954079340 msgr2=0x7fe954103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:49.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 shutdown_connections 2024-05-10T10:38:49.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:49.818+0000 7fe959f01700 1 -- 172.21.15.112:0/12313083 wait complete. 2024-05-10T10:38:50.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:50 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/12313083' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T10:38:50.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:50 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/12313083' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T10:38:50.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:50 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/12313083' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T10:38:50.721 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-10T10:38:50.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:38:50.969 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:51 smithi148 ceph-mon[33568]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-10T10:38:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:51 smithi195 ceph-mon[33773]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-10T10:38:51.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:51 smithi112 ceph-mon[30100]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 -- 172.21.15.112:0/2924106351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 msgr2=0x7fe4b4109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 --2- 172.21.15.112:0/2924106351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4109900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00b0a0 tx=0x7fe4ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 -- 172.21.15.112:0/2924106351 shutdown_connections 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 --2- 172.21.15.112:0/2924106351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4102450 0x7fe4b4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 --2- 172.21.15.112:0/2924106351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4109900 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 --2- 172.21.15.112:0/2924106351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4b4101a90 0x7fe4b4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 -- 172.21.15.112:0/2924106351 >> 172.21.15.112:0/2924106351 conn(0x7fe4b4078a10 msgr2=0x7fe4b40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 -- 172.21.15.112:0/2924106351 shutdown_connections 2024-05-10T10:38:52.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.759+0000 7fe4bb490700 1 -- 172.21.15.112:0/2924106351 wait complete. 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 Processor -- start 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 -- start start 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4b4101a90 0x7fe4b419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4102450 0x7fe4b419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b4113500 con 0x7fe4b4109520 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b4113380 con 0x7fe4b4102450 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4bb490700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b4113680 con 0x7fe4b4101a90 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4b9a2d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4b9a2d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46814/0 (socket says 172.21.15.112:46814) 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4b9a2d700 1 -- 172.21.15.112:0/886124890 learned_addr learned my addr 172.21.15.112:0/886124890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.760+0000 7fe4b922c700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4b4101a90 0x7fe4b419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b8a2b700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4102450 0x7fe4b419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b9a2d700 1 -- 172.21.15.112:0/886124890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4b4101a90 msgr2=0x7fe4b419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b9a2d700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4b4101a90 0x7fe4b419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b9a2d700 1 -- 172.21.15.112:0/886124890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4102450 msgr2=0x7fe4b419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b9a2d700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4102450 0x7fe4b419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b9a2d700 1 -- 172.21.15.112:0/886124890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b4199fe0 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b8a2b700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4102450 0x7fe4b419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b9a2d700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4199880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400d980 tx=0x7fe4a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b27fc700 1 -- 172.21.15.112:0/886124890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4017070 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b419a270 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b27fc700 1 -- 172.21.15.112:0/886124890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4a4005020 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4b27fc700 1 -- 172.21.15.112:0/886124890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4009990 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.761+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b41a6590 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.762+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b410ba00 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.763+0000 7fe4b27fc700 1 -- 172.21.15.112:0/886124890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe4a4004a30 con 0x7fe4b4109520 2024-05-10T10:38:52.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.764+0000 7fe4b27fc700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe49c0b2880 0x7fe49c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.764+0000 7fe4b922c700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe49c0b2880 0x7fe49c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.764+0000 7fe4b27fc700 1 -- 172.21.15.112:0/886124890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe4a40e2720 con 0x7fe4b4109520 2024-05-10T10:38:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.765+0000 7fe4b922c700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe49c0b2880 0x7fe49c0b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009e60 tx=0x7fe4ac00b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:52.768+0000 7fe4b27fc700 1 -- 172.21.15.112:0/886124890 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe4a4020020 con 0x7fe4b4109520 2024-05-10T10:38:53.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.021+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4b419adb0 con 0x7fe4b4109520 2024-05-10T10:38:53.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.022+0000 7fe4b27fc700 1 -- 172.21.15.112:0/886124890 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7fe4a40aa0b0 con 0x7fe4b4109520 2024-05-10T10:38:53.023 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:38:53.023 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:37:03.364184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24428,24445,24446,24451,24469]},"id":1}]} 2024-05-10T10:38:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe49c0b2880 msgr2=0x7fe49c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe49c0b2880 0x7fe49c0b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009e60 tx=0x7fe4ac00b7b0 comp rx=0 tx=0).stop 2024-05-10T10:38:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 msgr2=0x7fe4b4199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4199880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400d980 tx=0x7fe4a400dc90 comp rx=0 tx=0).stop 2024-05-10T10:38:53.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 shutdown_connections 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b4109520 0x7fe4b4199880 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe49c0b2880 0x7fe49c0b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4b4101a90 0x7fe4b419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 --2- 172.21.15.112:0/886124890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4102450 0x7fe4b419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 >> 172.21.15.112:0/886124890 conn(0x7fe4b4078a10 msgr2=0x7fe4b4100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 shutdown_connections 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:53.024+0000 7fe4bb490700 1 -- 172.21.15.112:0/886124890 wait complete. 2024-05-10T10:38:53.026 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-10T10:38:53.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:53 smithi148 ceph-mon[33568]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:53 smithi195 ceph-mon[33773]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:53.324 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:53 smithi112 ceph-mon[30100]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:38:53.877 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:38:54.126 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:54.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:54 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/886124890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:38:54.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:54 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/886124890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:38:54.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:54 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/886124890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:38:55.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:55 smithi112 ceph-mon[30100]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:55 smithi148 ceph-mon[33568]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:55 smithi195 ceph-mon[33773]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.910+0000 7fafe532b700 1 -- 172.21.15.112:0/806422328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe0109580 msgr2=0x7fafe0109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.910+0000 7fafe532b700 1 --2- 172.21.15.112:0/806422328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe0109580 0x7fafe0109960 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009990 tx=0x7fafd0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.910+0000 7fafe532b700 1 -- 172.21.15.112:0/806422328 shutdown_connections 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 --2- 172.21.15.112:0/806422328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafe006c5d0 0x7fafe0110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 --2- 172.21.15.112:0/806422328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe0109580 0x7fafe0109960 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 --2- 172.21.15.112:0/806422328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafe006bc10 0x7fafe006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 -- 172.21.15.112:0/806422328 >> 172.21.15.112:0/806422328 conn(0x7fafe00783f0 msgr2=0x7fafe0078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 -- 172.21.15.112:0/806422328 shutdown_connections 2024-05-10T10:38:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 -- 172.21.15.112:0/806422328 wait complete. 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 Processor -- start 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.911+0000 7fafe532b700 1 -- start start 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafe006bc10 0x7fafe019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe006c5d0 0x7fafe019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafe0109580 0x7fafe0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0113510 con 0x7fafe006c5d0 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0113390 con 0x7fafe006bc10 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0113690 con 0x7fafe0109580 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe006c5d0 0x7fafe019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe006c5d0 0x7fafe019fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51122/0 (socket says 172.21.15.112:51122) 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 -- 172.21.15.112:0/993207152 learned_addr learned my addr 172.21.15.112:0/993207152 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:55.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafdf7fe700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafe0109580 0x7fafe0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafdeffd700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafe006bc10 0x7fafe019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 -- 172.21.15.112:0/993207152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafe0109580 msgr2=0x7fafe0199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafe0109580 0x7fafe0199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 -- 172.21.15.112:0/993207152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafe006bc10 msgr2=0x7fafe019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafe006bc10 0x7fafe019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 -- 172.21.15.112:0/993207152 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe0199fd0 con 0x7fafe006c5d0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafdeffd700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafe006bc10 0x7fafe019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafde7fc700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe006c5d0 0x7fafe019fbf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fafd400c920 tx=0x7fafd400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafc7fff700 1 -- 172.21.15.112:0/993207152 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4007b00 con 0x7fafe006c5d0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe019a2c0 con 0x7fafe006c5d0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafc7fff700 1 -- 172.21.15.112:0/993207152 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafd4004cc0 con 0x7fafe006c5d0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafc7fff700 1 -- 172.21.15.112:0/993207152 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd40054a0 con 0x7fafe006c5d0 2024-05-10T10:38:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.912+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a6580 con 0x7fafe006c5d0 2024-05-10T10:38:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.913+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe010ba30 con 0x7fafe006c5d0 2024-05-10T10:38:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.915+0000 7fafc7fff700 1 -- 172.21.15.112:0/993207152 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fafd4004750 con 0x7fafe006c5d0 2024-05-10T10:38:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.915+0000 7fafc7fff700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fafc80b2880 0x7fafc80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.915+0000 7fafc7fff700 1 -- 172.21.15.112:0/993207152 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fafd40e25d0 con 0x7fafe006c5d0 2024-05-10T10:38:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.915+0000 7fafdeffd700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fafc80b2880 0x7fafc80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.916+0000 7fafdeffd700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fafc80b2880 0x7fafc80b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafd0003760 tx=0x7fafd001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:55.920+0000 7fafc7fff700 1 -- 172.21.15.112:0/993207152 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fafd4014030 con 0x7fafe006c5d0 2024-05-10T10:38:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.168+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafe006a190 con 0x7fafe006c5d0 2024-05-10T10:38:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.168+0000 7fafc7fff700 1 -- 172.21.15.112:0/993207152 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7fafd40a9f60 con 0x7fafe006c5d0 2024-05-10T10:38:56.171 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:38:56.171 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:37:03.364184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24428,24445,24446,24451,24469]},"id":1}]} 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fafc80b2880 msgr2=0x7fafc80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fafc80b2880 0x7fafc80b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafd0003760 tx=0x7fafd001a040 comp rx=0 tx=0).stop 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe006c5d0 msgr2=0x7fafe019fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe006c5d0 0x7fafe019fbf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fafd400c920 tx=0x7fafd400cce0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 shutdown_connections 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fafc80b2880 0x7fafc80b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafe0109580 0x7fafe0199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafe006bc10 0x7fafe019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 --2- 172.21.15.112:0/993207152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafe006c5d0 0x7fafe019fbf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 >> 172.21.15.112:0/993207152 conn(0x7fafe00783f0 msgr2=0x7fafe0101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 shutdown_connections 2024-05-10T10:38:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:56.170+0000 7fafe532b700 1 -- 172.21.15.112:0/993207152 wait complete. 2024-05-10T10:38:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-10T10:38:56.895 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:38:57.138 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:38:57.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:57 smithi112 ceph-mon[30100]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:38:57.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:57 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/993207152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:38:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:57 smithi148 ceph-mon[33568]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:38:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:57 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/993207152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:38:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:57 smithi195 ceph-mon[33773]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:38:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:57 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/993207152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:38:58.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.887+0000 7f58f51b8700 1 -- 172.21.15.112:0/661832173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0101a90 msgr2=0x7f58f0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.887+0000 7f58f51b8700 1 --2- 172.21.15.112:0/661832173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0101a90 0x7f58f0101f10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009990 tx=0x7f58e4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.887+0000 7f58f51b8700 1 -- 172.21.15.112:0/661832173 shutdown_connections 2024-05-10T10:38:58.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.887+0000 7f58f51b8700 1 --2- 172.21.15.112:0/661832173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f0102450 0x7f58f0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.887+0000 7f58f51b8700 1 --2- 172.21.15.112:0/661832173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0109520 0x7f58f0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.887+0000 7f58f51b8700 1 --2- 172.21.15.112:0/661832173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0101a90 0x7f58f0101f10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.887+0000 7f58f51b8700 1 -- 172.21.15.112:0/661832173 >> 172.21.15.112:0/661832173 conn(0x7f58f0078a10 msgr2=0x7f58f00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:58.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.888+0000 7f58f51b8700 1 -- 172.21.15.112:0/661832173 shutdown_connections 2024-05-10T10:38:58.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.888+0000 7f58f51b8700 1 -- 172.21.15.112:0/661832173 wait complete. 2024-05-10T10:38:58.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.888+0000 7f58f51b8700 1 Processor -- start 2024-05-10T10:38:58.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58f51b8700 1 -- start start 2024-05-10T10:38:58.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58f51b8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f0101a90 0x7f58f0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58f51b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0102450 0x7f58f01031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58f51b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0109520 0x7f58f01036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58f51b8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0113560 con 0x7f58f0102450 2024-05-10T10:38:58.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58f51b8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01133e0 con 0x7f58f0109520 2024-05-10T10:38:58.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58f51b8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01136e0 con 0x7f58f0101a90 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0102450 0x7f58f01031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0102450 0x7f58f01031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51154/0 (socket says 172.21.15.112:51154) 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 -- 172.21.15.112:0/1112946444 learned_addr learned my addr 172.21.15.112:0/1112946444 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58eed9d700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f0101a90 0x7f58f0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ef59e700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0109520 0x7f58f01036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 -- 172.21.15.112:0/1112946444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f0101a90 msgr2=0x7f58f0102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f0101a90 0x7f58f0102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 -- 172.21.15.112:0/1112946444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0109520 msgr2=0x7f58f01036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0109520 0x7f58f01036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ee59c700 1 -- 172.21.15.112:0/1112946444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01a7f80 con 0x7f58f0102450 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.889+0000 7f58ef59e700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0109520 0x7f58f01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.890+0000 7f58ee59c700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0102450 0x7f58f01031b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58e40057f0 tx=0x7f58e4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.890+0000 7f58d7fff700 1 -- 172.21.15.112:0/1112946444 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4027070 con 0x7f58f0102450 2024-05-10T10:38:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.890+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01a8210 con 0x7f58f0102450 2024-05-10T10:38:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.890+0000 7f58d7fff700 1 -- 172.21.15.112:0/1112946444 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58e400fc80 con 0x7f58f0102450 2024-05-10T10:38:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.890+0000 7f58d7fff700 1 -- 172.21.15.112:0/1112946444 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4022c70 con 0x7f58f0102450 2024-05-10T10:38:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.890+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a86f0 con 0x7f58f0102450 2024-05-10T10:38:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.891+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f010ba00 con 0x7f58f0102450 2024-05-10T10:38:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.892+0000 7f58d7fff700 1 -- 172.21.15.112:0/1112946444 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f58e402a070 con 0x7f58f0102450 2024-05-10T10:38:58.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.893+0000 7f58d7fff700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f58d80b2880 0x7f58d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.893+0000 7f58eed9d700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f58d80b2880 0x7f58d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.893+0000 7f58d7fff700 1 -- 172.21.15.112:0/1112946444 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f58e40e3840 con 0x7f58f0102450 2024-05-10T10:38:58.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.894+0000 7f58eed9d700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f58d80b2880 0x7f58d80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009a10 tx=0x7f58e0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:58.897+0000 7f58d7fff700 1 -- 172.21.15.112:0/1112946444 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f58e4031080 con 0x7f58f0102450 2024-05-10T10:38:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.102+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58f01041a0 con 0x7f58f0102450 2024-05-10T10:38:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.104+0000 7f58d7fff700 1 -- 172.21.15.112:0/1112946444 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f58e4013070 con 0x7f58f0102450 2024-05-10T10:38:59.106 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:38:59.106 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.106+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f58d80b2880 msgr2=0x7f58d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.106+0000 7f58f51b8700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f58d80b2880 0x7f58d80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009a10 tx=0x7f58e0009190 comp rx=0 tx=0).stop 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.106+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0102450 msgr2=0x7f58f01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.106+0000 7f58f51b8700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0102450 0x7f58f01031b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58e40057f0 tx=0x7f58e4009ee0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.106+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 shutdown_connections 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.106+0000 7f58f51b8700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0109520 0x7f58f01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.106+0000 7f58f51b8700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f58d80b2880 0x7f58d80b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.107+0000 7f58f51b8700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f0101a90 0x7f58f0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.107+0000 7f58f51b8700 1 --2- 172.21.15.112:0/1112946444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0102450 0x7f58f01031b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.107+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 >> 172.21.15.112:0/1112946444 conn(0x7f58f0078a10 msgr2=0x7f58f0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.107+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 shutdown_connections 2024-05-10T10:38:59.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:38:59.107+0000 7f58f51b8700 1 -- 172.21.15.112:0/1112946444 wait complete. 2024-05-10T10:38:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:38:59 smithi148 ceph-mon[33568]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:59.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:38:59 smithi195 ceph-mon[33773]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:59.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:38:59 smithi112 ceph-mon[30100]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:38:59.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:39:00.080 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:00.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:00 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1112946444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:39:00.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:00 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1112946444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:39:00.328 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:00 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1112946444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:39:01.241 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:01 smithi112 ceph-mon[30100]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:01.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:01 smithi148 ceph-mon[33568]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:01 smithi195 ceph-mon[33773]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:01.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 -- 172.21.15.112:0/2075249784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 msgr2=0x7f0e1406b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/2075249784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1406b600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009990 tx=0x7f0e08009ca0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 -- 172.21.15.112:0/2075249784 shutdown_connections 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/2075249784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1406bb40 0x7f0e14110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/2075249784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14109510 0x7f0e141098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/2075249784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1406b600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 -- 172.21.15.112:0/2075249784 >> 172.21.15.112:0/2075249784 conn(0x7f0e14079350 msgr2=0x7f0e14079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 -- 172.21.15.112:0/2075249784 shutdown_connections 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.771+0000 7f0e18fa6700 1 -- 172.21.15.112:0/2075249784 wait complete. 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 Processor -- start 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 -- start start 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1406bb40 0x7f0e1419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14109510 0x7f0e14199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141134d0 con 0x7f0e1406b180 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14113350 con 0x7f0e14109510 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e18fa6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14113650 con 0x7f0e1406bb40 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51160/0 (socket says 172.21.15.112:51160) 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 -- 172.21.15.112:0/3022963654 learned_addr learned my addr 172.21.15.112:0/3022963654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e12d9d700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14109510 0x7f0e14199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e11d9b700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1406bb40 0x7f0e1419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 -- 172.21.15.112:0/3022963654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1406bb40 msgr2=0x7f0e1419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1406bb40 0x7f0e1419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 -- 172.21.15.112:0/3022963654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14109510 msgr2=0x7f0e14199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14109510 0x7f0e14199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 -- 172.21.15.112:0/3022963654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1419a030 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.772+0000 7f0e1259c700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1419f6d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c920 tx=0x7f0e0400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.773+0000 7f0e037fe700 1 -- 172.21.15.112:0/3022963654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04007b00 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.773+0000 7f0e037fe700 1 -- 172.21.15.112:0/3022963654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e04004cc0 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.773+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1419a320 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.773+0000 7f0e037fe700 1 -- 172.21.15.112:0/3022963654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04005440 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.773+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e141a6530 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.774+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e14104bd0 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.775+0000 7f0e037fe700 1 -- 172.21.15.112:0/3022963654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0e04004750 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.776+0000 7f0e037fe700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0dfc0b2900 0x7f0dfc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.776+0000 7f0e11d9b700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0dfc0b2900 0x7f0dfc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.776+0000 7f0e037fe700 1 -- 172.21.15.112:0/3022963654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0e040e2430 con 0x7f0e1406b180 2024-05-10T10:39:01.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.777+0000 7f0e11d9b700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0dfc0b2900 0x7f0dfc0b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005850 tx=0x7f0e0801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:01.779+0000 7f0e037fe700 1 -- 172.21.15.112:0/3022963654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0e04014030 con 0x7f0e1406b180 2024-05-10T10:39:02.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.034+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e1419ad80 con 0x7f0e1406b180 2024-05-10T10:39:02.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.035+0000 7f0e037fe700 1 -- 172.21.15.112:0/3022963654 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7f0e040a9e40 con 0x7f0e1406b180 2024-05-10T10:39:02.037 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:02.037 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:37:03.364184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24428,24445,24446,24451,24469]},"id":1}]} 2024-05-10T10:39:02.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0dfc0b2900 msgr2=0x7f0dfc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0dfc0b2900 0x7f0dfc0b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005850 tx=0x7f0e0801a040 comp rx=0 tx=0).stop 2024-05-10T10:39:02.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 msgr2=0x7f0e1419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1419f6d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c920 tx=0x7f0e0400cc30 comp rx=0 tx=0).stop 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 shutdown_connections 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14109510 0x7f0e14199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e1406b180 0x7f0e1419f6d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0dfc0b2900 0x7f0dfc0b4dc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 --2- 172.21.15.112:0/3022963654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e1406bb40 0x7f0e1419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 >> 172.21.15.112:0/3022963654 conn(0x7f0e14079350 msgr2=0x7f0e14101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 shutdown_connections 2024-05-10T10:39:02.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:02.038+0000 7f0e18fa6700 1 -- 172.21.15.112:0/3022963654 wait complete. 2024-05-10T10:39:02.041 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-10T10:39:02.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T10:35:52.903233+0000', 'modified': '2024-05-10T10:37:03.364184+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24451, 'mds_2': 24446, 'mds_3': 24445, 'mds_4': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14592': {'gid': 14592, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6839/2952588053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2952588053}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2952588053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6841/1213307484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 1213307484}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 1213307484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/3726517324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3726517324}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3726517324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/2643629917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 2643629917}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2643629917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6833/38979975', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6832', 'nonce': 38979975}, {'type': 'v1', 'addr': '172.21.15.195:6833', 'nonce': 38979975}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.148:6837/1474359623', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1474359623}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1474359623}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.195:6835/3491127541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 3491127541}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 3491127541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.148:6839/1008412635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 1008412635}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 1008412635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.195:6837/3223375652', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 3223375652}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 3223375652}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.195:6839/2238321568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 2238321568}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 2238321568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24428, 24445, 24446, 24451, 24469]} 2024-05-10T10:39:02.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-10T10:39:02.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-10T10:39:02.967 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:03 smithi148 ceph-mon[33568]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:03 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3022963654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:03.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:03 smithi195 ceph-mon[33773]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:03.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:03 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3022963654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:03.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:03 smithi112 ceph-mon[30100]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:03.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:03 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3022963654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 -- 172.21.15.112:0/3087777014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 msgr2=0x7f1b7c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 --2- 172.21.15.112:0/3087777014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c102a70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0099c0 tx=0x7f1b6c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 -- 172.21.15.112:0/3087777014 shutdown_connections 2024-05-10T10:39:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 --2- 172.21.15.112:0/3087777014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c1076e0 0x7f1b7c10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 --2- 172.21.15.112:0/3087777014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c102a70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 --2- 172.21.15.112:0/3087777014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b7c103040 0x7f1b7c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 -- 172.21.15.112:0/3087777014 >> 172.21.15.112:0/3087777014 conn(0x7f1b7c0783f0 msgr2=0x7f1b7c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 -- 172.21.15.112:0/3087777014 shutdown_connections 2024-05-10T10:39:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.606+0000 7f1b81335700 1 -- 172.21.15.112:0/3087777014 wait complete. 2024-05-10T10:39:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.607+0000 7f1b81335700 1 Processor -- start 2024-05-10T10:39:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.607+0000 7f1b81335700 1 -- start start 2024-05-10T10:39:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.607+0000 7f1b81335700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.607+0000 7f1b81335700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b7c103040 0x7f1b7c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b81335700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c1076e0 0x7f1b7c1a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b81335700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1124a0 con 0x7f1b7c102690 2024-05-10T10:39:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b81335700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c112320 con 0x7f1b7c1076e0 2024-05-10T10:39:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b81335700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c112620 con 0x7f1b7c103040 2024-05-10T10:39:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51172/0 (socket says 172.21.15.112:51172) 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 -- 172.21.15.112:0/778480032 learned_addr learned my addr 172.21.15.112:0/778480032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c1076e0 0x7f1b7c1a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b7c103040 0x7f1b7c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 -- 172.21.15.112:0/778480032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b7c103040 msgr2=0x7f1b7c19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b7c103040 0x7f1b7c19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 -- 172.21.15.112:0/778480032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c1076e0 msgr2=0x7f1b7c1a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c1076e0 0x7f1b7c1a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 -- 172.21.15.112:0/778480032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c1a5fb0 con 0x7f1b7c102690 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b7affd700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c19a9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b73fff700 1 -- 172.21.15.112:0/778480032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00f8a0 con 0x7f1b7c102690 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b73fff700 1 -- 172.21.15.112:0/778480032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c004830 con 0x7f1b7c102690 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.608+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c1a6180 con 0x7f1b7c102690 2024-05-10T10:39:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.609+0000 7f1b73fff700 1 -- 172.21.15.112:0/778480032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c005410 con 0x7f1b7c102690 2024-05-10T10:39:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.609+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1a6520 con 0x7f1b7c102690 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.611+0000 7f1b73fff700 1 -- 172.21.15.112:0/778480032 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1b6c029060 con 0x7f1b7c102690 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.612+0000 7f1b73fff700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1b680b2880 0x7f1b680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.612+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1b680b2880 0x7f1b680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.612+0000 7f1b73fff700 1 -- 172.21.15.112:0/778480032 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1b6c0e3070 con 0x7f1b7c102690 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.612+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1b680b2880 0x7f1b680b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1b64006fd0 tx=0x7f1b64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.612+0000 7f1b8bbc8b80 1 Processor -- start 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.612+0000 7f1b8bbc8b80 1 -- start start 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761637920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761631bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x5597616325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55976157f8f0 con 0x559761632160 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5597614e4d50 con 0x559761637e60 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55976157b8b0 con 0x559761637540 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x5597616325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x5597616325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51182/0 (socket says 172.21.15.112:51182) 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2162178789 learned_addr learned my addr 172.21.15.112:0/2162178789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7affd700 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761637920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761631bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2162178789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 msgr2=0x559761637920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761637920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2162178789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 msgr2=0x559761631bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761631bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2162178789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597614cc880 con 0x559761632160 2024-05-10T10:39:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7affd700 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761637920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.613+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x5597616325c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1b740098f0 tx=0x7f1b74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd43263e121842b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.614+0000 7f1b717fa700 1 -- 172.21.15.112:0/2162178789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74004bb0 con 0x559761632160 2024-05-10T10:39:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.614+0000 7f1b717fa700 1 -- 172.21.15.112:0/2162178789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b74004d50 con 0x559761632160 2024-05-10T10:39:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.614+0000 7f1b717fa700 1 -- 172.21.15.112:0/2162178789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74007d90 con 0x559761632160 2024-05-10T10:39:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.614+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2162178789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 msgr2=0x5597616325c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.614+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x5597616325c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1b740098f0 tx=0x7f1b74009c00 comp rx=0 tx=0).stop 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2162178789 shutdown_connections 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x5597616325c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761637920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2162178789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761631bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2162178789 >> 172.21.15.112:0/2162178789 conn(0x559761559b20 msgr2=0x559761628250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2162178789 shutdown_connections 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2162178789 wait complete. 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- start start 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 Processor -- start 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x55976163daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637540 0x55976163ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637e60 0x55976163b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55976163e890 con 0x559761632160 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559761538e80 con 0x559761637540 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55976163ea10 con 0x559761637e60 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b8bbc8b80 10 client.0 _ll_register_callbacks cb 0x559761513560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x55976163daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x55976163daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51198/0 (socket says 172.21.15.112:51198) 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.615+0000 7f1b7affd700 1 -- 172.21.15.112:0/512810086 learned_addr learned my addr 172.21.15.112:0/512810086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637540 0x55976163ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637e60 0x55976163b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7affd700 1 -- 172.21.15.112:0/512810086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637e60 msgr2=0x55976163b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7affd700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637e60 0x55976163b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7affd700 1 -- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637540 msgr2=0x55976163ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7affd700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637540 0x55976163ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7affd700 1 -- 172.21.15.112:0/512810086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559761536710 con 0x559761632160 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637540 0x55976163ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637e60 0x55976163b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b7affd700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x55976163daa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0aa090 tx=0x7f1b6c004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304cd84348e1c2db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b627fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b627fc700 1 -- 172.21.15.112:0/512810086 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0c4060 con 0x559761632160 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b627fc700 1 -- 172.21.15.112:0/512810086 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1b6c0aa800 con 0x559761632160 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b74004d50 con 0x559761632160 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b627fc700 1 -- 172.21.15.112:0/512810086 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b6c0aa390 con 0x559761632160 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b8bbc8b80 10 client.15054 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559761649100 con 0x559761632160 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.616+0000 7f1b627fc700 1 -- 172.21.15.112:0/512810086 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b74004d50 con 0x559761632160 2024-05-10T10:39:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b627fc700 1 -- 172.21.15.112:0/512810086 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6c00fa40 con 0x559761632160 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b8bbc8b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b8bbc8b80 10 client.15054 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b8bbc8b80 20 client.15054 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55976164b730 0x55976164cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b8bbc8b80 4 client.15054 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55976158d2d0 con 0x55976164b730 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55976164b730 0x55976164cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b627fc700 10 client.15054 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.617+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55976164b730 0x55976164cba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ca5cd68400e21f server_cookie=f5e48e1208c81590 in_seq=0 out_seq=0 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b627fc700 1 -- 172.21.15.112:0/512810086 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55976158d2d0 con 0x55976164b730 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b627fc700 10 client.15054 handle_command_reply: tid=0 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 client.15054 shutdown 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 msgr2=0x55976163daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x55976163daa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0aa090 tx=0x7f1b6c004f10 comp rx=0 tx=0).stop 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 shutdown_connections 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55976164b730 0x55976164cba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b627fc700 0 client.15054 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637e60 0x55976163b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761632160 0x55976163daa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/512810086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637540 0x55976163ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 >> 172.21.15.112:0/512810086 conn(0x559761559b20 msgr2=0x559761622ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 shutdown_connections 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.618+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/512810086 wait complete. 2024-05-10T10:39:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.619+0000 7f1b8bbc8b80 20 client.15054 trim_cache size 0 max 16384 2024-05-10T10:39:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 Processor -- start 2024-05-10T10:39:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 -- start start 2024-05-10T10:39:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761632160 0x559761622bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637540 0x559761638240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761638780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55976164aee0 con 0x559761637540 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55976164b5b0 con 0x559761637e60 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55976164bc80 con 0x559761632160 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b7a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637540 0x559761638240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b7a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637540 0x559761638240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51200/0 (socket says 172.21.15.112:51200) 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.639+0000 7f1b7a7fc700 1 -- 172.21.15.112:0/3604375617 learned_addr learned my addr 172.21.15.112:0/3604375617 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7affd700 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761632160 0x559761622bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761638780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7a7fc700 1 -- 172.21.15.112:0/3604375617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761632160 msgr2=0x559761622bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761632160 0x559761622bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7a7fc700 1 -- 172.21.15.112:0/3604375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 msgr2=0x559761638780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761638780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7a7fc700 1 -- 172.21.15.112:0/3604375617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55976164b3d0 con 0x559761637540 2024-05-10T10:39:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761638780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637540 0x559761638240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c102120 tx=0x7f1b64013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fac941057c1f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b70ff9700 1 -- 172.21.15.112:0/3604375617 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400dea0 con 0x559761637540 2024-05-10T10:39:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b70ff9700 1 -- 172.21.15.112:0/3604375617 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b640265e0 con 0x559761637540 2024-05-10T10:39:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.640+0000 7f1b70ff9700 1 -- 172.21.15.112:0/3604375617 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640146d0 con 0x559761637540 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/3604375617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637540 msgr2=0x559761638240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637540 0x559761638240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c102120 tx=0x7f1b64013000 comp rx=0 tx=0).stop 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/3604375617 shutdown_connections 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761637e60 0x559761638780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761632160 0x559761622bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/3604375617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637540 0x559761638240 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/3604375617 >> 172.21.15.112:0/3604375617 conn(0x559761559b20 msgr2=0x559761639730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/3604375617 shutdown_connections 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/3604375617 wait complete. 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 -- start start 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.641+0000 7f1b8bbc8b80 1 Processor -- start 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761632160 0x559761621fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761622520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637e60 0x559761622a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55976164ca20 con 0x559761637e60 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55976164c350 con 0x559761632160 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55976164d0f0 con 0x559761637540 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 10 client.0 _ll_register_callbacks cb 0x559761513560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637e60 0x559761622a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637e60 0x559761622a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51208/0 (socket says 172.21.15.112:51208) 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2375051262 learned_addr learned my addr 172.21.15.112:0/2375051262 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7a7fc700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761622520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7affd700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761632160 0x559761621fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2375051262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 msgr2=0x559761622520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761622520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761632160 msgr2=0x559761621fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761632160 0x559761621fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 -- 172.21.15.112:0/2375051262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597616542d0 con 0x559761637e60 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7affd700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761632160 0x559761621fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b7b7fe700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637e60 0x559761622a60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b74018000 tx=0x7f1b74009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b10415a4425a814 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b627fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b627fc700 1 -- 172.21.15.112:0/2375051262 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74022070 con 0x559761637e60 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b640265e0 con 0x559761637e60 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.642+0000 7f1b627fc700 1 -- 172.21.15.112:0/2375051262 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1b74004ca0 con 0x559761637e60 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b627fc700 1 -- 172.21.15.112:0/2375051262 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b7402e050 con 0x559761637e60 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 10 client.15060 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5597616b6dd0 con 0x559761637e60 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b627fc700 1 -- 172.21.15.112:0/2375051262 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b640265e0 con 0x559761637e60 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b627fc700 1 -- 172.21.15.112:0/2375051262 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b74015610 con 0x559761637e60 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 10 client.15060 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 20 client.15060 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5597616bb630 0x5597616ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 4 client.15060 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "status"}) -- 0x559761587370 con 0x5597616bb630 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.643+0000 7f1b7affd700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5597616bb630 0x5597616ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b627fc700 10 client.15060 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b7affd700 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5597616bb630 0x5597616ba340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2676dfd5f2e7e09d server_cookie=9accff48a5e6906f in_seq=0 out_seq=0 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b627fc700 1 -- 172.21.15.112:0/2375051262 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x559761587370 con 0x5597616bb630 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b627fc700 10 client.15060 handle_command_reply: tid=0 2024-05-10T10:39:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 client.15060 shutdown 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637e60 msgr2=0x559761622a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637e60 0x559761622a60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b74018000 tx=0x7f1b74009690 comp rx=0 tx=0).stop 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 shutdown_connections 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559761637e60 0x559761622a60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559761632160 0x559761621fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5597616bb630 0x5597616ba340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 --2- 172.21.15.112:0/2375051262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559761637540 0x559761622520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 >> 172.21.15.112:0/2375051262 conn(0x559761559b20 msgr2=0x55976162c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b627fc700 0 client.15060 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 shutdown_connections 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 1 -- 172.21.15.112:0/2375051262 wait complete. 2024-05-10T10:39:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.644+0000 7f1b8bbc8b80 20 client.15060 trim_cache size 0 max 16384 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 4, 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24428, 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 124.252358, 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 222.901237568 2024-05-10T10:39:04.652 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1b680b2880 msgr2=0x7f1b680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1b680b2880 0x7f1b680b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1b64006fd0 tx=0x7f1b64008040 comp rx=0 tx=0).stop 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 msgr2=0x7f1b7c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c19a9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c0047b0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 shutdown_connections 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c1076e0 0x7f1b7c1a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b7c102690 0x7f1b7c19a9f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1b680b2880 0x7f1b680b4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 --2- 172.21.15.112:0/778480032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b7c103040 0x7f1b7c19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:04.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 >> 172.21.15.112:0/778480032 conn(0x7f1b7c0783f0 msgr2=0x7f1b7c1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:04.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 shutdown_connections 2024-05-10T10:39:04.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:04.645+0000 7f1b81335700 1 -- 172.21.15.112:0/778480032 wait complete. 2024-05-10T10:39:05.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:05 smithi148 ceph-mon[33568]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:05 smithi195 ceph-mon[33773]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:05.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:05 smithi112 ceph-mon[30100]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:05.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.f status 2024-05-10T10:39:05.670 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:07.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:07 smithi112 ceph-mon[30100]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:07.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:07 smithi195 ceph-mon[33773]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 -- 172.21.15.112:0/1050039139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 msgr2=0x7f1e20102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/1050039139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e20102820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009990 tx=0x7f1e10009ca0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 -- 172.21.15.112:0/1050039139 shutdown_connections 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/1050039139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e20102d60 0x7f1e2010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/1050039139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e20108410 0x7f1e201087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/1050039139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e20102820 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 -- 172.21.15.112:0/1050039139 >> 172.21.15.112:0/1050039139 conn(0x7f1e20079350 msgr2=0x7f1e20079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 -- 172.21.15.112:0/1050039139 shutdown_connections 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 -- 172.21.15.112:0/1050039139 wait complete. 2024-05-10T10:39:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 Processor -- start 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.390+0000 7f1e27bd5700 1 -- start start 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e2019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e20102d60 0x7f1e2019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e20108410 0x7f1e201a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20112410 con 0x7f1e201023a0 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20112290 con 0x7f1e20108410 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20112590 con 0x7f1e20102d60 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e2019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e2019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53660/0 (socket says 172.21.15.112:53660) 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 -- 172.21.15.112:0/2001477069 learned_addr learned my addr 172.21.15.112:0/2001477069 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25170700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e20102d60 0x7f1e2019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 -- 172.21.15.112:0/2001477069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e20102d60 msgr2=0x7f1e2019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e20102d60 0x7f1e2019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e26172700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e20108410 0x7f1e201a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 -- 172.21.15.112:0/2001477069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e20108410 msgr2=0x7f1e201a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e20108410 0x7f1e201a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 -- 172.21.15.112:0/2001477069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e201a5f10 con 0x7f1e201023a0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e25971700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e2019a950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00d980 tx=0x7f1e1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e16ffd700 1 -- 172.21.15.112:0/2001477069 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c016070 con 0x7f1e201023a0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e201a61a0 con 0x7f1e201023a0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e16ffd700 1 -- 172.21.15.112:0/2001477069 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1c004830 con 0x7f1e201023a0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e16ffd700 1 -- 172.21.15.112:0/2001477069 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c005440 con 0x7f1e201023a0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.391+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a6700 con 0x7f1e201023a0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.394+0000 7f1e16ffd700 1 -- 172.21.15.112:0/2001477069 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1e1c0055e0 con 0x7f1e201023a0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e16ffd700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e25170700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e16ffd700 1 -- 172.21.15.112:0/2001477069 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1e1c0e2320 con 0x7f1e201023a0 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 Processor -- start 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e25170700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e2006bd30 tx=0x7f1e10005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 -- start start 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713b160 0x5592c713b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c71357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c7135d00 0x5592c7136140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592c70878f0 con 0x5592c713ba80 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5592c6fecd50 con 0x5592c713b160 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.395+0000 7f1e32468b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592c70838b0 con 0x5592c7135d00 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c71357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c71357c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53670/0 (socket says 172.21.15.112:53670) 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 -- 172.21.15.112:0/742668729 learned_addr learned my addr 172.21.15.112:0/742668729 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25971700 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713b160 0x5592c713b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e26172700 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c7135d00 0x5592c7136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 -- 172.21.15.112:0/742668729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c7135d00 msgr2=0x5592c7136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c7135d00 0x5592c7136140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 -- 172.21.15.112:0/742668729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713b160 msgr2=0x5592c713b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713b160 0x5592c713b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 -- 172.21.15.112:0/742668729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c6fd4880 con 0x5592c713ba80 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e25170700 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c71357c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e10015000 tx=0x7f1e1001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d590504e94d2cdbe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e0bfff700 1 -- 172.21.15.112:0/742668729 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000b6b0 con 0x5592c713ba80 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e0bfff700 1 -- 172.21.15.112:0/742668729 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1000bce0 con 0x5592c713ba80 2024-05-10T10:39:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.396+0000 7f1e0bfff700 1 -- 172.21.15.112:0/742668729 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10025d90 con 0x5592c713ba80 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 -- 172.21.15.112:0/742668729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 msgr2=0x5592c71357c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c71357c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e10015000 tx=0x7f1e1001f000 comp rx=0 tx=0).stop 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 -- 172.21.15.112:0/742668729 shutdown_connections 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c7135d00 0x5592c7136140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713b160 0x5592c713b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 --2- 172.21.15.112:0/742668729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c71357c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 -- 172.21.15.112:0/742668729 >> 172.21.15.112:0/742668729 conn(0x5592c7061b20 msgr2=0x5592c7130250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 -- 172.21.15.112:0/742668729 shutdown_connections 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.397+0000 7f1e32468b80 1 -- 172.21.15.112:0/742668729 wait complete. 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 -- start start 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 Processor -- start 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c7135d00 0x5592c71416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c713ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c713efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592c71423a0 con 0x5592c713ba80 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5592c7040e80 con 0x5592c7135d00 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592c7142520 con 0x5592c713b160 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e32468b80 10 client.0 _ll_register_callbacks cb 0x5592c701b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.398+0000 7f1e26172700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c713efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c713efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53682/0 (socket says 172.21.15.112:53682) 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 -- 172.21.15.112:0/3707821408 learned_addr learned my addr 172.21.15.112:0/3707821408 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e25971700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c7135d00 0x5592c71416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e25170700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c713ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 -- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 msgr2=0x5592c713ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c713ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 -- 172.21.15.112:0/3707821408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c7135d00 msgr2=0x5592c71416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c7135d00 0x5592c71416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 -- 172.21.15.112:0/3707821408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c703e710 con 0x5592c713ba80 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e25971700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c7135d00 0x5592c71416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e26172700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c713efd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e180079e0 tx=0x7f1e18007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b066b05589f7fef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e097fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e097fa700 1 -- 172.21.15.112:0/3707821408 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1801a070 con 0x5592c713ba80 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e097fa700 1 -- 172.21.15.112:0/3707821408 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1e18012ac0 con 0x5592c713ba80 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e1000bce0 con 0x5592c713ba80 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e097fa700 1 -- 172.21.15.112:0/3707821408 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e18026050 con 0x5592c713ba80 2024-05-10T10:39:07.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e32468b80 10 client.15072 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592c714cd10 con 0x5592c713ba80 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.399+0000 7f1e097fa700 1 -- 172.21.15.112:0/3707821408 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1000bce0 con 0x5592c713ba80 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e097fa700 1 -- 172.21.15.112:0/3707821408 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e180153c0 con 0x5592c713ba80 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e32468b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e32468b80 10 client.15072 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e32468b80 20 client.15072 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e32468b80 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c714f340 0x5592c71507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e32468b80 4 client.15072 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c70952d0 con 0x5592c714f340 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.400+0000 7f1e25971700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c714f340 0x5592c71507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.401+0000 7f1e25971700 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c714f340 0x5592c71507b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fbe3410b19fd8456 server_cookie=65c48d8d7f420409 in_seq=0 out_seq=0 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.401+0000 7f1e097fa700 10 client.15072 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.401+0000 7f1e097fa700 1 -- 172.21.15.112:0/3707821408 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592c70952d0 con 0x5592c714f340 2024-05-10T10:39:07.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.401+0000 7f1e097fa700 10 client.15072 handle_command_reply: tid=0 2024-05-10T10:39:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.401+0000 7f1e32468b80 1 client.15072 shutdown 2024-05-10T10:39:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.401+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 msgr2=0x5592c713efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.401+0000 7f1e32468b80 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c713efd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e180079e0 tx=0x7f1e18007cf0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 shutdown_connections 2024-05-10T10:39:07.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c714f340 0x5592c71507b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c713ba80 0x5592c713efd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c7135d00 0x5592c71416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e097fa700 0 client.15072 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 --2- 172.21.15.112:0/3707821408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c713ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 >> 172.21.15.112:0/3707821408 conn(0x5592c7061b20 msgr2=0x5592c712aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 shutdown_connections 2024-05-10T10:39:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 1 -- 172.21.15.112:0/3707821408 wait complete. 2024-05-10T10:39:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.402+0000 7f1e32468b80 20 client.15072 trim_cache size 0 max 16384 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 Processor -- start 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 -- start start 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c71357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c713f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c713c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592c714eaf0 con 0x5592c7135d00 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5592c714f1c0 con 0x5592c713ba80 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e32468b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592c714f890 con 0x5592c713b160 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c71357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c71357c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53698/0 (socket says 172.21.15.112:53698) 2024-05-10T10:39:07.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 -- 172.21.15.112:0/2790444372 learned_addr learned my addr 172.21.15.112:0/2790444372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 -- 172.21.15.112:0/2790444372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 msgr2=0x5592c713f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 --2- 172.21.15.112:0/2790444372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c713f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 -- 172.21.15.112:0/2790444372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 msgr2=0x5592c713c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 --2- 172.21.15.112:0/2790444372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c713c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 -- 172.21.15.112:0/2790444372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c713d000 con 0x5592c7135d00 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.423+0000 7f1e25971700 1 --2- 172.21.15.112:0/2790444372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c71357c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0dfe00 tx=0x7f1e1c0df960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88082ae7807cbf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e0b7fe700 1 -- 172.21.15.112:0/2790444372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0b2410 con 0x5592c7135d00 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e0b7fe700 1 -- 172.21.15.112:0/2790444372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1c0c2020 con 0x5592c7135d00 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e0b7fe700 1 -- 172.21.15.112:0/2790444372 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0049e0 con 0x5592c7135d00 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 -- 172.21.15.112:0/2790444372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 msgr2=0x5592c71357c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2790444372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c71357c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0dfe00 tx=0x7f1e1c0df960 comp rx=0 tx=0).stop 2024-05-10T10:39:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 -- 172.21.15.112:0/2790444372 shutdown_connections 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2790444372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c713c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2790444372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c71357c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2790444372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c713f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 -- 172.21.15.112:0/2790444372 >> 172.21.15.112:0/2790444372 conn(0x5592c7061b20 msgr2=0x5592c713be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 -- 172.21.15.112:0/2790444372 shutdown_connections 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.424+0000 7f1e32468b80 1 -- 172.21.15.112:0/2790444372 wait complete. 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 -- start start 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 Processor -- start 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c715a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c7129c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c712a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592c7150630 con 0x5592c7135d00 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5592c714ff60 con 0x5592c713ba80 2024-05-10T10:39:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592c7150d00 con 0x5592c713b160 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e32468b80 10 client.0 _ll_register_callbacks cb 0x5592c701b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e25971700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c715a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e25971700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c715a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53712/0 (socket says 172.21.15.112:53712) 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.425+0000 7f1e25971700 1 -- 172.21.15.112:0/2708334473 learned_addr learned my addr 172.21.15.112:0/2708334473 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e26172700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c712a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e25971700 1 -- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 msgr2=0x5592c7129c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e25170700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c7129c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e25971700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c7129c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e25971700 1 -- 172.21.15.112:0/2708334473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 msgr2=0x5592c712a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e25971700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c712a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e25971700 1 -- 172.21.15.112:0/2708334473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c712a9a0 con 0x5592c7135d00 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e26172700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c712a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e25971700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c715a0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0cc050 tx=0x7f1e1c0a91e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b7d7df88d17f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e097fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e097fa700 1 -- 172.21.15.112:0/2708334473 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0a5ba0 con 0x5592c7135d00 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e097fa700 1 -- 172.21.15.112:0/2708334473 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1e1c0910b0 con 0x5592c7135d00 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e1c0c2020 con 0x5592c7135d00 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e097fa700 1 -- 172.21.15.112:0/2708334473 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e1c0049e0 con 0x5592c7135d00 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e32468b80 10 client.15078 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:07.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592c71ba7b0 con 0x5592c7135d00 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e097fa700 1 -- 172.21.15.112:0/2708334473 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1c0c2020 con 0x5592c7135d00 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e097fa700 1 -- 172.21.15.112:0/2708334473 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e1c0b0ce0 con 0x5592c7135d00 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e32468b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e32468b80 10 client.15078 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.426+0000 7f1e32468b80 20 client.15078 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c71beef0 0x5592c71bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e32468b80 4 client.15078 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "status"}) -- 0x5592c708f370 con 0x5592c71beef0 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e25170700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c71beef0 0x5592c71bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e097fa700 10 client.15078 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e25170700 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c71beef0 0x5592c71bdc00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c7798fb6ddd573d server_cookie=7dc9c5759c603f62 in_seq=0 out_seq=0 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e097fa700 1 -- 172.21.15.112:0/2708334473 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5592c708f370 con 0x5592c71beef0 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e097fa700 10 client.15078 handle_command_reply: tid=0 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e32468b80 1 client.15078 shutdown 2024-05-10T10:39:07.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 msgr2=0x5592c715a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.427+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c715a0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0cc050 tx=0x7f1e1c0a91e0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 shutdown_connections 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5592c71beef0 0x5592c71bdc00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592c713ba80 0x5592c712a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592c7135d00 0x5592c715a0e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 --2- 172.21.15.112:0/2708334473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592c713b160 0x5592c7129c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e097fa700 0 client.15078 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 >> 172.21.15.112:0/2708334473 conn(0x5592c7061b20 msgr2=0x5592c7139bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 shutdown_connections 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 1 -- 172.21.15.112:0/2708334473 wait complete. 2024-05-10T10:39:07.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e32468b80 20 client.15078 trim_cache size 0 max 16384 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 3, 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24445, 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 128.04439485899999, 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 231.568939965 2024-05-10T10:39:07.438 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e0c0b2880 msgr2=0x7f1e0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e2006bd30 tx=0x7f1e10005fb0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 msgr2=0x7f1e2019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.428+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e2019a950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00d980 tx=0x7f1e1c00dc90 comp rx=0 tx=0).stop 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 shutdown_connections 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e20108410 0x7f1e201a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e201023a0 0x7f1e2019a950 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 --2- 172.21.15.112:0/2001477069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e20102d60 0x7f1e2019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 >> 172.21.15.112:0/2001477069 conn(0x7f1e20079350 msgr2=0x7f1e20101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 shutdown_connections 2024-05-10T10:39:07.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:07.429+0000 7f1e27bd5700 1 -- 172.21.15.112:0/2001477069 wait complete. 2024-05-10T10:39:07.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:07 smithi148 ceph-mon[33568]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:08.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-10T10:39:08.397 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:09.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:09 smithi112 ceph-mon[30100]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:09 smithi195 ceph-mon[33773]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:09.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:09 smithi148 ceph-mon[33568]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:10.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 -- 172.21.15.112:0/892781353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78103750 msgr2=0x7f0c78103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/892781353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78103750 0x7f0c78103b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0c680099c0 tx=0x7f0c68009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 -- 172.21.15.112:0/892781353 shutdown_connections 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/892781353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c78108850 0x7f0c781103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/892781353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78103750 0x7f0c78103b30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/892781353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78104100 0x7f0c78108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 -- 172.21.15.112:0/892781353 >> 172.21.15.112:0/892781353 conn(0x7f0c7807a600 msgr2=0x7f0c7807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 -- 172.21.15.112:0/892781353 shutdown_connections 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.133+0000 7f0c7e20b700 1 -- 172.21.15.112:0/892781353 wait complete. 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 Processor -- start 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 -- start start 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c78103750 0x7f0c7819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78104100 0x7f0c7819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78108850 0x7f0c781998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c78113550 con 0x7f0c78108850 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781133d0 con 0x7f0c78104100 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c7e20b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781136d0 con 0x7f0c78103750 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78108850 0x7f0c781998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78108850 0x7f0c781998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53740/0 (socket says 172.21.15.112:53740) 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 -- 172.21.15.112:0/1834606524 learned_addr learned my addr 172.21.15.112:0/1834606524 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c777fe700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c78103750 0x7f0c7819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78104100 0x7f0c7819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 -- 172.21.15.112:0/1834606524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c78103750 msgr2=0x7f0c7819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c78103750 0x7f0c7819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 -- 172.21.15.112:0/1834606524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78104100 msgr2=0x7f0c7819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78104100 0x7f0c7819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.134+0000 7f0c77fff700 1 -- 172.21.15.112:0/1834606524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7819a010 con 0x7f0c78108850 2024-05-10T10:39:10.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.135+0000 7f0c77fff700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78108850 0x7f0c781998b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b970 tx=0x7f0c7000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.135+0000 7f0c757fa700 1 -- 172.21.15.112:0/1834606524 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7000d860 con 0x7f0c78108850 2024-05-10T10:39:10.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.135+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7819a300 con 0x7f0c78108850 2024-05-10T10:39:10.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.135+0000 7f0c757fa700 1 -- 172.21.15.112:0/1834606524 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c7000e430 con 0x7f0c78108850 2024-05-10T10:39:10.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.135+0000 7f0c757fa700 1 -- 172.21.15.112:0/1834606524 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70017660 con 0x7f0c78108850 2024-05-10T10:39:10.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.135+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a65c0 con 0x7f0c78108850 2024-05-10T10:39:10.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.137+0000 7f0c757fa700 1 -- 172.21.15.112:0/1834606524 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0c7000da00 con 0x7f0c78108850 2024-05-10T10:39:10.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.138+0000 7f0c757fa700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c640b2880 0x7f0c640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.138+0000 7f0c777fe700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c640b2880 0x7f0c640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.138+0000 7f0c757fa700 1 -- 172.21.15.112:0/1834606524 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c700e21d0 con 0x7f0c78108850 2024-05-10T10:39:10.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.138+0000 7f0c88a9eb80 1 Processor -- start 2024-05-10T10:39:10.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.138+0000 7f0c88a9eb80 1 -- start start 2024-05-10T10:39:10.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c777fe700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c640b2880 0x7f0c640b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009990 tx=0x7f0c6801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc89a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc88c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc7de810 con 0x55bcfc88c2a0 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc77d770 con 0x55bcfc88cc60 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc7da7d0 con 0x55bcfc89a350 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc88c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc88c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53752/0 (socket says 172.21.15.112:53752) 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 -- 172.21.15.112:0/835044233 learned_addr learned my addr 172.21.15.112:0/835044233 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c77fff700 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c777fe700 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc89a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 -- 172.21.15.112:0/835044233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 msgr2=0x55bcfc89a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc89a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 -- 172.21.15.112:0/835044233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 msgr2=0x55bcfc8a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 -- 172.21.15.112:0/835044233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfc72b880 con 0x55bcfc88c2a0 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c777fe700 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc89a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c77fff700 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc88c720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000ac80 tx=0x7f0c6000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1c00a6746ec8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6dffb700 1 -- 172.21.15.112:0/835044233 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6001d070 con 0x55bcfc88c2a0 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6dffb700 1 -- 172.21.15.112:0/835044233 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c6000fe30 con 0x55bcfc88c2a0 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.139+0000 7f0c6dffb700 1 -- 172.21.15.112:0/835044233 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60010720 con 0x55bcfc88c2a0 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/835044233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 msgr2=0x55bcfc88c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc88c720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000ac80 tx=0x7f0c6000af90 comp rx=0 tx=0).stop 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/835044233 shutdown_connections 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc89a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/835044233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc88c720 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/835044233 >> 172.21.15.112:0/835044233 conn(0x55bcfc7b80b0 msgr2=0x55bcfc886c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/835044233 shutdown_connections 2024-05-10T10:39:10.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/835044233 wait complete. 2024-05-10T10:39:10.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.140+0000 7f0c88a9eb80 1 -- start start 2024-05-10T10:39:10.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 1 Processor -- start 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc89e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc89a350 0x55bcfc89e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8a1770 con 0x55bcfc89a350 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc744be0 con 0x55bcfc88cc60 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8a18f0 con 0x55bcfc88c2a0 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c88a9eb80 10 client.0 _ll_register_callbacks cb 0x55bcfc7768d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc89a350 0x55bcfc89e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc89a350 0x55bcfc89e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53756/0 (socket says 172.21.15.112:53756) 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 -- 172.21.15.112:0/2731457774 learned_addr learned my addr 172.21.15.112:0/2731457774 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc89e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c777fe700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 -- 172.21.15.112:0/2731457774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 msgr2=0x55bcfc8a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 -- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 msgr2=0x55bcfc89e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc89e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c77fff700 1 -- 172.21.15.112:0/2731457774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfc77a640 con 0x55bcfc89a350 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc89e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.141+0000 7f0c777fe700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c77fff700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc89a350 0x55bcfc89e590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a66d0 tx=0x7f0c700a9c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d48b8c54dbf06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c56ffd700 1 -- 172.21.15.112:0/2731457774 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700c3060 con 0x55bcfc89a350 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c56ffd700 1 -- 172.21.15.112:0/2731457774 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c700a5a50 con 0x55bcfc89a350 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c6000fe30 con 0x55bcfc89a350 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c56ffd700 1 -- 172.21.15.112:0/2731457774 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0c700b22c0 con 0x55bcfc89a350 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 10 client.15090 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bcfc8abf60 con 0x55bcfc89a350 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c56ffd700 1 -- 172.21.15.112:0/2731457774 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c6000fe30 con 0x55bcfc89a350 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c56ffd700 1 -- 172.21.15.112:0/2731457774 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0c7006d030 con 0x55bcfc89a350 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 20 client.15090 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc8ae550 0x55bcfc8af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 4 client.15090 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.142+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcfc858600 con 0x55bcfc8ae550 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.143+0000 7f0c777fe700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc8ae550 0x55bcfc8af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.143+0000 7f0c56ffd700 10 client.15090 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.143+0000 7f0c777fe700 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc8ae550 0x55bcfc8af9c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f74633aa66e4f29 server_cookie=3371f75bb0860b04 in_seq=0 out_seq=0 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c56ffd700 1 -- 172.21.15.112:0/2731457774 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcfc858600 con 0x55bcfc8ae550 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c56ffd700 10 client.15090 handle_command_reply: tid=0 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 client.15090 shutdown 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc89a350 msgr2=0x55bcfc89e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc89a350 0x55bcfc89e590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a66d0 tx=0x7f0c700a9c10 comp rx=0 tx=0).stop 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 shutdown_connections 2024-05-10T10:39:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc8ae550 0x55bcfc8af9c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc89a350 0x55bcfc89e590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/2731457774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc89e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c56ffd700 0 client.15090 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 >> 172.21.15.112:0/2731457774 conn(0x55bcfc7b80b0 msgr2=0x55bcfc8816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 shutdown_connections 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/2731457774 wait complete. 2024-05-10T10:39:10.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.144+0000 7f0c88a9eb80 20 client.15090 trim_cache size 0 max 16384 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.164+0000 7f0c88a9eb80 1 Processor -- start 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c88a9eb80 1 -- start start 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc899170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88cc60 0x55bcfc8996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc89a350 0x55bcfc899bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8add00 con 0x55bcfc88cc60 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8ae3d0 con 0x55bcfc89a350 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8aeaa0 con 0x55bcfc88c2a0 2024-05-10T10:39:10.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88cc60 0x55bcfc8996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88cc60 0x55bcfc8996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53770/0 (socket says 172.21.15.112:53770) 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 -- 172.21.15.112:0/660453556 learned_addr learned my addr 172.21.15.112:0/660453556 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c77fff700 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc89a350 0x55bcfc899bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 -- 172.21.15.112:0/660453556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 msgr2=0x55bcfc899170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc899170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 -- 172.21.15.112:0/660453556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc89a350 msgr2=0x55bcfc899bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc89a350 0x55bcfc899bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 -- 172.21.15.112:0/660453556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfc8a18a0 con 0x55bcfc88cc60 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.165+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88cc60 0x55bcfc8996b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000a720 tx=0x7f0c60005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948e0f8423036c14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c6d7fa700 1 -- 172.21.15.112:0/660453556 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6001d050 con 0x55bcfc88cc60 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c6d7fa700 1 -- 172.21.15.112:0/660453556 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c6001e040 con 0x55bcfc88cc60 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c6d7fa700 1 -- 172.21.15.112:0/660453556 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60005530 con 0x55bcfc88cc60 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/660453556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88cc60 msgr2=0x55bcfc8996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88cc60 0x55bcfc8996b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000a720 tx=0x7f0c60005db0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/660453556 shutdown_connections 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc89a350 0x55bcfc899bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc88c2a0 0x55bcfc899170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/660453556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88cc60 0x55bcfc8996b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/660453556 >> 172.21.15.112:0/660453556 conn(0x55bcfc7b80b0 msgr2=0x55bcfc89c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/660453556 shutdown_connections 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.166+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/660453556 wait complete. 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 -- start start 2024-05-10T10:39:10.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 Processor -- start 2024-05-10T10:39:10.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc8aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8af840 con 0x55bcfc88c2a0 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8af170 con 0x55bcfc88cc60 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcfc8aff10 con 0x55bcfc89a350 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c88a9eb80 10 client.0 _ll_register_callbacks cb 0x55bcfc7768d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8b9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53782/0 (socket says 172.21.15.112:53782) 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.167+0000 7f0c777fe700 1 -- 172.21.15.112:0/3097967405 learned_addr learned my addr 172.21.15.112:0/3097967405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c77fff700 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc8aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c777fe700 1 -- 172.21.15.112:0/3097967405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 msgr2=0x55bcfc8aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c777fe700 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc8aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c777fe700 1 -- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 msgr2=0x55bcfc8a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c777fe700 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c777fe700 1 -- 172.21.15.112:0/3097967405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfc8aa940 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c777fe700 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8b9410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005b40 tx=0x7f0c68005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd1c18420ab8ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c56ffd700 1 -- 172.21.15.112:0/3097967405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c680341b0 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c56ffd700 1 -- 172.21.15.112:0/3097967405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c68034350 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c6001e040 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c56ffd700 1 -- 172.21.15.112:0/3097967405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0c68042050 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c88a9eb80 10 client.15096 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.168+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bcfc919a70 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c56ffd700 1 -- 172.21.15.112:0/3097967405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c6001e040 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c56ffd700 1 -- 172.21.15.112:0/3097967405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0c68003b80 con 0x55bcfc88c2a0 2024-05-10T10:39:10.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c88a9eb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c88a9eb80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c88a9eb80 20 client.15096 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc91e640 0x55bcfc91d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c88a9eb80 4 client.15096 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "status"}) -- 0x55bcfc7ec1f0 con 0x55bcfc91e640 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.169+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc91e640 0x55bcfc91d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c6ffff700 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc91e640 0x55bcfc91d350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17e98cd3ec938b76 server_cookie=57cb9f539365294a in_seq=0 out_seq=0 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c56ffd700 10 client.15096 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c56ffd700 1 -- 172.21.15.112:0/3097967405 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bcfc7ec1f0 con 0x55bcfc91e640 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c56ffd700 10 client.15096 handle_command_reply: tid=0 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 client.15096 shutdown 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 msgr2=0x55bcfc8b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8b9410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005b40 tx=0x7f0c68005fb0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 shutdown_connections 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfc89a350 0x55bcfc8aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55bcfc91e640 0x55bcfc91d350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bcfc88c2a0 0x55bcfc8b9410 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 --2- 172.21.15.112:0/3097967405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfc88cc60 0x55bcfc8a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 >> 172.21.15.112:0/3097967405 conn(0x55bcfc7b80b0 msgr2=0x55bcfc897eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c56ffd700 0 client.15096 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 shutdown_connections 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.170+0000 7f0c88a9eb80 1 -- 172.21.15.112:0/3097967405 wait complete. 2024-05-10T10:39:10.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c88a9eb80 20 client.15096 trim_cache size 0 max 16384 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24446, 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 131.795806492, 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:10.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:10.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 214.94175817199999 2024-05-10T10:39:10.180 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c640b2880 msgr2=0x7f0c640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c640b2880 0x7f0c640b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009990 tx=0x7f0c6801a040 comp rx=0 tx=0).stop 2024-05-10T10:39:10.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78108850 msgr2=0x7f0c781998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78108850 0x7f0c781998b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b970 tx=0x7f0c7000bc80 comp rx=0 tx=0).stop 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 shutdown_connections 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c78108850 0x7f0c781998b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c640b2880 0x7f0c640b4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c78103750 0x7f0c7819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 --2- 172.21.15.112:0/1834606524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78104100 0x7f0c7819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 >> 172.21.15.112:0/1834606524 conn(0x7f0c7807a600 msgr2=0x7f0c781018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 shutdown_connections 2024-05-10T10:39:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:10.171+0000 7f0c7e20b700 1 -- 172.21.15.112:0/1834606524 wait complete. 2024-05-10T10:39:10.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-10T10:39:11.237 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:11 smithi195 ceph-mon[33773]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:11 smithi112 ceph-mon[30100]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:11 smithi148 ceph-mon[33568]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 -- 172.21.15.112:0/1710022890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006b180 msgr2=0x7fc24006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 --2- 172.21.15.112:0/1710022890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006b180 0x7fc24006b600 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc234009990 tx=0x7fc234009ca0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 -- 172.21.15.112:0/1710022890 shutdown_connections 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 --2- 172.21.15.112:0/1710022890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc24006bb40 0x7fc240110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 --2- 172.21.15.112:0/1710022890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc240109510 0x7fc2401098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 --2- 172.21.15.112:0/1710022890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006b180 0x7fc24006b600 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 -- 172.21.15.112:0/1710022890 >> 172.21.15.112:0/1710022890 conn(0x7fc240079350 msgr2=0x7fc240079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:13.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 -- 172.21.15.112:0/1710022890 shutdown_connections 2024-05-10T10:39:13.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.050+0000 7fc246225700 1 -- 172.21.15.112:0/1710022890 wait complete. 2024-05-10T10:39:13.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 Processor -- start 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 -- start start 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc24006b180 0x7fc24019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006bb40 0x7fc24019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc240109510 0x7fc240199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401135d0 con 0x7fc24006bb40 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc240113450 con 0x7fc24006b180 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.051+0000 7fc246225700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc240113750 con 0x7fc240109510 2024-05-10T10:39:13.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006bb40 0x7fc24019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006bb40 0x7fc24019fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53802/0 (socket says 172.21.15.112:53802) 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 -- 172.21.15.112:0/388388207 learned_addr learned my addr 172.21.15.112:0/388388207 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc24006b180 0x7fc24019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23ffff700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc240109510 0x7fc240199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 -- 172.21.15.112:0/388388207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc240109510 msgr2=0x7fc240199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc240109510 0x7fc240199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 -- 172.21.15.112:0/388388207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc24006b180 msgr2=0x7fc24019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc24006b180 0x7fc24019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 -- 172.21.15.112:0/388388207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24019a190 con 0x7fc24006bb40 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23ffff700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc240109510 0x7fc240199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:13.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.052+0000 7fc23effd700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006bb40 0x7fc24019fdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc234000c00 tx=0x7fc2340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:13.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.055+0000 7fc23cff9700 1 -- 172.21.15.112:0/388388207 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234027070 con 0x7fc24006bb40 2024-05-10T10:39:13.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.055+0000 7fc23cff9700 1 -- 172.21.15.112:0/388388207 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc23400fd00 con 0x7fc24006bb40 2024-05-10T10:39:13.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.055+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24019a420 con 0x7fc24006bb40 2024-05-10T10:39:13.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.055+0000 7fc23cff9700 1 -- 172.21.15.112:0/388388207 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234022e20 con 0x7fc24006bb40 2024-05-10T10:39:13.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.055+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2401a6740 con 0x7fc24006bb40 2024-05-10T10:39:13.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.058+0000 7fc23cff9700 1 -- 172.21.15.112:0/388388207 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc23402a070 con 0x7fc24006bb40 2024-05-10T10:39:13.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.058+0000 7fc23cff9700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc2280b2900 0x7fc2280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc2280b2900 0x7fc2280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc23cff9700 1 -- 172.21.15.112:0/388388207 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc2340e32e0 con 0x7fc24006bb40 2024-05-10T10:39:13.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 Processor -- start 2024-05-10T10:39:13.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc2280b2900 0x7fc2280b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc24019ab70 tx=0x7fc230009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 -- start start 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc544197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5441a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc5436b8f0 con 0x55cc54419d00 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cc542d0d50 con 0x55cc5441f160 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc543678b0 con 0x55cc5441fa80 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc23ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5441a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc23ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5441a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53806/0 (socket says 172.21.15.112:53806) 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc23ffff700 1 -- 172.21.15.112:0/154244965 learned_addr learned my addr 172.21.15.112:0/154244965 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.059+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23effd700 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc544197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23ffff700 1 -- 172.21.15.112:0/154244965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 msgr2=0x55cc544197c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23ffff700 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc544197c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23ffff700 1 -- 172.21.15.112:0/154244965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 msgr2=0x55cc5441f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23ffff700 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23ffff700 1 -- 172.21.15.112:0/154244965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc542b8880 con 0x55cc54419d00 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc23ffff700 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5441a140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c450 tx=0x7fc23800c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca761fb578626b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc225ffb700 1 -- 172.21.15.112:0/154244965 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23800cbb0 con 0x55cc54419d00 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc225ffb700 1 -- 172.21.15.112:0/154244965 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc23800cd50 con 0x55cc54419d00 2024-05-10T10:39:13.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.060+0000 7fc225ffb700 1 -- 172.21.15.112:0/154244965 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380155e0 con 0x55cc54419d00 2024-05-10T10:39:13.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 -- 172.21.15.112:0/154244965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 msgr2=0x55cc5441a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5441a140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c450 tx=0x7fc23800c480 comp rx=0 tx=0).stop 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 -- 172.21.15.112:0/154244965 shutdown_connections 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5441a140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/154244965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc544197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 -- 172.21.15.112:0/154244965 >> 172.21.15.112:0/154244965 conn(0x55cc54345b20 msgr2=0x55cc54414250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 -- 172.21.15.112:0/154244965 shutdown_connections 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 -- 172.21.15.112:0/154244965 wait complete. 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 -- start start 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.061+0000 7fc250ab8b80 1 Processor -- start 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc54425620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc544229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54422f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc54426400 con 0x55cc54419d00 2024-05-10T10:39:13.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cc54324e80 con 0x55cc5441fa80 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc54426580 con 0x55cc5441f160 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 10 client.0 _ll_register_callbacks cb 0x55cc542ff560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc54425620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc54425620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53822/0 (socket says 172.21.15.112:53822) 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 -- 172.21.15.112:0/3597762523 learned_addr learned my addr 172.21.15.112:0/3597762523 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23effd700 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc544229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23ffff700 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54422f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 -- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 msgr2=0x55cc544229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc544229c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 -- 172.21.15.112:0/3597762523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 msgr2=0x55cc54422f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54422f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 -- 172.21.15.112:0/3597762523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc54322710 con 0x55cc54419d00 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc54425620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc230015780 tx=0x7fc230004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e29eaf08b595863 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:13.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc21effd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc21effd700 1 -- 172.21.15.112:0/3597762523 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230025020 con 0x55cc54419d00 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc21effd700 1 -- 172.21.15.112:0/3597762523 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc230015cf0 con 0x55cc54419d00 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.062+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23800cd50 con 0x55cc54419d00 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc21effd700 1 -- 172.21.15.112:0/3597762523 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc230030050 con 0x55cc54419d00 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 10 client.15108 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc54430bd0 con 0x55cc54419d00 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc21effd700 1 -- 172.21.15.112:0/3597762523 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc23800cd50 con 0x55cc54419d00 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc21effd700 1 -- 172.21.15.112:0/3597762523 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc230013450 con 0x55cc54419d00 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 10 client.15108 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 20 client.15108 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc54433280 0x55cc544346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 4 client.15108 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc543792d0 con 0x55cc54433280 2024-05-10T10:39:13.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.063+0000 7fc23effd700 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc54433280 0x55cc544346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.064+0000 7fc21effd700 10 client.15108 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.064+0000 7fc23effd700 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc54433280 0x55cc544346f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a05e060d78491871 server_cookie=70c086909c84932b in_seq=0 out_seq=0 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.064+0000 7fc21effd700 1 -- 172.21.15.112:0/3597762523 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc543792d0 con 0x55cc54433280 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.064+0000 7fc21effd700 10 client.15108 handle_command_reply: tid=0 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.064+0000 7fc250ab8b80 1 client.15108 shutdown 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 msgr2=0x55cc54425620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc54425620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc230015780 tx=0x7fc230004860 comp rx=0 tx=0).stop 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 shutdown_connections 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc54433280 0x55cc544346f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54422f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc54425620 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/3597762523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc544229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 >> 172.21.15.112:0/3597762523 conn(0x55cc54345b20 msgr2=0x55cc5440eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:13.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc21effd700 0 client.15108 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 shutdown_connections 2024-05-10T10:39:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 1 -- 172.21.15.112:0/3597762523 wait complete. 2024-05-10T10:39:13.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.065+0000 7fc250ab8b80 20 client.15108 trim_cache size 0 max 16384 2024-05-10T10:39:13.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 Processor -- start 2024-05-10T10:39:13.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 -- start start 2024-05-10T10:39:13.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc544197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc5440ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc54432a30 con 0x55cc54419d00 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cc54433100 con 0x55cc5441f160 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc544337d0 con 0x55cc5441fa80 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc23f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc544197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.085+0000 7fc23f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc544197c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53834/0 (socket says 172.21.15.112:53834) 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23f7fe700 1 -- 172.21.15.112:0/130936596 learned_addr learned my addr 172.21.15.112:0/130936596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23effd700 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23ffff700 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc5440ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23f7fe700 1 -- 172.21.15.112:0/130936596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 msgr2=0x55cc5440ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc5440ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23f7fe700 1 -- 172.21.15.112:0/130936596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 msgr2=0x55cc5441a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23f7fe700 1 -- 172.21.15.112:0/130936596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5441eb60 con 0x55cc54419d00 2024-05-10T10:39:13.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc544197c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc230013870 tx=0x7fc23000aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba03e0b5c121f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc2257fa700 1 -- 172.21.15.112:0/130936596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230025040 con 0x55cc54419d00 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc2257fa700 1 -- 172.21.15.112:0/130936596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc230039070 con 0x55cc54419d00 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.086+0000 7fc2257fa700 1 -- 172.21.15.112:0/130936596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23001b040 con 0x55cc54419d00 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 -- 172.21.15.112:0/130936596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 msgr2=0x55cc544197c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc544197c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc230013870 tx=0x7fc23000aec0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 -- 172.21.15.112:0/130936596 shutdown_connections 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441fa80 0x55cc5440ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc544197c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/130936596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441f160 0x55cc5441a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 -- 172.21.15.112:0/130936596 >> 172.21.15.112:0/130936596 conn(0x55cc54345b20 msgr2=0x55cc544212b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 -- 172.21.15.112:0/130936596 shutdown_connections 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 -- 172.21.15.112:0/130936596 wait complete. 2024-05-10T10:39:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 -- start start 2024-05-10T10:39:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.087+0000 7fc250ab8b80 1 Processor -- start 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5443db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc5443e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54438b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc54434570 con 0x55cc54419d00 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cc54433ea0 con 0x55cc5441fa80 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc54434c40 con 0x55cc5441f160 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 10 client.0 _ll_register_callbacks cb 0x55cc542ff560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5443db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5443db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53840/0 (socket says 172.21.15.112:53840) 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 -- 172.21.15.112:0/492259925 learned_addr learned my addr 172.21.15.112:0/492259925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23effd700 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc5443e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23ffff700 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54438b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 -- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 msgr2=0x55cc5443e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc5443e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 -- 172.21.15.112:0/492259925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 msgr2=0x55cc54438b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54438b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 -- 172.21.15.112:0/492259925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc544390e0 con 0x55cc54419d00 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc23f7fe700 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5443db70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc230004780 tx=0x7fc230009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10da5e541d649338 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc21effd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc21effd700 1 -- 172.21.15.112:0/492259925 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230042060 con 0x55cc54419d00 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc21effd700 1 -- 172.21.15.112:0/492259925 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc230005a50 con 0x55cc54419d00 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.088+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc230039070 con 0x55cc54419d00 2024-05-10T10:39:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc21effd700 1 -- 172.21.15.112:0/492259925 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc230049050 con 0x55cc54419d00 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 10 client.15114 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc5449efb0 con 0x55cc54419d00 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc21effd700 1 -- 172.21.15.112:0/492259925 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc230039070 con 0x55cc54419d00 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc21effd700 1 -- 172.21.15.112:0/492259925 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc230021470 con 0x55cc54419d00 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 10 client.15114 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 20 client.15114 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc544a37a0 0x55cc544a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 4 client.15114 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "status"}) -- 0x55cc54373370 con 0x55cc544a37a0 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.089+0000 7fc23effd700 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc544a37a0 0x55cc544a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc21effd700 10 client.15114 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc23effd700 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc544a37a0 0x55cc544a24b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b93beac534823c6 server_cookie=15e4070b838543ee in_seq=0 out_seq=0 2024-05-10T10:39:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc21effd700 1 -- 172.21.15.112:0/492259925 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cc54373370 con 0x55cc544a37a0 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc21effd700 10 client.15114 handle_command_reply: tid=0 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 client.15114 shutdown 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 msgr2=0x55cc5443db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5443db70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc230004780 tx=0x7fc230009860 comp rx=0 tx=0).stop 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 shutdown_connections 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55cc544a37a0 0x55cc544a24b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc5441fa80 0x55cc54438b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc54419d00 0x55cc5443db70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 --2- 172.21.15.112:0/492259925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc5441f160 0x55cc5443e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 >> 172.21.15.112:0/492259925 conn(0x55cc54345b20 msgr2=0x55cc5441fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc21effd700 0 client.15114 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 shutdown_connections 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 1 -- 172.21.15.112:0/492259925 wait complete. 2024-05-10T10:39:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.090+0000 7fc250ab8b80 20 client.15114 trim_cache size 0 max 16384 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24451, 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 135.72388437699999, 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 234.50075042700001 2024-05-10T10:39:13.099 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc2280b2900 msgr2=0x7fc2280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc2280b2900 0x7fc2280b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc24019ab70 tx=0x7fc230009190 comp rx=0 tx=0).stop 2024-05-10T10:39:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006bb40 msgr2=0x7fc24019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006bb40 0x7fc24019fdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc234000c00 tx=0x7fc2340047b0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 shutdown_connections 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc2280b2900 0x7fc2280b4dc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc240109510 0x7fc240199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc24006b180 0x7fc24019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 --2- 172.21.15.112:0/388388207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24006bb40 0x7fc24019fdb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 >> 172.21.15.112:0/388388207 conn(0x7fc240079350 msgr2=0x7fc240101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 shutdown_connections 2024-05-10T10:39:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:13.091+0000 7fc246225700 1 -- 172.21.15.112:0/388388207 wait complete. 2024-05-10T10:39:13.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:13 smithi195 ceph-mon[33773]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:13.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:13 smithi112 ceph-mon[30100]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:13.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:13 smithi148 ceph-mon[33568]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:13.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T10:39:14.032 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:15.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:15 smithi112 ceph-mon[30100]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:15 smithi195 ceph-mon[33773]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:15.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:15 smithi148 ceph-mon[33568]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:15.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3921266316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 msgr2=0x7fe9b41087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3921266316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b41087f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b0a0 tx=0x7fe9ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3921266316 shutdown_connections 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3921266316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b4102d60 0x7fe9b410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3921266316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b41087f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3921266316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b41023a0 0x7fe9b4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3921266316 >> 172.21.15.112:0/3921266316 conn(0x7fe9b4079350 msgr2=0x7fe9b4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3921266316 shutdown_connections 2024-05-10T10:39:15.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3921266316 wait complete. 2024-05-10T10:39:15.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.810+0000 7fe9ba2a1700 1 Processor -- start 2024-05-10T10:39:15.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9ba2a1700 1 -- start start 2024-05-10T10:39:15.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9ba2a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b41023a0 0x7fe9b419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9ba2a1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b4102d60 0x7fe9b419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9ba2a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9ba2a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41123e0 con 0x7fe9b4108410 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9ba2a1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b4112260 con 0x7fe9b4102d60 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9ba2a1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b4112560 con 0x7fe9b41023a0 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38260/0 (socket says 172.21.15.112:38260) 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 -- 172.21.15.112:0/3966176838 learned_addr learned my addr 172.21.15.112:0/3966176838 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b4102d60 0x7fe9b419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b41023a0 0x7fe9b419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 -- 172.21.15.112:0/3966176838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b41023a0 msgr2=0x7fe9b419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b41023a0 0x7fe9b419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 -- 172.21.15.112:0/3966176838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b4102d60 msgr2=0x7fe9b419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b4102d60 0x7fe9b419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.811+0000 7fe9b883e700 1 -- 172.21.15.112:0/3966176838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b419a0d0 con 0x7fe9b4108410 2024-05-10T10:39:15.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.812+0000 7fe9b883e700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b4199970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400d980 tx=0x7fe9a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:15.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.812+0000 7fe9b17fa700 1 -- 172.21.15.112:0/3966176838 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4016070 con 0x7fe9b4108410 2024-05-10T10:39:15.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.812+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b419a360 con 0x7fe9b4108410 2024-05-10T10:39:15.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.812+0000 7fe9b17fa700 1 -- 172.21.15.112:0/3966176838 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9a4004830 con 0x7fe9b4108410 2024-05-10T10:39:15.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.812+0000 7fe9b17fa700 1 -- 172.21.15.112:0/3966176838 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4005480 con 0x7fe9b4108410 2024-05-10T10:39:15.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.812+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b41a6680 con 0x7fe9b4108410 2024-05-10T10:39:15.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.814+0000 7fe9b17fa700 1 -- 172.21.15.112:0/3966176838 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe9a400f690 con 0x7fe9b4108410 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.815+0000 7fe9b17fa700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe9a00b2880 0x7fe9a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.815+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe9a00b2880 0x7fe9a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.815+0000 7fe9b17fa700 1 -- 172.21.15.112:0/3966176838 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe9a40e2290 con 0x7fe9b4108410 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.815+0000 7fe9c4b34b80 1 Processor -- start 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.815+0000 7fe9c4b34b80 1 -- start start 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe9a00b2880 0x7fe9a00b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009e60 tx=0x7fe9ac00b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594affe6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594affea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594affeb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5594aff38810 con 0x5594affe6480 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594afed7770 con 0x5594affeac30 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594aff347d0 con 0x5594affe6da0 2024-05-10T10:39:15.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594affe6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594affe6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38266/0 (socket says 172.21.15.112:38266) 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 -- 172.21.15.112:0/299862470 learned_addr learned my addr 172.21.15.112:0/299862470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594affea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b883e700 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594affeb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 -- 172.21.15.112:0/299862470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 msgr2=0x5594affea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594affea6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 -- 172.21.15.112:0/299862470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 msgr2=0x5594affeb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594affeb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 -- 172.21.15.112:0/299862470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594afe85880 con 0x5594affe6480 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b883e700 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594affeb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.816+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594affe6860 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b0a0 tx=0x7fe9ac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7eed63fc2b1753 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9aa7fc700 1 -- 172.21.15.112:0/299862470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac003e60 con 0x5594affe6480 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9aa7fc700 1 -- 172.21.15.112:0/299862470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9ac02e070 con 0x5594affe6480 2024-05-10T10:39:15.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9aa7fc700 1 -- 172.21.15.112:0/299862470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac029810 con 0x5594affe6480 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/299862470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 msgr2=0x5594affe6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594affe6860 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b0a0 tx=0x7fe9ac01d000 comp rx=0 tx=0).stop 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/299862470 shutdown_connections 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594affeb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594affe6860 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/299862470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594affea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/299862470 >> 172.21.15.112:0/299862470 conn(0x5594aff120b0 msgr2=0x5594affe0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/299862470 shutdown_connections 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.817+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/299862470 wait complete. 2024-05-10T10:39:15.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 -- start start 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 Processor -- start 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594afff20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6da0 0x5594affef450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 0x5594affef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5594afff2de0 con 0x5594affe6480 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594afe9ebe0 con 0x5594affe6da0 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594afff2f60 con 0x5594affeac30 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9c4b34b80 10 client.0 _ll_register_callbacks cb 0x5594afed08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594afff20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594afff20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38280/0 (socket says 172.21.15.112:38280) 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9b3fff700 1 -- 172.21.15.112:0/1508597104 learned_addr learned my addr 172.21.15.112:0/1508597104 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.818+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6da0 0x5594affef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9b883e700 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 0x5594affef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9b3fff700 1 -- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 msgr2=0x5594affef990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 0x5594affef990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9b3fff700 1 -- 172.21.15.112:0/1508597104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6da0 msgr2=0x5594affef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6da0 0x5594affef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9b3fff700 1 -- 172.21.15.112:0/1508597104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594afed4640 con 0x5594affe6480 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594afff20b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b070 tx=0x7fe9ac007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ea19cc56e13d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9937fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9937fe700 1 -- 172.21.15.112:0/1508597104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac003e60 con 0x5594affe6480 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9937fe700 1 -- 172.21.15.112:0/1508597104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe9ac029680 con 0x5594affe6480 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9ac02e070 con 0x5594affe6480 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9937fe700 1 -- 172.21.15.112:0/1508597104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9ac044050 con 0x5594affe6480 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9c4b34b80 10 client.15126 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:15.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594afffd5d0 con 0x5594affe6480 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.819+0000 7fe9937fe700 1 -- 172.21.15.112:0/1508597104 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9ac02e070 con 0x5594affe6480 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9937fe700 1 -- 172.21.15.112:0/1508597104 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe9ac011070 con 0x5594affe6480 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9c4b34b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9c4b34b80 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9c4b34b80 20 client.15126 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594affff730 0x5594b0000ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9c4b34b80 4 client.15126 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594affb2600 con 0x5594affff730 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594affff730 0x5594b0000ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594affff730 0x5594b0000ba0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccaceedd283b8551 server_cookie=6aa11002bdc19dd7 in_seq=0 out_seq=0 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.820+0000 7fe9937fe700 10 client.15126 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9937fe700 1 -- 172.21.15.112:0/1508597104 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594affb2600 con 0x5594affff730 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9937fe700 10 client.15126 handle_command_reply: tid=0 2024-05-10T10:39:15.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 client.15126 shutdown 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 msgr2=0x5594afff20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594afff20b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b070 tx=0x7fe9ac007a40 comp rx=0 tx=0).stop 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 shutdown_connections 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594affff730 0x5594b0000ba0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 0x5594affef990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594afff20b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/1508597104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6da0 0x5594affef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 >> 172.21.15.112:0/1508597104 conn(0x5594aff120b0 msgr2=0x5594affdb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9937fe700 0 client.15126 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 shutdown_connections 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/1508597104 wait complete. 2024-05-10T10:39:15.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.821+0000 7fe9c4b34b80 20 client.15126 trim_cache size 0 max 16384 2024-05-10T10:39:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 Processor -- start 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 -- start start 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6480 0x5594affefa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6da0 0x5594affe0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 0x5594b000a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5594afffeee0 con 0x5594affe6da0 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594affff5b0 con 0x5594affe6480 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594affffc80 con 0x5594affeac30 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6da0 0x5594affe0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6da0 0x5594affe0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38290/0 (socket says 172.21.15.112:38290) 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 -- 172.21.15.112:0/541036042 learned_addr learned my addr 172.21.15.112:0/541036042 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6480 0x5594affefa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 -- 172.21.15.112:0/541036042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 msgr2=0x5594b000a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 0x5594b000a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 -- 172.21.15.112:0/541036042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6480 msgr2=0x5594affefa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6480 0x5594affefa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.842+0000 7fe9b37fe700 1 -- 172.21.15.112:0/541036042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594affeda20 con 0x5594affe6da0 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6da0 0x5594affe0b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe99c004790 tx=0x7fe99c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f176415d4fc8eae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9a9ffb700 1 -- 172.21.15.112:0/541036042 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c004310 con 0x5594affe6da0 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9a9ffb700 1 -- 172.21.15.112:0/541036042 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe99c0044b0 con 0x5594affe6da0 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9a9ffb700 1 -- 172.21.15.112:0/541036042 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c003680 con 0x5594affe6da0 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/541036042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6da0 msgr2=0x5594affe0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6da0 0x5594affe0b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe99c004790 tx=0x7fe99c009260 comp rx=0 tx=0).stop 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/541036042 shutdown_connections 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affeac30 0x5594b000a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affe6480 0x5594affefa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/541036042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6da0 0x5594affe0b50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/541036042 >> 172.21.15.112:0/541036042 conn(0x5594aff120b0 msgr2=0x5594affdb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/541036042 shutdown_connections 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.843+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/541036042 wait complete. 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 -- start start 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 Processor -- start 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594b0005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594b0006460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594b0069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5594b0000a20 con 0x5594affe6480 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594b0000350 con 0x5594affeac30 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594b00010f0 con 0x5594affe6da0 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9c4b34b80 10 client.0 _ll_register_callbacks cb 0x5594afed08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594b0005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594b0005f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38306/0 (socket says 172.21.15.112:38306) 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9b3fff700 1 -- 172.21.15.112:0/797188642 learned_addr learned my addr 172.21.15.112:0/797188642 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.844+0000 7fe9b883e700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594b0069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594b0006460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b3fff700 1 -- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 msgr2=0x5594b0006460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594b0006460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b3fff700 1 -- 172.21.15.112:0/797188642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 msgr2=0x5594b0069ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594b0069ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b3fff700 1 -- 172.21.15.112:0/797188642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b0006d40 con 0x5594affe6480 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b883e700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594b0069ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9b3fff700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594b0005f20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00a010 tx=0x7fe9ac005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabe326c0ecb4815 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9937fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9937fe700 1 -- 172.21.15.112:0/797188642 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac020070 con 0x5594affe6480 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9937fe700 1 -- 172.21.15.112:0/797188642 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe9ac005250 con 0x5594affe6480 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99c0044b0 con 0x5594affe6480 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9937fe700 1 -- 172.21.15.112:0/797188642 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9ac029480 con 0x5594affe6480 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9c4b34b80 10 client.15132 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594b006b1f0 con 0x5594affe6480 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.845+0000 7fe9937fe700 1 -- 172.21.15.112:0/797188642 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe99c0044b0 con 0x5594affe6480 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9937fe700 1 -- 172.21.15.112:0/797188642 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe9ac011070 con 0x5594affe6480 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9c4b34b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9c4b34b80 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9c4b34b80 20 client.15132 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594b006fd50 0x5594b006ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9c4b34b80 4 client.15132 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "status"}) -- 0x5594aff461f0 con 0x5594b006fd50 2024-05-10T10:39:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594b006fd50 0x5594b006ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9937fe700 10 client.15132 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9b37fe700 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594b006fd50 0x5594b006ea60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b771b278353409b1 server_cookie=bd6ebe7ded8059ec in_seq=0 out_seq=0 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9937fe700 1 -- 172.21.15.112:0/797188642 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5594aff461f0 con 0x5594b006fd50 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9937fe700 10 client.15132 handle_command_reply: tid=0 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.846+0000 7fe9c4b34b80 1 client.15132 shutdown 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 msgr2=0x5594b0005f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594b0005f20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00a010 tx=0x7fe9ac005b40 comp rx=0 tx=0).stop 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 shutdown_connections 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5594b006fd50 0x5594b006ea60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594affeac30 0x5594b0069ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594affe6480 0x5594b0005f20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 --2- 172.21.15.112:0/797188642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594affe6da0 0x5594b0006460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9937fe700 0 client.15132 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 >> 172.21.15.112:0/797188642 conn(0x5594aff120b0 msgr2=0x5594affdb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:15.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 shutdown_connections 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 1 -- 172.21.15.112:0/797188642 wait complete. 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9c4b34b80 20 client.15132 trim_cache size 0 max 16384 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24469, 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 202.916805555, 2024-05-10T10:39:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 231.28529483700001 2024-05-10T10:39:15.859 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:15.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe9a00b2880 msgr2=0x7fe9a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe9a00b2880 0x7fe9a00b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009e60 tx=0x7fe9ac00b700 comp rx=0 tx=0).stop 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 msgr2=0x7fe9b4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.847+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b4199970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400d980 tx=0x7fe9a400dc90 comp rx=0 tx=0).stop 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 shutdown_connections 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9b4108410 0x7fe9b4199970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe9a00b2880 0x7fe9a00b4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b41023a0 0x7fe9b419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 --2- 172.21.15.112:0/3966176838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b4102d60 0x7fe9b419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 >> 172.21.15.112:0/3966176838 conn(0x7fe9b4079350 msgr2=0x7fe9b406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 shutdown_connections 2024-05-10T10:39:15.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:15.848+0000 7fe9ba2a1700 1 -- 172.21.15.112:0/3966176838 wait complete. 2024-05-10T10:39:16.557 INFO:tasks.cephadm:Setup complete, yielding 2024-05-10T10:39:16.557 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T10:39:16.569 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:39:16.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-10T10:39:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:17.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:17 smithi112 ceph-mon[30100]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:17 smithi195 ceph-mon[33773]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:17.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:17 smithi148 ceph-mon[33568]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:18.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 -- 172.21.15.112:0/4261521210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1023f0 msgr2=0x7fc05c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 --2- 172.21.15.112:0/4261521210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1023f0 0x7fc05c102810 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0099c0 tx=0x7fc04c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 -- 172.21.15.112:0/4261521210 shutdown_connections 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 --2- 172.21.15.112:0/4261521210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc05c103f90 0x7fc05c10a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 --2- 172.21.15.112:0/4261521210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1023f0 0x7fc05c102810 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 --2- 172.21.15.112:0/4261521210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc05c1035f0 0x7fc05c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 -- 172.21.15.112:0/4261521210 >> 172.21.15.112:0/4261521210 conn(0x7fc05c0fd990 msgr2=0x7fc05c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 -- 172.21.15.112:0/4261521210 shutdown_connections 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.550+0000 7fc0622bd700 1 -- 172.21.15.112:0/4261521210 wait complete. 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 Processor -- start 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 -- start start 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc05c1023f0 0x7fc05c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1035f0 0x7fc05c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc05c103f90 0x7fc05c1a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c10dd30 con 0x7fc05c1035f0 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c10dbb0 con 0x7fc05c1023f0 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc0622bd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c10deb0 con 0x7fc05c103f90 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1035f0 0x7fc05c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1035f0 0x7fc05c1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38334/0 (socket says 172.21.15.112:38334) 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc05b7fe700 1 -- 172.21.15.112:0/3508517127 learned_addr learned my addr 172.21.15.112:0/3508517127 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.551+0000 7fc05bfff700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc05c1023f0 0x7fc05c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc05b7fe700 1 -- 172.21.15.112:0/3508517127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc05c103f90 msgr2=0x7fc05c1a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc05b7fe700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc05c103f90 0x7fc05c1a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc05b7fe700 1 -- 172.21.15.112:0/3508517127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc05c1023f0 msgr2=0x7fc05c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc05b7fe700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc05c1023f0 0x7fc05c06c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc05b7fe700 1 -- 172.21.15.112:0/3508517127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c1a33f0 con 0x7fc05c1035f0 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc05b7fe700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1035f0 0x7fc05c1a06c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc04400d980 tx=0x7fc04400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc0597fa700 1 -- 172.21.15.112:0/3508517127 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044016070 con 0x7fc05c1035f0 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc0597fa700 1 -- 172.21.15.112:0/3508517127 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc044004830 con 0x7fc05c1035f0 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c1a36e0 con 0x7fc05c1035f0 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc0597fa700 1 -- 172.21.15.112:0/3508517127 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044005440 con 0x7fc05c1035f0 2024-05-10T10:39:18.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.552+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c1a3c70 con 0x7fc05c1035f0 2024-05-10T10:39:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.553+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05c06a190 con 0x7fc05c1035f0 2024-05-10T10:39:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.554+0000 7fc0597fa700 1 -- 172.21.15.112:0/3508517127 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc0440055e0 con 0x7fc05c1035f0 2024-05-10T10:39:18.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.555+0000 7fc0597fa700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc0480b2900 0x7fc0480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.555+0000 7fc05bfff700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc0480b2900 0x7fc0480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.555+0000 7fc0597fa700 1 -- 172.21.15.112:0/3508517127 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc0440e2380 con 0x7fc05c1035f0 2024-05-10T10:39:18.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.556+0000 7fc05bfff700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc0480b2900 0x7fc0480b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009990 tx=0x7fc04c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.560+0000 7fc0597fa700 1 -- 172.21.15.112:0/3508517127 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc04401f020 con 0x7fc05c1035f0 2024-05-10T10:39:18.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.781+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc05c0638e0 con 0x7fc0480b2900 2024-05-10T10:39:18.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.782+0000 7fc0597fa700 1 -- 172.21.15.112:0/3508517127 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc05c0638e0 con 0x7fc0480b2900 2024-05-10T10:39:18.783 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-05-10T10:39:18.783 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-05-10T10:39:18.783 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc0480b2900 msgr2=0x7fc0480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc0480b2900 0x7fc0480b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009990 tx=0x7fc04c005fb0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1035f0 msgr2=0x7fc05c1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1035f0 0x7fc05c1a06c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc04400d980 tx=0x7fc04400dc90 comp rx=0 tx=0).stop 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 shutdown_connections 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc0480b2900 0x7fc0480b4dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc05c103f90 0x7fc05c1a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc05c1023f0 0x7fc05c06c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 --2- 172.21.15.112:0/3508517127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc05c1035f0 0x7fc05c1a06c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.784+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 >> 172.21.15.112:0/3508517127 conn(0x7fc05c0fd990 msgr2=0x7fc05c106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.785+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 shutdown_connections 2024-05-10T10:39:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:18.785+0000 7fc0622bd700 1 -- 172.21.15.112:0/3508517127 wait complete. 2024-05-10T10:39:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:19 smithi195 ceph-mon[33773]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:19.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:19 smithi112 ceph-mon[30100]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:19.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-10T10:39:19.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:19 smithi148 ceph-mon[33568]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:20.137 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:20 smithi112 ceph-mon[30100]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:20.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:20 smithi148 ceph-mon[33568]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:20.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:20 smithi195 ceph-mon[33773]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:21.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:21 smithi112 ceph-mon[30100]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:21.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:21 smithi148 ceph-mon[33568]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:21.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:21 smithi195 ceph-mon[33773]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 -- 172.21.15.112:0/4089491105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 msgr2=0x7f41c8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4089491105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c8102820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0099c0 tx=0x7f41bc009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 -- 172.21.15.112:0/4089491105 shutdown_connections 2024-05-10T10:39:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4089491105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c8103fa0 0x7f41c810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4089491105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c8102820 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4089491105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c8103600 0x7f41c8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 -- 172.21.15.112:0/4089491105 >> 172.21.15.112:0/4089491105 conn(0x7f41c80fd9c0 msgr2=0x7f41c80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 -- 172.21.15.112:0/4089491105 shutdown_connections 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.588+0000 7f41cd32e700 1 -- 172.21.15.112:0/4089491105 wait complete. 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 Processor -- start 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 -- start start 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c8103600 0x7f41c819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c8103fa0 0x7f41c81a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810dda0 con 0x7f41c8102400 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810dc20 con 0x7f41c8103600 2024-05-10T10:39:21.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41cd32e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810df20 con 0x7f41c8103fa0 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41c6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c806c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38344/0 (socket says 172.21.15.112:38344) 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41c6ffd700 1 -- 172.21.15.112:0/4214232393 learned_addr learned my addr 172.21.15.112:0/4214232393 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.589+0000 7f41c67fc700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c8103600 0x7f41c819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41c77fe700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c8103fa0 0x7f41c81a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41c6ffd700 1 -- 172.21.15.112:0/4214232393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c8103fa0 msgr2=0x7f41c81a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41c6ffd700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c8103fa0 0x7f41c81a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41c6ffd700 1 -- 172.21.15.112:0/4214232393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c8103600 msgr2=0x7f41c819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41c6ffd700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c8103600 0x7f41c819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41c6ffd700 1 -- 172.21.15.112:0/4214232393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c81a1250 con 0x7f41c8102400 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41c6ffd700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c806c6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0098e0 tx=0x7f41bc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41affff700 1 -- 172.21.15.112:0/4214232393 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc027070 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c81a14e0 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41affff700 1 -- 172.21.15.112:0/4214232393 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41bc022470 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41affff700 1 -- 172.21.15.112:0/4214232393 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc003a50 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.590+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c81a1a10 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.591+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c806a190 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.592+0000 7f41affff700 1 -- 172.21.15.112:0/4214232393 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f41bc02a070 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.594+0000 7f41affff700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41b00b2880 0x7f41b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.594+0000 7f41c67fc700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41b00b2880 0x7f41b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.594+0000 7f41affff700 1 -- 172.21.15.112:0/4214232393 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f41bc0e3a90 con 0x7f41c8102400 2024-05-10T10:39:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.594+0000 7f41c67fc700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41b00b2880 0x7f41b00b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41c806cc50 tx=0x7f41b800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:21.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.597+0000 7f41affff700 1 -- 172.21.15.112:0/4214232393 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f41bc031070 con 0x7f41c8102400 2024-05-10T10:39:21.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.808+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f41c80638e0 con 0x7f41b00b2880 2024-05-10T10:39:21.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.822+0000 7f41affff700 1 -- 172.21.15.112:0/4214232393 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f41c80638e0 con 0x7f41b00b2880 2024-05-10T10:39:21.824 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-10T10:39:21.824 INFO:teuthology.orchestra.run.smithi112.stdout:mds.a smithi112 running (3m) 49s ago 3m 19.0M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 a70829d91c71 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.b smithi148 running (3m) 90s ago 3m 20.9M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 564e4250fa82 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.c smithi195 running (4m) 91s ago 4m 21.1M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 edd25aea5ac3 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.d smithi112 running (4m) 49s ago 4m 14.6M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 1f93a651e3f4 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.e smithi148 running (4m) 90s ago 4m 17.1M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 567a75ddddc0 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.f smithi195 running (4m) 91s ago 4m 19.9M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 3f98fe838b65 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.g smithi112 running (3m) 49s ago 3m 17.5M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 325f4ad18017 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.h smithi148 running (4m) 90s ago 4m 18.3M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 3c8d2f32cc7e 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.i smithi195 running (3m) 91s ago 3m 26.7M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 8ede9de1eb86 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.j smithi112 running (4m) 49s ago 4m 14.5M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 40dc4b06be95 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.k smithi148 running (4m) 90s ago 4m 16.5M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 3e3306709d8f 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mds.l smithi195 running (4m) 91s ago 4m 20.9M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 61f4e1feb903 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.x smithi112 *:9283,8765 running (11m) 49s ago 11m 534M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 5cb65258e897 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.y smithi148 *:8443,8765 running (9m) 90s ago 9m 467M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 6f205209571b 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.z smithi195 *:8443,8765 running (9m) 91s ago 9m 468M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 c839f810009d 2024-05-10T10:39:21.825 INFO:teuthology.orchestra.run.smithi112.stdout:mon.a smithi112 running (11m) 49s ago 11m 54.0M 2048M 19.0.0-3449-gd39fa088 ee5e4f21ef58 41fabb039a5b 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:mon.b smithi148 running (9m) 90s ago 9m 42.9M 2048M 19.0.0-3449-gd39fa088 ee5e4f21ef58 0b5202427c32 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:mon.c smithi195 running (10m) 91s ago 10m 40.9M 2048M 19.0.0-3449-gd39fa088 ee5e4f21ef58 249aec4d15a6 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 49s ago 9m 55.7M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 a0f16b7e93cc 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi148 running (8m) 90s ago 8m 56.9M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 ceeb370a15fa 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi195 running (8m) 91s ago 8m 46.8M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 f139b1b25450 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi112 running (8m) 49s ago 8m 52.4M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 c43bc1a2a31a 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi148 running (7m) 90s ago 7m 46.4M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 3fe9ad244e0f 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi195 running (7m) 91s ago 7m 48.0M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 6659666bc9a9 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 smithi112 running (6m) 49s ago 6m 50.7M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 43170962fdb4 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 smithi148 running (6m) 90s ago 6m 50.9M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 933ad6e6d139 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 smithi195 running (6m) 91s ago 6m 47.4M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 e50ad85efcca 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 smithi112 running (5m) 49s ago 5m 43.4M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 ae78d0dde9d2 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 smithi148 running (5m) 90s ago 5m 48.3M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 97ccf899a2e5 2024-05-10T10:39:21.826 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 smithi195 running (4m) 91s ago 4m 44.9M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 de7cf6e855b9 2024-05-10T10:39:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41b00b2880 msgr2=0x7f41b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41b00b2880 0x7f41b00b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41c806cc50 tx=0x7f41b800a300 comp rx=0 tx=0).stop 2024-05-10T10:39:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 msgr2=0x7f41c806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c806c6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0098e0 tx=0x7f41bc00f690 comp rx=0 tx=0).stop 2024-05-10T10:39:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 shutdown_connections 2024-05-10T10:39:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41b00b2880 0x7f41b00b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c8103fa0 0x7f41c81a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8102400 0x7f41c806c6f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 --2- 172.21.15.112:0/4214232393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41c8103600 0x7f41c819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.825+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 >> 172.21.15.112:0/4214232393 conn(0x7f41c80fd9c0 msgr2=0x7f41c8106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.826+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 shutdown_connections 2024-05-10T10:39:21.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:21.826+0000 7f41cd32e700 1 -- 172.21.15.112:0/4214232393 wait complete. 2024-05-10T10:39:22.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-10T10:39:22.886 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:23.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:23 smithi112 ceph-mon[30100]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:23.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:23 smithi112 ceph-mon[30100]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:23.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:23 smithi148 ceph-mon[33568]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:23.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:23 smithi148 ceph-mon[33568]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:23.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:23 smithi195 ceph-mon[33773]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:23.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:23 smithi195 ceph-mon[33773]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:24.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 -- 172.21.15.112:0/1464413666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 msgr2=0x7ffa94103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:24.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 --2- 172.21.15.112:0/1464413666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa94103920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa880099c0 tx=0x7ffa88009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:24.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 -- 172.21.15.112:0/1464413666 shutdown_connections 2024-05-10T10:39:24.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 --2- 172.21.15.112:0/1464413666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa941050a0 0x7ffa9410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:24.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 --2- 172.21.15.112:0/1464413666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa94103920 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:24.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 --2- 172.21.15.112:0/1464413666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa94104700 0x7ffa94104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:24.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 -- 172.21.15.112:0/1464413666 >> 172.21.15.112:0/1464413666 conn(0x7ffa940feaa0 msgr2=0x7ffa94100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:24.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 -- 172.21.15.112:0/1464413666 shutdown_connections 2024-05-10T10:39:24.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 -- 172.21.15.112:0/1464413666 wait complete. 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.783+0000 7ffa99439700 1 Processor -- start 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa99439700 1 -- start start 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa99439700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa940771f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa99439700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa94104700 0x7ffa94075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa99439700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa941050a0 0x7ffa94075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa99439700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9410ef20 con 0x7ffa94103500 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa99439700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9410eda0 con 0x7ffa94104700 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa99439700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9410f0a0 con 0x7ffa941050a0 2024-05-10T10:39:24.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa940771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa940771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38360/0 (socket says 172.21.15.112:38360) 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 -- 172.21.15.112:0/3464065580 learned_addr learned my addr 172.21.15.112:0/3464065580 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa937fe700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa941050a0 0x7ffa94075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa927fc700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa94104700 0x7ffa94075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 -- 172.21.15.112:0/3464065580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa941050a0 msgr2=0x7ffa94075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa941050a0 0x7ffa94075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 -- 172.21.15.112:0/3464065580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa94104700 msgr2=0x7ffa94075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa94104700 0x7ffa94075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 -- 172.21.15.112:0/3464065580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa940766a0 con 0x7ffa94103500 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa927fc700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa94104700 0x7ffa94075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.784+0000 7ffa92ffd700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa940771f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa880098e0 tx=0x7ffa8800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.785+0000 7ffa7bfff700 1 -- 172.21.15.112:0/3464065580 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88027070 con 0x7ffa94103500 2024-05-10T10:39:24.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.785+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa941a37a0 con 0x7ffa94103500 2024-05-10T10:39:24.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.785+0000 7ffa7bfff700 1 -- 172.21.15.112:0/3464065580 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa88022470 con 0x7ffa94103500 2024-05-10T10:39:24.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.785+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa941a3d00 con 0x7ffa94103500 2024-05-10T10:39:24.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.785+0000 7ffa7bfff700 1 -- 172.21.15.112:0/3464065580 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88003a50 con 0x7ffa94103500 2024-05-10T10:39:24.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.786+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9406a190 con 0x7ffa94103500 2024-05-10T10:39:24.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.788+0000 7ffa7bfff700 1 -- 172.21.15.112:0/3464065580 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ffa8802a070 con 0x7ffa94103500 2024-05-10T10:39:24.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.789+0000 7ffa7bfff700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa7c0b2880 0x7ffa7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:24.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.789+0000 7ffa927fc700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa7c0b2880 0x7ffa7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:24.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.789+0000 7ffa7bfff700 1 -- 172.21.15.112:0/3464065580 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ffa880e3b70 con 0x7ffa94103500 2024-05-10T10:39:24.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.789+0000 7ffa927fc700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa7c0b2880 0x7ffa7c0b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffa84007900 tx=0x7ffa84008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:24.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:24.795+0000 7ffa7bfff700 1 -- 172.21.15.112:0/3464065580 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffa88031070 con 0x7ffa94103500 2024-05-10T10:39:25.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.007+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ffa940638e0 con 0x7ffa7c0b2880 2024-05-10T10:39:25.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.013+0000 7ffa7bfff700 1 -- 172.21.15.112:0/3464065580 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ffa940638e0 con 0x7ffa7c0b2880 2024-05-10T10:39:25.014 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-10T10:39:25.014 INFO:teuthology.orchestra.run.smithi112.stdout:mds.all 12/12 94s ago 4m smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-10T10:39:25.015 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 3/3 94s ago 9m smithi112=x;smithi148=y;smithi195=z;count:3 2024-05-10T10:39:25.015 INFO:teuthology.orchestra.run.smithi112.stdout:mon 3/3 94s ago 10m smithi112:172.21.15.112=a;smithi148:172.21.15.148=b;smithi195:172.21.15.195=c;count:3 2024-05-10T10:39:25.015 INFO:teuthology.orchestra.run.smithi112.stdout:osd 12 94s ago - 2024-05-10T10:39:25.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa7c0b2880 msgr2=0x7ffa7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:25.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa7c0b2880 0x7ffa7c0b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffa84007900 tx=0x7ffa84008040 comp rx=0 tx=0).stop 2024-05-10T10:39:25.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 msgr2=0x7ffa940771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa940771f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa880098e0 tx=0x7ffa8800f740 comp rx=0 tx=0).stop 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 shutdown_connections 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffa7c0b2880 0x7ffa7c0b4d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa941050a0 0x7ffa94075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa94103500 0x7ffa940771f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 --2- 172.21.15.112:0/3464065580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa94104700 0x7ffa94075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 >> 172.21.15.112:0/3464065580 conn(0x7ffa940feaa0 msgr2=0x7ffa94107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 shutdown_connections 2024-05-10T10:39:25.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:25.015+0000 7ffa99439700 1 -- 172.21.15.112:0/3464065580 wait complete. 2024-05-10T10:39:25.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:25 smithi112 ceph-mon[30100]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:25.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:25 smithi148 ceph-mon[33568]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:25.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:25 smithi195 ceph-mon[33773]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:25.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-10T10:39:25.987 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:26.364 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:26 smithi112 ceph-mon[30100]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:26.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:26 smithi148 ceph-mon[33568]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:26.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:26 smithi195 ceph-mon[33773]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:27.169 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:27 smithi112 ceph-mon[30100]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:27.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:27 smithi148 ceph-mon[33568]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:27.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:27 smithi195 ceph-mon[33773]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 -- 172.21.15.112:0/1282885343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8103510 msgr2=0x7fadf8103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 --2- 172.21.15.112:0/1282885343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8103510 0x7fadf8103930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fade80099c0 tx=0x7fade8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 -- 172.21.15.112:0/1282885343 shutdown_connections 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 --2- 172.21.15.112:0/1282885343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadf81050b0 0x7fadf810b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 --2- 172.21.15.112:0/1282885343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8103510 0x7fadf8103930 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 --2- 172.21.15.112:0/1282885343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8104710 0x7fadf8104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 -- 172.21.15.112:0/1282885343 >> 172.21.15.112:0/1282885343 conn(0x7fadf80fea70 msgr2=0x7fadf8100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 -- 172.21.15.112:0/1282885343 shutdown_connections 2024-05-10T10:39:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.765+0000 7fadfca76700 1 -- 172.21.15.112:0/1282885343 wait complete. 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 Processor -- start 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 -- start start 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadf8103510 0x7fadf80771f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8104710 0x7fadf8075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf81050b0 0x7fadf8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810ee50 con 0x7fadf81050b0 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810ecd0 con 0x7fadf8103510 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadfca76700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810efd0 con 0x7fadf8104710 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadf6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf81050b0 0x7fadf8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:27.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadf6d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf81050b0 0x7fadf8075dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39300/0 (socket says 172.21.15.112:39300) 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadf6d9d700 1 -- 172.21.15.112:0/2676266140 learned_addr learned my addr 172.21.15.112:0/2676266140 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadf659c700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadf8103510 0x7fadf80771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.766+0000 7fadeffff700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8104710 0x7fadf8075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadf6d9d700 1 -- 172.21.15.112:0/2676266140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8104710 msgr2=0x7fadf8075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadf6d9d700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8104710 0x7fadf8075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadf6d9d700 1 -- 172.21.15.112:0/2676266140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadf8103510 msgr2=0x7fadf80771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadf6d9d700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadf8103510 0x7fadf80771f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadf6d9d700 1 -- 172.21.15.112:0/2676266140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf80766a0 con 0x7fadf81050b0 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadf6d9d700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf81050b0 0x7fadf8075dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fadf000ada0 tx=0x7fadf000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadef7fe700 1 -- 172.21.15.112:0/2676266140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0017020 con 0x7fadf81050b0 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadef7fe700 1 -- 172.21.15.112:0/2676266140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf000cd60 con 0x7fadf81050b0 2024-05-10T10:39:27.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf81a36f0 con 0x7fadf81050b0 2024-05-10T10:39:27.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadef7fe700 1 -- 172.21.15.112:0/2676266140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0005280 con 0x7fadf81050b0 2024-05-10T10:39:27.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.767+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81a3c50 con 0x7fadf81050b0 2024-05-10T10:39:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.768+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf806a190 con 0x7fadf81050b0 2024-05-10T10:39:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.770+0000 7fadef7fe700 1 -- 172.21.15.112:0/2676266140 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fadf000c8d0 con 0x7fadf81050b0 2024-05-10T10:39:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.771+0000 7fadef7fe700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fade40b2900 0x7fade40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.771+0000 7fadf659c700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fade40b2900 0x7fade40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.771+0000 7fadef7fe700 1 -- 172.21.15.112:0/2676266140 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fadf00e2250 con 0x7fadf81050b0 2024-05-10T10:39:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.772+0000 7fadf659c700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fade40b2900 0x7fade40b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fade8009990 tx=0x7fade8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:27.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.774+0000 7fadef7fe700 1 -- 172.21.15.112:0/2676266140 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fadf0021020 con 0x7fadf81050b0 2024-05-10T10:39:27.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.984+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fadf80638e0 con 0x7fade40b2900 2024-05-10T10:39:27.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.985+0000 7fadef7fe700 1 -- 172.21.15.112:0/2676266140 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fadf80638e0 con 0x7fade40b2900 2024-05-10T10:39:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-05-10T10:39:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-05-10T10:39:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:smithi148 172.21.15.148 2024-05-10T10:39:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:smithi195 172.21.15.195 2024-05-10T10:39:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:3 hosts in cluster 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.987+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fade40b2900 msgr2=0x7fade40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.987+0000 7fadfca76700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fade40b2900 0x7fade40b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fade8009990 tx=0x7fade8005fb0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.987+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf81050b0 msgr2=0x7fadf8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.987+0000 7fadfca76700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf81050b0 0x7fadf8075dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fadf000ada0 tx=0x7fadf000c330 comp rx=0 tx=0).stop 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 shutdown_connections 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf81050b0 0x7fadf8075dd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadf8103510 0x7fadf80771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fade40b2900 0x7fade40b4dc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 --2- 172.21.15.112:0/2676266140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadf8104710 0x7fadf8075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:27.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 >> 172.21.15.112:0/2676266140 conn(0x7fadf80fea70 msgr2=0x7fadf8107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:27.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 shutdown_connections 2024-05-10T10:39:27.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:27.988+0000 7fadfca76700 1 -- 172.21.15.112:0/2676266140 wait complete. 2024-05-10T10:39:28.725 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-10T10:39:28.968 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:29.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:29 smithi112 ceph-mon[30100]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:29.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:29 smithi112 ceph-mon[30100]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:29 smithi148 ceph-mon[33568]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:29 smithi148 ceph-mon[33568]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:29.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:29 smithi195 ceph-mon[33773]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:29.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:29 smithi195 ceph-mon[33773]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 -- 172.21.15.112:0/3946736469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103fa0 msgr2=0x7f46fc10a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 --2- 172.21.15.112:0/3946736469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103fa0 0x7f46fc10a850 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b0d0 tx=0x7f46f400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 -- 172.21.15.112:0/3946736469 shutdown_connections 2024-05-10T10:39:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 --2- 172.21.15.112:0/3946736469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103fa0 0x7f46fc10a850 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 --2- 172.21.15.112:0/3946736469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46fc102400 0x7f46fc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 --2- 172.21.15.112:0/3946736469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46fc103600 0x7f46fc103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 -- 172.21.15.112:0/3946736469 >> 172.21.15.112:0/3946736469 conn(0x7f46fc0fd9a0 msgr2=0x7f46fc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 -- 172.21.15.112:0/3946736469 shutdown_connections 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.702+0000 7f4700914700 1 -- 172.21.15.112:0/3946736469 wait complete. 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 Processor -- start 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 -- start start 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46fc102400 0x7f46fc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103600 0x7f46fc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46fc103fa0 0x7f46fc19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10de40 con 0x7f46fc103600 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10dcc0 con 0x7f46fc102400 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f4700914700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10dfc0 con 0x7f46fc103fa0 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f46f9d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103600 0x7f46fc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:30.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f46f9d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103600 0x7f46fc198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39320/0 (socket says 172.21.15.112:39320) 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f46f9d9b700 1 -- 172.21.15.112:0/3648914583 learned_addr learned my addr 172.21.15.112:0/3648914583 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.703+0000 7f46fad9d700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46fc103fa0 0x7f46fc19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46fa59c700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46fc102400 0x7f46fc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46f9d9b700 1 -- 172.21.15.112:0/3648914583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46fc103fa0 msgr2=0x7f46fc19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46f9d9b700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46fc103fa0 0x7f46fc19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46f9d9b700 1 -- 172.21.15.112:0/3648914583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46fc102400 msgr2=0x7f46fc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46f9d9b700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46fc102400 0x7f46fc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46f9d9b700 1 -- 172.21.15.112:0/3648914583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc1a0230 con 0x7f46fc103600 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46fa59c700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46fc102400 0x7f46fc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46f9d9b700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103600 0x7f46fc198b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f46f000b560 tx=0x7f46f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46eb7fe700 1 -- 172.21.15.112:0/3648914583 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0015020 con 0x7f46fc103600 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46eb7fe700 1 -- 172.21.15.112:0/3648914583 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46f0011cc0 con 0x7f46fc103600 2024-05-10T10:39:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc1a0520 con 0x7f46fc103600 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f46eb7fe700 1 -- 172.21.15.112:0/3648914583 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0010d30 con 0x7f46fc103600 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.704+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc1a0ae0 con 0x7f46fc103600 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.705+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc06a190 con 0x7f46fc103600 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.706+0000 7f46eb7fe700 1 -- 172.21.15.112:0/3648914583 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f46f0018020 con 0x7f46fc103600 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.707+0000 7f46eb7fe700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46e40b2880 0x7f46e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.707+0000 7f46fa59c700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46e40b2880 0x7f46e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.707+0000 7f46eb7fe700 1 -- 172.21.15.112:0/3648914583 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f46f00e1d20 con 0x7f46fc103600 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.708+0000 7f46fa59c700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46e40b2880 0x7f46e40b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46fc103460 tx=0x7f46ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.713+0000 7f46eb7fe700 1 -- 172.21.15.112:0/3648914583 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f46f001f020 con 0x7f46fc103600 2024-05-10T10:39:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.924+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f46fc0638e0 con 0x7f46e40b2880 2024-05-10T10:39:30.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.925+0000 7f46eb7fe700 1 -- 172.21.15.112:0/3648914583 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f46fc0638e0 con 0x7f46e40b2880 2024-05-10T10:39:30.927 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-10T10:39:30.927 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-10T10:39:30.927 INFO:teuthology.orchestra.run.smithi112.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-10T10:39:30.927 INFO:teuthology.orchestra.run.smithi112.stdout:smithi195 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300FS480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46e40b2880 msgr2=0x7f46e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46e40b2880 0x7f46e40b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46fc103460 tx=0x7f46ec008040 comp rx=0 tx=0).stop 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103600 msgr2=0x7f46fc198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103600 0x7f46fc198b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f46f000b560 tx=0x7f46f000b870 comp rx=0 tx=0).stop 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 shutdown_connections 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46e40b2880 0x7f46e40b4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46fc103fa0 0x7f46fc19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46fc102400 0x7f46fc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 --2- 172.21.15.112:0/3648914583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46fc103600 0x7f46fc198b10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 >> 172.21.15.112:0/3648914583 conn(0x7f46fc0fd9a0 msgr2=0x7f46fc106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 shutdown_connections 2024-05-10T10:39:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:30.928+0000 7f4700914700 1 -- 172.21.15.112:0/3648914583 wait complete. 2024-05-10T10:39:31.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:31 smithi148 ceph-mon[33568]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:31.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:31 smithi195 ceph-mon[33773]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:31 smithi112 ceph-mon[30100]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:31.684 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T10:39:31.694 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:39:31.694 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-10T10:39:31.937 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:32.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:32 smithi112 ceph-mon[30100]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:32.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:32 smithi148 ceph-mon[33568]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:32.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:32 smithi195 ceph-mon[33773]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T10:39:33.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:33 smithi112 ceph-mon[30100]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:33.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:33 smithi148 ceph-mon[33568]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:33 smithi195 ceph-mon[33773]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 -- 172.21.15.112:0/4227110824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 msgr2=0x7f8fdc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/4227110824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc102820 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00b0d0 tx=0x7f8fcc00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 -- 172.21.15.112:0/4227110824 shutdown_connections 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/4227110824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fdc103fa0 0x7f8fdc10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/4227110824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc102820 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/4227110824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc103600 0x7f8fdc103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 -- 172.21.15.112:0/4227110824 >> 172.21.15.112:0/4227110824 conn(0x7f8fdc0fd9c0 msgr2=0x7f8fdc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 -- 172.21.15.112:0/4227110824 shutdown_connections 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 -- 172.21.15.112:0/4227110824 wait complete. 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 Processor -- start 2024-05-10T10:39:33.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.630+0000 7f8fe13fb700 1 -- start start 2024-05-10T10:39:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc06c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fdc103600 0x7f8fdc19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc103fa0 0x7f8fdc1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc10dd20 con 0x7f8fdc102400 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc10dba0 con 0x7f8fdc103fa0 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc10dea0 con 0x7f8fdc103600 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc06c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39336/0 (socket says 172.21.15.112:39336) 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 -- 172.21.15.112:0/1731922619 learned_addr learned my addr 172.21.15.112:0/1731922619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fda7fc700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fdc103600 0x7f8fdc19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdb7fe700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc103fa0 0x7f8fdc1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 -- 172.21.15.112:0/1731922619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fdc103600 msgr2=0x7f8fdc19e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fdc103600 0x7f8fdc19e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 -- 172.21.15.112:0/1731922619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc103fa0 msgr2=0x7f8fdc1a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc103fa0 0x7f8fdc1a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 -- 172.21.15.112:0/1731922619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc1a11b0 con 0x7f8fdc102400 2024-05-10T10:39:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdb7fe700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc103fa0 0x7f8fdc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fdaffd700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc06c770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc004fd0 tx=0x7f8fcc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fd3fff700 1 -- 172.21.15.112:0/1731922619 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc01f030 con 0x7f8fdc102400 2024-05-10T10:39:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc1a1440 con 0x7f8fdc102400 2024-05-10T10:39:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.631+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc1a18f0 con 0x7f8fdc102400 2024-05-10T10:39:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.632+0000 7f8fd3fff700 1 -- 172.21.15.112:0/1731922619 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fcc007bc0 con 0x7f8fdc102400 2024-05-10T10:39:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.632+0000 7f8fd3fff700 1 -- 172.21.15.112:0/1731922619 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0074e0 con 0x7f8fdc102400 2024-05-10T10:39:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.632+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc005360 con 0x7f8fdc102400 2024-05-10T10:39:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.634+0000 7f8fd3fff700 1 -- 172.21.15.112:0/1731922619 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8fcc022030 con 0x7f8fdc102400 2024-05-10T10:39:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.634+0000 7f8fd3fff700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8fc80b2900 0x7f8fc80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.635+0000 7f8fda7fc700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8fc80b2900 0x7f8fc80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.635+0000 7f8fd3fff700 1 -- 172.21.15.112:0/1731922619 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8fcc0e3140 con 0x7f8fdc102400 2024-05-10T10:39:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.635+0000 7f8fda7fc700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8fc80b2900 0x7f8fc80b4dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005ea0 tx=0x7f8fc4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.637+0000 7f8fd3fff700 1 -- 172.21.15.112:0/1731922619 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8fcc029030 con 0x7f8fdc102400 2024-05-10T10:39:33.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.884+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8fbc005920 con 0x7f8fdc102400 2024-05-10T10:39:33.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.884+0000 7f8fd3fff700 1 -- 172.21.15.112:0/1731922619 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3106 (secure 0 0 0) 0x7f8fcc0aab60 con 0x7f8fdc102400 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:e29 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 29 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-05-10T10:35:52.903233+0000 2024-05-10T10:39:33.886 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-05-10T10:37:03.364184+0000 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 300 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 5 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1,2,3,4 2024-05-10T10:39:33.887 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=24469,1=24451,2=24446,3=24445,4=24428} 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3,4] 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 24469 members: 24428,24445,24446,24451,24469 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.i{0:24469} state up:active seq 9 addr [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.l{0:24415} state up:standby-replay seq 1 addr [v2:172.21.15.195:6832/38979975,v1:172.21.15.195:6833/38979975] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c{1:24451} state up:active seq 26 addr [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.888 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e{1:24404} state up:standby-replay seq 1 addr [v2:172.21.15.148:6834/2643629917,v1:172.21.15.148:6835/2643629917] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b{2:24446} state up:active seq 22 addr [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.k{2:24389} state up:standby-replay seq 1 addr [v2:172.21.15.148:6832/3726517324,v1:172.21.15.148:6833/3726517324] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f{3:24445} state up:active seq 27 addr [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a{3:14604} state up:standby-replay seq 1 addr [v2:172.21.15.112:6840/1213307484,v1:172.21.15.112:6841/1213307484] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.h{4:24428} state up:active seq 26 addr [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.g{4:14592} state up:standby-replay seq 1 addr [v2:172.21.15.112:6838/2952588053,v1:172.21.15.112:6839/2952588053] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.112:6834/441557684,v1:172.21.15.112:6835/441557684] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.112:6836/2497530637,v1:172.21.15.112:6837/2497530637] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T10:39:33.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8fc80b2900 msgr2=0x7f8fc80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:33.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8fc80b2900 0x7f8fc80b4dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005ea0 tx=0x7f8fc4005e30 comp rx=0 tx=0).stop 2024-05-10T10:39:33.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 msgr2=0x7f8fdc06c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:33.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc06c770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc004fd0 tx=0x7f8fcc00bd60 comp rx=0 tx=0).stop 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 shutdown_connections 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc103fa0 0x7f8fdc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc102400 0x7f8fdc06c770 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8fc80b2900 0x7f8fc80b4dc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 --2- 172.21.15.112:0/1731922619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fdc103600 0x7f8fdc19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.887+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 >> 172.21.15.112:0/1731922619 conn(0x7f8fdc0fd9c0 msgr2=0x7f8fdc106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.888+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 shutdown_connections 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:33.888+0000 7f8fe13fb700 1 -- 172.21.15.112:0/1731922619 wait complete. 2024-05-10T10:39:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-10T10:39:34.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:34 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1731922619' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T10:39:34.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:34 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1731922619' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T10:39:34.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-10T10:39:34.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:34 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1731922619' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T10:39:34.807 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:35.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:35 smithi112 ceph-mon[30100]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:35.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:35 smithi148 ceph-mon[33568]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:35 smithi195 ceph-mon[33773]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:36.303 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:36 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:39:36.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:36 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:39:36.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:36 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:39:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 -- 172.21.15.112:0/4020992702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb0103f90 msgr2=0x7f8bb010a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 --2- 172.21.15.112:0/4020992702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb0103f90 0x7f8bb010a840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b980099c0 tx=0x7f8b98009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 -- 172.21.15.112:0/4020992702 shutdown_connections 2024-05-10T10:39:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 --2- 172.21.15.112:0/4020992702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb0103f90 0x7f8bb010a840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 --2- 172.21.15.112:0/4020992702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bb01023f0 0x7f8bb0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 --2- 172.21.15.112:0/4020992702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb01035f0 0x7f8bb0103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 -- 172.21.15.112:0/4020992702 >> 172.21.15.112:0/4020992702 conn(0x7f8bb00fd990 msgr2=0x7f8bb00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 -- 172.21.15.112:0/4020992702 shutdown_connections 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.557+0000 7f8bb7039700 1 -- 172.21.15.112:0/4020992702 wait complete. 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 Processor -- start 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 -- start start 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb01023f0 0x7f8bb0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bb01035f0 0x7f8bb0199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb0103f90 0x7f8bb01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb010dd10 con 0x7f8bb01023f0 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb010db90 con 0x7f8bb0103f90 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb7039700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb010de90 con 0x7f8bb01035f0 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb01023f0 0x7f8bb0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb01023f0 0x7f8bb0199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44650/0 (socket says 172.21.15.112:44650) 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 -- 172.21.15.112:0/201315116 learned_addr learned my addr 172.21.15.112:0/201315116 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:36.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb55d6700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb0103f90 0x7f8bb01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8baffff700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bb01035f0 0x7f8bb0199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 -- 172.21.15.112:0/201315116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bb01035f0 msgr2=0x7f8bb0199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bb01035f0 0x7f8bb0199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 -- 172.21.15.112:0/201315116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb0103f90 msgr2=0x7f8bb01a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb0103f90 0x7f8bb01a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.558+0000 7f8bb4dd5700 1 -- 172.21.15.112:0/201315116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb01a1180 con 0x7f8bb01023f0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.559+0000 7f8bb4dd5700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb01023f0 0x7f8bb0199520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800d9b0 tx=0x7f8ba800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.559+0000 7f8badffb700 1 -- 172.21.15.112:0/201315116 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8016070 con 0x7f8bb01023f0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.559+0000 7f8badffb700 1 -- 172.21.15.112:0/201315116 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba800bde0 con 0x7f8bb01023f0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.559+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb01a1470 con 0x7f8bb01023f0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.559+0000 7f8badffb700 1 -- 172.21.15.112:0/201315116 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8010d70 con 0x7f8bb01023f0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.559+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01a1a00 con 0x7f8bb01023f0 2024-05-10T10:39:36.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.560+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb006a190 con 0x7f8bb01023f0 2024-05-10T10:39:36.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.561+0000 7f8badffb700 1 -- 172.21.15.112:0/201315116 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8ba8019070 con 0x7f8bb01023f0 2024-05-10T10:39:36.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.562+0000 7f8badffb700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b9c0b2900 0x7f8b9c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:36.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.562+0000 7f8baffff700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b9c0b2900 0x7f8b9c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:36.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.562+0000 7f8badffb700 1 -- 172.21.15.112:0/201315116 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8ba80e2360 con 0x7f8bb01023f0 2024-05-10T10:39:36.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.562+0000 7f8baffff700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b9c0b2900 0x7f8b9c0b4dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009550 tx=0x7f8ba00091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:36.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.566+0000 7f8badffb700 1 -- 172.21.15.112:0/201315116 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8ba8020020 con 0x7f8bb01023f0 2024-05-10T10:39:36.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.783+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f8bb01a23f0 con 0x7f8bb01023f0 2024-05-10T10:39:36.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.784+0000 7f8badffb700 1 -- 172.21.15.112:0/201315116 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5675 (secure 0 0 0) 0x7f8ba80a9cf0 con 0x7f8bb01023f0 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 90 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-05-10T10:27:34.800106+0000 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-05-10T10:37:35.554118+0000 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:crush_version 27 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:full_ratio 0.95 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:backfillfull_ratio 0.9 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:nearfull_ratio 0.85 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:require_min_compat_client luminous 2024-05-10T10:39:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:min_compat_client luminous 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:require_osd_release squid 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:stretch_mode_enabled false 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:max_osd 12 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6802/221387326,v1:172.21.15.112:6803/221387326] [v2:172.21.15.112:6804/221387326,v1:172.21.15.112:6805/221387326] exists,up a67e2c4c-f481-4fd7-b502-3d561eb6ffc6 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/3233597314,v1:172.21.15.148:6801/3233597314] [v2:172.21.15.148:6802/3233597314,v1:172.21.15.148:6803/3233597314] exists,up de68637a-80f9-450c-9208-784f54dc3df2 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/693285528,v1:172.21.15.195:6801/693285528] [v2:172.21.15.195:6802/693285528,v1:172.21.15.195:6803/693285528] exists,up 951b8444-0377-457c-b0c4-916113883584 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6810/1175679850,v1:172.21.15.112:6811/1175679850] [v2:172.21.15.112:6812/1175679850,v1:172.21.15.112:6813/1175679850] exists,up 6c27d781-3ab0-4739-836d-c4016bb1e0a3 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 up in weight 1 up_from 31 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/455926414,v1:172.21.15.148:6809/455926414] [v2:172.21.15.148:6810/455926414,v1:172.21.15.148:6811/455926414] exists,up 4a8781c4-9d04-449c-a8df-45568364e555 2024-05-10T10:39:36.787 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 up in weight 1 up_from 36 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6808/465167884,v1:172.21.15.195:6809/465167884] [v2:172.21.15.195:6810/465167884,v1:172.21.15.195:6811/465167884] exists,up c65aefb7-ce11-4194-b1f4-ab1e323102cb 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 up in weight 1 up_from 42 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6818/979146261,v1:172.21.15.112:6819/979146261] [v2:172.21.15.112:6820/979146261,v1:172.21.15.112:6821/979146261] exists,up 333835c8-bba4-4777-b7f9-ed8e6bb54ec0 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/3546374134,v1:172.21.15.148:6817/3546374134] [v2:172.21.15.148:6818/3546374134,v1:172.21.15.148:6819/3546374134] exists,up 57dc7935-dd5e-47f4-87e0-8aae25736340 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 up in weight 1 up_from 54 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6816/3432850695,v1:172.21.15.195:6817/3432850695] [v2:172.21.15.195:6818/3432850695,v1:172.21.15.195:6819/3432850695] exists,up e756afdf-f8df-4bda-ad15-a943ba0527d6 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 up in weight 1 up_from 60 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6826/2726539509,v1:172.21.15.112:6827/2726539509] [v2:172.21.15.112:6828/2726539509,v1:172.21.15.112:6829/2726539509] exists,up 1111b3c0-269c-451e-8e48-bc2ae180960c 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 up in weight 1 up_from 65 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/2277563117,v1:172.21.15.148:6825/2277563117] [v2:172.21.15.148:6826/2277563117,v1:172.21.15.148:6827/2277563117] exists,up 841869b8-c760-4899-9e09-33c930cb81da 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 up in weight 1 up_from 71 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6824/1653173109,v1:172.21.15.195:6825/1653173109] [v2:172.21.15.195:6826/1653173109,v1:172.21.15.195:6827/1653173109] exists,up e59ef38d-f7e9-4721-9779-75eb13dfddcc 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.6 [8,11] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.a [2,11] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.14 [2,4] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.2 [3,9] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.1b [4,8] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.25 [5,9] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.31 [4,6] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.34 [4,9] 2024-05-10T10:39:36.788 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.11 [5,8] 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.14 [9,8] 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.17 [9,8] 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.1c [9,10] 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3420175550 expires 2024-05-11T10:28:33.178665+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/2836147288 expires 2024-05-11T10:28:33.178665+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/2112593785 expires 2024-05-11T10:28:33.178665+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/2836147288 expires 2024-05-11T10:28:33.178665+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1230086877 expires 2024-05-11T10:28:06.482690+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/1796554484 expires 2024-05-11T10:28:06.482690+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3235282798 expires 2024-05-11T10:28:06.482690+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/2711978475 expires 2024-05-11T10:28:06.482690+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/1796554484 expires 2024-05-11T10:28:06.482690+0000 2024-05-10T10:39:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3592023612 expires 2024-05-11T10:28:33.178665+0000 2024-05-10T10:39:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b9c0b2900 msgr2=0x7f8b9c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b9c0b2900 0x7f8b9c0b4dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009550 tx=0x7f8ba00091b0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb01023f0 msgr2=0x7f8bb0199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb01023f0 0x7f8bb0199520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800d9b0 tx=0x7f8ba800dcc0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 shutdown_connections 2024-05-10T10:39:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bb0103f90 0x7f8bb01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bb01023f0 0x7f8bb0199520 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f8b9c0b2900 0x7f8b9c0b4dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 --2- 172.21.15.112:0/201315116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bb01035f0 0x7f8bb0199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 >> 172.21.15.112:0/201315116 conn(0x7f8bb00fd990 msgr2=0x7f8bb00ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 shutdown_connections 2024-05-10T10:39:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:36.788+0000 7f8bb7039700 1 -- 172.21.15.112:0/201315116 wait complete. 2024-05-10T10:39:37.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:37.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:37 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/201315116' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:37 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/201315116' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:37 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/201315116' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T10:39:37.611 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-10T10:39:37.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:39:37.866 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:38.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:39:38.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:39:38.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:38.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:39:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:39:38.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:39:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:39:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:39:39.298 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:39 smithi112 ceph-mon[30100]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 -- 172.21.15.112:0/3568161083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174103750 msgr2=0x7fc174103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 --2- 172.21.15.112:0/3568161083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174103750 0x7fc174103b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc1640099c0 tx=0x7fc164009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 -- 172.21.15.112:0/3568161083 shutdown_connections 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 --2- 172.21.15.112:0/3568161083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc174108850 0x7fc1741103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 --2- 172.21.15.112:0/3568161083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174103750 0x7fc174103b30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 --2- 172.21.15.112:0/3568161083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc174104100 0x7fc174108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 -- 172.21.15.112:0/3568161083 >> 172.21.15.112:0/3568161083 conn(0x7fc17407a600 msgr2=0x7fc17407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 -- 172.21.15.112:0/3568161083 shutdown_connections 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.533+0000 7fc178df0700 1 -- 172.21.15.112:0/3568161083 wait complete. 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 Processor -- start 2024-05-10T10:39:39.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 -- start start 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc174103750 0x7fc17419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174104100 0x7fc17419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc174108850 0x7fc1741998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741134f0 con 0x7fc174104100 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc174113370 con 0x7fc174103750 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc178df0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc174113670 con 0x7fc174108850 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174104100 0x7fc17419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174104100 0x7fc17419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44664/0 (socket says 172.21.15.112:44664) 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 -- 172.21.15.112:0/3118866001 learned_addr learned my addr 172.21.15.112:0/3118866001 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc17259c700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc174103750 0x7fc17419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc172d9d700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc174108850 0x7fc1741998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 -- 172.21.15.112:0/3118866001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc174108850 msgr2=0x7fc1741998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:39.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc174108850 0x7fc1741998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 -- 172.21.15.112:0/3118866001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc174103750 msgr2=0x7fc17419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc174103750 0x7fc17419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc171d9b700 1 -- 172.21.15.112:0/3118866001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17419a010 con 0x7fc174104100 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.534+0000 7fc17259c700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc174103750 0x7fc17419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.535+0000 7fc171d9b700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174104100 0x7fc17419fc30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c970 tx=0x7fc16800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.535+0000 7fc1637fe700 1 -- 172.21.15.112:0/3118866001 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168007aa0 con 0x7fc174104100 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.535+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17419a300 con 0x7fc174104100 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.535+0000 7fc1637fe700 1 -- 172.21.15.112:0/3118866001 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc168004cc0 con 0x7fc174104100 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.535+0000 7fc1637fe700 1 -- 172.21.15.112:0/3118866001 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168005500 con 0x7fc174104100 2024-05-10T10:39:39.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.535+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741a65c0 con 0x7fc174104100 2024-05-10T10:39:39.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.536+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17410ba90 con 0x7fc174104100 2024-05-10T10:39:39.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.537+0000 7fc1637fe700 1 -- 172.21.15.112:0/3118866001 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc168007c40 con 0x7fc174104100 2024-05-10T10:39:39.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.538+0000 7fc1637fe700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc15c0b2880 0x7fc15c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:39.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.538+0000 7fc17259c700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc15c0b2880 0x7fc15c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:39.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.538+0000 7fc1637fe700 1 -- 172.21.15.112:0/3118866001 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc1680e2430 con 0x7fc174104100 2024-05-10T10:39:39.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.538+0000 7fc17259c700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc15c0b2880 0x7fc15c0b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc164009990 tx=0x7fc16401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:39.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.542+0000 7fc1637fe700 1 -- 172.21.15.112:0/3118866001 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc1680a9e50 con 0x7fc174104100 2024-05-10T10:39:39.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:39 smithi148 ceph-mon[33568]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:39.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:39 smithi195 ceph-mon[33773]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:39.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.797+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc17419ade0 con 0x7fc174104100 2024-05-10T10:39:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.798+0000 7fc1637fe700 1 -- 172.21.15.112:0/3118866001 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7fc1680a9750 con 0x7fc174104100 2024-05-10T10:39:39.800 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:39.800 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:37:03.364184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24428,24445,24446,24451,24469]},"id":1}]} 2024-05-10T10:39:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc15c0b2880 msgr2=0x7fc15c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc15c0b2880 0x7fc15c0b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc164009990 tx=0x7fc16401a040 comp rx=0 tx=0).stop 2024-05-10T10:39:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174104100 msgr2=0x7fc17419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174104100 0x7fc17419fc30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c970 tx=0x7fc16800cd30 comp rx=0 tx=0).stop 2024-05-10T10:39:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 shutdown_connections 2024-05-10T10:39:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc15c0b2880 0x7fc15c0b4d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc174108850 0x7fc1741998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc174103750 0x7fc17419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 --2- 172.21.15.112:0/3118866001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc174104100 0x7fc17419fc30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 >> 172.21.15.112:0/3118866001 conn(0x7fc17407a600 msgr2=0x7fc174101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 shutdown_connections 2024-05-10T10:39:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:39.800+0000 7fc178df0700 1 -- 172.21.15.112:0/3118866001 wait complete. 2024-05-10T10:39:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-10T10:39:40.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:39:40.752 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:40.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:40 smithi112 ceph-mon[30100]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:40.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:40 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3118866001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:41.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:40 smithi148 ceph-mon[33568]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:41.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:40 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3118866001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:40 smithi195 ceph-mon[33773]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:40 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3118866001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 -- 172.21.15.112:0/3153410410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc102540 msgr2=0x7f13dc10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3153410410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc102540 0x7f13dc10e9b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f13d400b0d0 tx=0x7f13d400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 -- 172.21.15.112:0/3153410410 shutdown_connections 2024-05-10T10:39:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3153410410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc102540 0x7f13dc10e9b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3153410410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc107be0 0x7f13dc107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3153410410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13dc101b80 0x7f13dc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 -- 172.21.15.112:0/3153410410 >> 172.21.15.112:0/3153410410 conn(0x7f13dc078a10 msgr2=0x7f13dc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 -- 172.21.15.112:0/3153410410 shutdown_connections 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.450+0000 7f13e0d38700 1 -- 172.21.15.112:0/3153410410 wait complete. 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 Processor -- start 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 -- start start 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13dc101b80 0x7f13dc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc102540 0x7f13dc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc107be0 0x7f13dc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc111c10 con 0x7f13dc107be0 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc111a90 con 0x7f13dc102540 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13e0d38700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc111d90 con 0x7f13dc101b80 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc107be0 0x7f13dc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc107be0 0x7f13dc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44680/0 (socket says 172.21.15.112:44680) 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 -- 172.21.15.112:0/3900644755 learned_addr learned my addr 172.21.15.112:0/3900644755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13d3fff700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc102540 0x7f13dc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 -- 172.21.15.112:0/3900644755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13dc101b80 msgr2=0x7f13dc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13dc101b80 0x7f13dc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 -- 172.21.15.112:0/3900644755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc102540 msgr2=0x7f13dc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc102540 0x7f13dc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.451+0000 7f13dad9d700 1 -- 172.21.15.112:0/3900644755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc199ff0 con 0x7f13dc107be0 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.452+0000 7f13dad9d700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc107be0 0x7f13dc199890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13d4007d90 tx=0x7f13d4007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:42.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.452+0000 7f13d37fe700 1 -- 172.21.15.112:0/3900644755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d401f030 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.452+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc19a280 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.452+0000 7f13d37fe700 1 -- 172.21.15.112:0/3900644755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13d401bb90 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.452+0000 7f13d37fe700 1 -- 172.21.15.112:0/3900644755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4012c90 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.452+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc1a65a0 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.453+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13dc06a190 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.454+0000 7f13d37fe700 1 -- 172.21.15.112:0/3900644755 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f13d4022030 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.455+0000 7f13d37fe700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f13c80b2880 0x7f13c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.455+0000 7f13da59c700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f13c80b2880 0x7f13c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.455+0000 7f13d37fe700 1 -- 172.21.15.112:0/3900644755 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f13d400f020 con 0x7f13dc107be0 2024-05-10T10:39:42.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.455+0000 7f13da59c700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f13c80b2880 0x7f13c80b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13dc19a9d0 tx=0x7f13cc006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:42.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.458+0000 7f13d37fe700 1 -- 172.21.15.112:0/3900644755 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f13d4029020 con 0x7f13dc107be0 2024-05-10T10:39:42.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.708+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13dc19adc0 con 0x7f13dc107be0 2024-05-10T10:39:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.708+0000 7f13d37fe700 1 -- 172.21.15.112:0/3900644755 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7f13dc19adc0 con 0x7f13dc107be0 2024-05-10T10:39:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:37:03.364184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24428,24445,24446,24451,24469]},"id":1}]} 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f13c80b2880 msgr2=0x7f13c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f13c80b2880 0x7f13c80b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13dc19a9d0 tx=0x7f13cc006d20 comp rx=0 tx=0).stop 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc107be0 msgr2=0x7f13dc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc107be0 0x7f13dc199890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13d4007d90 tx=0x7f13d4007e70 comp rx=0 tx=0).stop 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 shutdown_connections 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13dc107be0 0x7f13dc199890 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f13c80b2880 0x7f13c80b4d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13dc101b80 0x7f13dc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 --2- 172.21.15.112:0/3900644755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc102540 0x7f13dc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:42.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 >> 172.21.15.112:0/3900644755 conn(0x7f13dc078a10 msgr2=0x7f13dc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:42.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 shutdown_connections 2024-05-10T10:39:42.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:42.711+0000 7f13e0d38700 1 -- 172.21.15.112:0/3900644755 wait complete. 2024-05-10T10:39:42.713 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-10T10:39:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:42 smithi112 ceph-mon[30100]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:43.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:42 smithi148 ceph-mon[33568]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:42 smithi195 ceph-mon[33773]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:43.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:39:43.791 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:43.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:43 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3900644755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:43.967 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:43 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3900644755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:44.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:43 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3900644755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:39:44.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:44 smithi112 ceph-mon[30100]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:45.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:44 smithi148 ceph-mon[33568]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:45.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:44 smithi195 ceph-mon[33773]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 -- 172.21.15.112:0/93924573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c102630 msgr2=0x7f2f3c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 --2- 172.21.15.112:0/93924573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c102630 0x7f2f3c102a10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2f380099c0 tx=0x7f2f38009cd0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 -- 172.21.15.112:0/93924573 shutdown_connections 2024-05-10T10:39:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 --2- 172.21.15.112:0/93924573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f3c1076c0 0x7f2f3c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 --2- 172.21.15.112:0/93924573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c102630 0x7f2f3c102a10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 --2- 172.21.15.112:0/93924573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f3c102fe0 0x7f2f3c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 -- 172.21.15.112:0/93924573 >> 172.21.15.112:0/93924573 conn(0x7f2f3c079350 msgr2=0x7f2f3c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 -- 172.21.15.112:0/93924573 shutdown_connections 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.487+0000 7f2f42e24700 1 -- 172.21.15.112:0/93924573 wait complete. 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 Processor -- start 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 -- start start 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f3c102630 0x7f2f3c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f3c102fe0 0x7f2f3c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c1076c0 0x7f2f3c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c112420 con 0x7f2f3c1076c0 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1122a0 con 0x7f2f3c102630 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f42e24700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1125a0 con 0x7f2f3c102fe0 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c1076c0 0x7f2f3c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c1076c0 0x7f2f3c199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44700/0 (socket says 172.21.15.112:44700) 2024-05-10T10:39:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 -- 172.21.15.112:0/951867879 learned_addr learned my addr 172.21.15.112:0/951867879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f33fff700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f3c102fe0 0x7f2f3c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 -- 172.21.15.112:0/951867879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f3c102fe0 msgr2=0x7f2f3c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f3c102fe0 0x7f2f3c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 -- 172.21.15.112:0/951867879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f3c102630 msgr2=0x7f2f3c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f40bc0700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f3c102630 0x7f2f3c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f3c102630 0x7f2f3c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f413c1700 1 -- 172.21.15.112:0/951867879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c19a0b0 con 0x7f2f3c1076c0 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f33fff700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f3c102fe0 0x7f2f3c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.488+0000 7f2f40bc0700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f3c102630 0x7f2f3c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.489+0000 7f2f413c1700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c1076c0 0x7f2f3c199950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400ada0 tx=0x7f2f3400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.489+0000 7f2f31ffb700 1 -- 172.21.15.112:0/951867879 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34017020 con 0x7f2f3c1076c0 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.489+0000 7f2f31ffb700 1 -- 172.21.15.112:0/951867879 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f3400ce10 con 0x7f2f3c1076c0 2024-05-10T10:39:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.489+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c19a3a0 con 0x7f2f3c1076c0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.489+0000 7f2f31ffb700 1 -- 172.21.15.112:0/951867879 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34005440 con 0x7f2f3c1076c0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.489+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c1a6660 con 0x7f2f3c1076c0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.490+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3c06a190 con 0x7f2f3c1076c0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.491+0000 7f2f31ffb700 1 -- 172.21.15.112:0/951867879 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2f34004ad0 con 0x7f2f3c1076c0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.492+0000 7f2f31ffb700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2f240b2880 0x7f2f240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.492+0000 7f2f40bc0700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2f240b2880 0x7f2f240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.492+0000 7f2f31ffb700 1 -- 172.21.15.112:0/951867879 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2f340e22e0 con 0x7f2f3c1076c0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.492+0000 7f2f40bc0700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2f240b2880 0x7f2f240b4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009fd0 tx=0x7f2f38005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.495+0000 7f2f31ffb700 1 -- 172.21.15.112:0/951867879 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2f34021020 con 0x7f2f3c1076c0 2024-05-10T10:39:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.704+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f3c1a6890 con 0x7f2f3c1076c0 2024-05-10T10:39:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.706+0000 7f2f31ffb700 1 -- 172.21.15.112:0/951867879 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f2f340a9d00 con 0x7f2f3c1076c0 2024-05-10T10:39:45.707 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:39:45.707 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:39:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2f240b2880 msgr2=0x7f2f240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2f240b2880 0x7f2f240b4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009fd0 tx=0x7f2f38005fb0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c1076c0 msgr2=0x7f2f3c199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c1076c0 0x7f2f3c199950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400ada0 tx=0x7f2f3400c330 comp rx=0 tx=0).stop 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 shutdown_connections 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3c1076c0 0x7f2f3c199950 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f3c102630 0x7f2f3c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2f240b2880 0x7f2f240b4d40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 --2- 172.21.15.112:0/951867879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f3c102fe0 0x7f2f3c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 >> 172.21.15.112:0/951867879 conn(0x7f2f3c079350 msgr2=0x7f2f3c06bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 shutdown_connections 2024-05-10T10:39:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:45.708+0000 7f2f42e24700 1 -- 172.21.15.112:0/951867879 wait complete. 2024-05-10T10:39:46.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T10:35:52.903233+0000', 'modified': '2024-05-10T10:37:03.364184+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24451, 'mds_2': 24446, 'mds_3': 24445, 'mds_4': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14592': {'gid': 14592, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6839/2952588053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2952588053}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2952588053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6841/1213307484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 1213307484}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 1213307484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/3726517324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3726517324}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3726517324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/2643629917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 2643629917}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2643629917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6833/38979975', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6832', 'nonce': 38979975}, {'type': 'v1', 'addr': '172.21.15.195:6833', 'nonce': 38979975}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.148:6837/1474359623', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1474359623}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1474359623}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.195:6835/3491127541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 3491127541}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 3491127541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.148:6839/1008412635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 1008412635}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 1008412635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.195:6837/3223375652', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 3223375652}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 3223375652}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.195:6839/2238321568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 2238321568}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 2238321568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24428, 24445, 24446, 24451, 24469]} 2024-05-10T10:39:46.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-10T10:39:46.576 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-10T10:39:46.821 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:46.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:46 smithi112 ceph-mon[30100]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:46.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:46 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/951867879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:39:47.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:46 smithi148 ceph-mon[33568]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:47.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:46 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/951867879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:39:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:46 smithi195 ceph-mon[33773]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:46 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/951867879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:39:48.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 -- 172.21.15.112:0/948339572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 msgr2=0x7f888810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 --2- 172.21.15.112:0/948339572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888810f250 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f888000b0a0 tx=0x7f888000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 -- 172.21.15.112:0/948339572 shutdown_connections 2024-05-10T10:39:48.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 --2- 172.21.15.112:0/948339572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888810f250 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 --2- 172.21.15.112:0/948339572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8888108450 0x7f8888108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 --2- 172.21.15.112:0/948339572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88881023d0 0x7f8888102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 -- 172.21.15.112:0/948339572 >> 172.21.15.112:0/948339572 conn(0x7f8888078410 msgr2=0x7f8888078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 -- 172.21.15.112:0/948339572 shutdown_connections 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.516+0000 7f888e7eb700 1 -- 172.21.15.112:0/948339572 wait complete. 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 Processor -- start 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 -- start start 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88881023d0 0x7f888819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8888108450 0x7f88881998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8888112450 con 0x7f8888102d90 2024-05-10T10:39:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f88881122d0 con 0x7f88881023d0 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f888e7eb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88881125d0 con 0x7f8888108450 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f88877fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f88877fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34334/0 (socket says 172.21.15.112:34334) 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.517+0000 7f88877fe700 1 -- 172.21.15.112:0/586016948 learned_addr learned my addr 172.21.15.112:0/586016948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f8887fff700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88881023d0 0x7f888819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f888cd88700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8888108450 0x7f88881998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88877fe700 1 -- 172.21.15.112:0/586016948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8888108450 msgr2=0x7f88881998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88877fe700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8888108450 0x7f88881998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88877fe700 1 -- 172.21.15.112:0/586016948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88881023d0 msgr2=0x7f888819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88877fe700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88881023d0 0x7f888819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88877fe700 1 -- 172.21.15.112:0/586016948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888819a040 con 0x7f8888102d90 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88877fe700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888819fc60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d950 tx=0x7f887c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88857fa700 1 -- 172.21.15.112:0/586016948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c016070 con 0x7f8888102d90 2024-05-10T10:39:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88857fa700 1 -- 172.21.15.112:0/586016948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f887c009dd0 con 0x7f8888102d90 2024-05-10T10:39:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f88857fa700 1 -- 172.21.15.112:0/586016948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00f3c0 con 0x7f8888102d90 2024-05-10T10:39:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888819a330 con 0x7f8888102d90 2024-05-10T10:39:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.518+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a65f0 con 0x7f8888102d90 2024-05-10T10:39:48.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.521+0000 7f88857fa700 1 -- 172.21.15.112:0/586016948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f887c019070 con 0x7f8888102d90 2024-05-10T10:39:48.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.521+0000 7f88857fa700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88700b2900 0x7f88700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f8887fff700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88700b2900 0x7f88700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f88857fa700 1 -- 172.21.15.112:0/586016948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f887c0e21a0 con 0x7f8888102d90 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 Processor -- start 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f8887fff700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88700b2900 0x7f88700b4dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f888819aa20 tx=0x7f8878009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 -- start start 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21480 0x5559cdb21860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb260b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5559cda73810 con 0x5559cdb25c30 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5559cda12770 con 0x5559cdb21480 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.522+0000 7f889907eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5559cda6f7d0 con 0x5559cdb21da0 2024-05-10T10:39:48.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb260b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34346/0 (socket says 172.21.15.112:34346) 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 -- 172.21.15.112:0/2807038037 learned_addr learned my addr 172.21.15.112:0/2807038037 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f8887fff700 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21480 0x5559cdb21860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f88877fe700 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 -- 172.21.15.112:0/2807038037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 msgr2=0x5559cdb256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb256a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 -- 172.21.15.112:0/2807038037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21480 msgr2=0x5559cdb21860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21480 0x5559cdb21860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 -- 172.21.15.112:0/2807038037 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559cd9c0880 con 0x5559cdb25c30 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f8887fff700 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21480 0x5559cdb21860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f888cd88700 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb260b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8880015000 tx=0x7f8880009150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f82d358cdf96dc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f886e7fc700 1 -- 172.21.15.112:0/2807038037 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880004a50 con 0x5559cdb25c30 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f886e7fc700 1 -- 172.21.15.112:0/2807038037 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8880004bf0 con 0x5559cdb25c30 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.523+0000 7f886e7fc700 1 -- 172.21.15.112:0/2807038037 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888001c680 con 0x5559cdb25c30 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 -- 172.21.15.112:0/2807038037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 msgr2=0x5559cdb260b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb260b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8880015000 tx=0x7f8880009150 comp rx=0 tx=0).stop 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 -- 172.21.15.112:0/2807038037 shutdown_connections 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb260b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21480 0x5559cdb21860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 --2- 172.21.15.112:0/2807038037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 -- 172.21.15.112:0/2807038037 >> 172.21.15.112:0/2807038037 conn(0x5559cda4d0b0 msgr2=0x5559cdb1bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 -- 172.21.15.112:0/2807038037 shutdown_connections 2024-05-10T10:39:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 -- 172.21.15.112:0/2807038037 wait complete. 2024-05-10T10:39:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.524+0000 7f889907eb80 1 -- start start 2024-05-10T10:39:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 1 Processor -- start 2024-05-10T10:39:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb2a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb2a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb2dde0 con 0x5559cdb21480 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5559cd9d9be0 con 0x5559cdb25c30 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb2df60 con 0x5559cdb21da0 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34360/0 (socket says 172.21.15.112:34360) 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 -- 172.21.15.112:0/1680672442 learned_addr learned my addr 172.21.15.112:0/1680672442 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f889907eb80 10 client.0 _ll_register_callbacks cb 0x5559cda0b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f88877fe700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb2a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f888cd88700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb2a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 -- 172.21.15.112:0/1680672442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 msgr2=0x5559cdb2a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb2a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 -- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 msgr2=0x5559cdb2a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb2a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 -- 172.21.15.112:0/1680672442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cda0f640 con 0x5559cdb21480 2024-05-10T10:39:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f88877fe700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb2a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f888cd88700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb2a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f8887fff700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2cf80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8878015780 tx=0x7f8878004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f63f21c97dad6a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.525+0000 7f88677fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f88677fe700 1 -- 172.21.15.112:0/1680672442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878025020 con 0x5559cdb21480 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f88677fe700 1 -- 172.21.15.112:0/1680672442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8878015da0 con 0x5559cdb21480 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8880004bf0 con 0x5559cdb21480 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f88677fe700 1 -- 172.21.15.112:0/1680672442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8878030050 con 0x5559cdb21480 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 10 client.15204 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5559cdb385e0 con 0x5559cdb21480 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f88677fe700 1 -- 172.21.15.112:0/1680672442 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8880004bf0 con 0x5559cdb21480 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f88677fe700 1 -- 172.21.15.112:0/1680672442 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8878013450 con 0x5559cdb21480 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 10 client.15204 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 20 client.15204 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdb3acf0 0x5559cdb3c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 4 client.15204 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.526+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559cdaed600 con 0x5559cdb3acf0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.527+0000 7f88877fe700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdb3acf0 0x5559cdb3c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.527+0000 7f88677fe700 10 client.15204 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.527+0000 7f88877fe700 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdb3acf0 0x5559cdb3c160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b68dc682d1933350 server_cookie=d7f32f6add92e3ee in_seq=0 out_seq=0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f88677fe700 1 -- 172.21.15.112:0/1680672442 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559cdaed600 con 0x5559cdb3acf0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f88677fe700 10 client.15204 handle_command_reply: tid=0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 client.15204 shutdown 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 msgr2=0x5559cdb2cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2cf80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8878015780 tx=0x7f8878004860 comp rx=0 tx=0).stop 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 shutdown_connections 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb2a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2cf80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdb3acf0 0x5559cdb3c160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 --2- 172.21.15.112:0/1680672442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb2a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 >> 172.21.15.112:0/1680672442 conn(0x5559cda4d0b0 msgr2=0x5559cdb26030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f88677fe700 0 client.15204 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 shutdown_connections 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 1 -- 172.21.15.112:0/1680672442 wait complete. 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.528+0000 7f889907eb80 20 client.15204 trim_cache size 0 max 16384 2024-05-10T10:39:48.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 Processor -- start 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 -- start start 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb16550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb3a4a0 con 0x5559cdb21480 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb3ab70 con 0x5559cdb25c30 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f889907eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb3b240 con 0x5559cdb21da0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34376/0 (socket says 172.21.15.112:34376) 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 -- 172.21.15.112:0/3238601550 learned_addr learned my addr 172.21.15.112:0/3238601550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f88877fe700 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 -- 172.21.15.112:0/3238601550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 msgr2=0x5559cdb156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb156c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f888cd88700 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb16550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 -- 172.21.15.112:0/3238601550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 msgr2=0x5559cdb16550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb16550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 -- 172.21.15.112:0/3238601550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559cdb187c0 con 0x5559cdb21480 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f88877fe700 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f888cd88700 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb16550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.555+0000 7f8887fff700 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2a940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f88780138c0 tx=0x7f887800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667f4e80c5fa2406 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f886dffb700 1 -- 172.21.15.112:0/3238601550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878025070 con 0x5559cdb21480 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f886dffb700 1 -- 172.21.15.112:0/3238601550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8878039070 con 0x5559cdb21480 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f886dffb700 1 -- 172.21.15.112:0/3238601550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887801b040 con 0x5559cdb21480 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 -- 172.21.15.112:0/3238601550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 msgr2=0x5559cdb2a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2a940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f88780138c0 tx=0x7f887800e6c0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 -- 172.21.15.112:0/3238601550 shutdown_connections 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb25c30 0x5559cdb16550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb21480 0x5559cdb2a940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 --2- 172.21.15.112:0/3238601550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21da0 0x5559cdb156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 -- 172.21.15.112:0/3238601550 >> 172.21.15.112:0/3238601550 conn(0x5559cda4d0b0 msgr2=0x5559cdb15e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 -- 172.21.15.112:0/3238601550 shutdown_connections 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.556+0000 7f889907eb80 1 -- 172.21.15.112:0/3238601550 wait complete. 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 -- start start 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 Processor -- start 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21480 0x5559cdb3f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21da0 0x5559cdb3fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb3bfe0 con 0x5559cdb25c30 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb3b910 con 0x5559cdb21da0 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5559cdb3c6b0 con 0x5559cdb21480 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f889907eb80 10 client.0 _ll_register_callbacks cb 0x5559cda0b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f888cd88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f888cd88700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb43ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34388/0 (socket says 172.21.15.112:34388) 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f888cd88700 1 -- 172.21.15.112:0/151632754 learned_addr learned my addr 172.21.15.112:0/151632754 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f8887fff700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21480 0x5559cdb3f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.557+0000 7f88877fe700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21da0 0x5559cdb3fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f888cd88700 1 -- 172.21.15.112:0/151632754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21480 msgr2=0x5559cdb3f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f888cd88700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21480 0x5559cdb3f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f888cd88700 1 -- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21da0 msgr2=0x5559cdb3fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f888cd88700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21da0 0x5559cdb3fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f888cd88700 1 -- 172.21.15.112:0/151632754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cdb44530 con 0x5559cdb25c30 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f88877fe700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21da0 0x5559cdb3fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f888cd88700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb43ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8880009a20 tx=0x7f8880007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18417ccb85ca8fdb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f88677fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f88677fe700 1 -- 172.21.15.112:0/151632754 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888002a070 con 0x5559cdb25c30 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f88677fe700 1 -- 172.21.15.112:0/151632754 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8880004a50 con 0x5559cdb25c30 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8878039070 con 0x5559cdb25c30 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f88677fe700 1 -- 172.21.15.112:0/151632754 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8880036050 con 0x5559cdb25c30 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f889907eb80 10 client.15210 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5559cdba69f0 con 0x5559cdb25c30 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f88677fe700 1 -- 172.21.15.112:0/151632754 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8878039070 con 0x5559cdb25c30 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f88677fe700 1 -- 172.21.15.112:0/151632754 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8880011070 con 0x5559cdb25c30 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f889907eb80 10 client.15210 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f889907eb80 10 client.15210 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.558+0000 7f889907eb80 20 client.15210 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f889907eb80 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdbab0e0 0x5559cdba9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f889907eb80 4 client.15210 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "status"}) -- 0x5559cda811f0 con 0x5559cdbab0e0 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f8887fff700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdbab0e0 0x5559cdba9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f88677fe700 10 client.15210 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f8887fff700 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdbab0e0 0x5559cdba9df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=223e5af2ce910805 server_cookie=fd3ebac84dba6d5c in_seq=0 out_seq=0 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f88677fe700 1 -- 172.21.15.112:0/151632754 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5559cda811f0 con 0x5559cdbab0e0 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f88677fe700 10 client.15210 handle_command_reply: tid=0 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.559+0000 7f889907eb80 1 client.15210 shutdown 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 msgr2=0x5559cdb43ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb43ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8880009a20 tx=0x7f8880007bd0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 shutdown_connections 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559cdb25c30 0x5559cdb43ef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x5559cdbab0e0 0x5559cdba9df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5559cdb21480 0x5559cdb3f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 --2- 172.21.15.112:0/151632754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559cdb21da0 0x5559cdb3fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 >> 172.21.15.112:0/151632754 conn(0x5559cda4d0b0 msgr2=0x5559cdb166d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f88677fe700 0 client.15210 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:39:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 shutdown_connections 2024-05-10T10:39:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 1 -- 172.21.15.112:0/151632754 wait complete. 2024-05-10T10:39:48.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f889907eb80 20 client.15210 trim_cache size 0 max 16384 2024-05-10T10:39:48.569 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 4, 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24428, 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 168.167773471, 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:48.570 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:48.571 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 266.81665956199998 2024-05-10T10:39:48.571 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88700b2900 msgr2=0x7f88700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f888e7eb700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88700b2900 0x7f88700b4dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f888819aa20 tx=0x7f8878009190 comp rx=0 tx=0).stop 2024-05-10T10:39:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 msgr2=0x7f888819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.560+0000 7f888e7eb700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888819fc60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d950 tx=0x7f887c00dc60 comp rx=0 tx=0).stop 2024-05-10T10:39:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 shutdown_connections 2024-05-10T10:39:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88700b2900 0x7f88700b4dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8888108450 0x7f88881998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88881023d0 0x7f888819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 --2- 172.21.15.112:0/586016948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8888102d90 0x7f888819fc60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 >> 172.21.15.112:0/586016948 conn(0x7f8888078410 msgr2=0x7f888806bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 shutdown_connections 2024-05-10T10:39:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:48.561+0000 7f888e7eb700 1 -- 172.21.15.112:0/586016948 wait complete. 2024-05-10T10:39:48.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:48 smithi112 ceph-mon[30100]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:48.847 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:48 smithi148 ceph-mon[33568]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:48 smithi195 ceph-mon[33773]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:49.324 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.f status 2024-05-10T10:39:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:50.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:50 smithi112 ceph-mon[30100]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:51.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:50 smithi148 ceph-mon[33568]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:50 smithi195 ceph-mon[33773]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:39:51.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 -- 172.21.15.112:0/3162750575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8108850 msgr2=0x7fdfd81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 --2- 172.21.15.112:0/3162750575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8108850 0x7fdfd81103d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000b0d0 tx=0x7fdfd000b3e0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 -- 172.21.15.112:0/3162750575 shutdown_connections 2024-05-10T10:39:51.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 --2- 172.21.15.112:0/3162750575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8108850 0x7fdfd81103d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 --2- 172.21.15.112:0/3162750575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfd8103750 0x7fdfd8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 --2- 172.21.15.112:0/3162750575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfd8104100 0x7fdfd8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 -- 172.21.15.112:0/3162750575 >> 172.21.15.112:0/3162750575 conn(0x7fdfd807a600 msgr2=0x7fdfd807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:51.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 -- 172.21.15.112:0/3162750575 shutdown_connections 2024-05-10T10:39:51.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.419+0000 7fdfdf463700 1 -- 172.21.15.112:0/3162750575 wait complete. 2024-05-10T10:39:51.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 Processor -- start 2024-05-10T10:39:51.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 -- start start 2024-05-10T10:39:51.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfd8103750 0x7fdfd80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8104100 0x7fdfd8075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfd8108850 0x7fdfd8076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd8113570 con 0x7fdfd8104100 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd81133f0 con 0x7fdfd8108850 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdf463700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd81136f0 con 0x7fdfd8103750 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdc9fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8104100 0x7fdfd8075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdc9fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8104100 0x7fdfd8075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34408/0 (socket says 172.21.15.112:34408) 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2005197952 learned_addr learned my addr 172.21.15.112:0/2005197952 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdda00700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfd8108850 0x7fdfd8076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.420+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfd8103750 0x7fdfd80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2005197952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfd8103750 msgr2=0x7fdfd80757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfd8103750 0x7fdfd80757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2005197952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfd8108850 msgr2=0x7fdfd8076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfd8108850 0x7fdfd8076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2005197952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd81a7f50 con 0x7fdfd8104100 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdda00700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfd8108850 0x7fdfd8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8104100 0x7fdfd8075ce0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800d980 tx=0x7fdfc800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfce7fc700 1 -- 172.21.15.112:0/2005197952 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8017070 con 0x7fdfd8104100 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd81a8240 con 0x7fdfd8104100 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfce7fc700 1 -- 172.21.15.112:0/2005197952 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfc8004830 con 0x7fdfd8104100 2024-05-10T10:39:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfce7fc700 1 -- 172.21.15.112:0/2005197952 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80053a0 con 0x7fdfd8104100 2024-05-10T10:39:51.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.421+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd81a8780 con 0x7fdfd8104100 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.424+0000 7fdfce7fc700 1 -- 172.21.15.112:0/2005197952 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdfc80049e0 con 0x7fdfd8104100 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.425+0000 7fdfce7fc700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdfc40b2900 0x7fdfc40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.425+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdfc40b2900 0x7fdfc40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.425+0000 7fdfce7fc700 1 -- 172.21.15.112:0/2005197952 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fdfc80e2520 con 0x7fdfd8104100 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.425+0000 7fdfe9cf6b80 1 Processor -- start 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.425+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdfc40b2900 0x7fdfc40b4dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4006fd0 tx=0x7fdfd4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.425+0000 7fdfe9cf6b80 1 -- start start 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a53540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d1799f8f0 con 0x562d17a4dd00 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562d17904d50 con 0x562d17a53160 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d1799b8b0 con 0x562d17a53a80 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a4e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34410/0 (socket says 172.21.15.112:34410) 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 -- 172.21.15.112:0/1446048602 learned_addr learned my addr 172.21.15.112:0/1446048602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a53540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 -- 172.21.15.112:0/1446048602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 msgr2=0x562d17a4d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a4d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 -- 172.21.15.112:0/1446048602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 msgr2=0x562d17a53540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a53540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 -- 172.21.15.112:0/1446048602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d178ec880 con 0x562d17a4dd00 2024-05-10T10:39:51.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfdda00700 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a4e140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009de0 tx=0x7fdfd0009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd571f3ed04356e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.426+0000 7fdfc37fe700 1 -- 172.21.15.112:0/1446048602 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0022070 con 0x562d17a4dd00 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfc37fe700 1 -- 172.21.15.112:0/1446048602 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfd0009480 con 0x562d17a4dd00 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfc37fe700 1 -- 172.21.15.112:0/1446048602 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd001c890 con 0x562d17a4dd00 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/1446048602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 msgr2=0x562d17a4e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a4e140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009de0 tx=0x7fdfd0009e10 comp rx=0 tx=0).stop 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/1446048602 shutdown_connections 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a4e140 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a53540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/1446048602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a4d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/1446048602 >> 172.21.15.112:0/1446048602 conn(0x562d17979b20 msgr2=0x562d17a48250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/1446048602 shutdown_connections 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.427+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/1446048602 wait complete. 2024-05-10T10:39:51.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 -- start start 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 Processor -- start 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 0x562d17a59720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a56ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 0x562d17a57000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d17a5a3a0 con 0x562d17a53160 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562d17958e80 con 0x562d17a53a80 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d17a5a520 con 0x562d17a4dd00 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfe9cf6b80 10 client.0 _ll_register_callbacks cb 0x562d17933560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfdc9fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a56ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfdc9fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a56ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34424/0 (socket says 172.21.15.112:34424) 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2722653106 learned_addr learned my addr 172.21.15.112:0/2722653106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.428+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 0x562d17a59720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfdda00700 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 0x562d17a57000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 msgr2=0x562d17a59720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 0x562d17a59720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2722653106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 msgr2=0x562d17a57000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 0x562d17a57000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2722653106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d17956710 con 0x562d17a53160 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a56ac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80a96e0 tx=0x7fdfc80a9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927d3c2cee00cdd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2722653106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80a9750 con 0x562d17a53160 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2722653106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fdfc80a99b0 con 0x562d17a53160 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d179ad2d0 con 0x562d17a53160 2024-05-10T10:39:51.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2722653106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdfc80adb00 con 0x562d17a53160 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfe9cf6b80 10 client.15222 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d17a64c80 con 0x562d17a53160 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2722653106 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562d179ad2d0 con 0x562d17a53160 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.429+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2722653106 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdfc80beba0 con 0x562d17a53160 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfe9cf6b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfe9cf6b80 10 client.15222 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfe9cf6b80 20 client.15222 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17a67290 0x562d17a68700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfe9cf6b80 4 client.15222 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d179a7370 con 0x562d17a67290 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17a67290 0x562d17a68700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfc0ff9700 10 client.15222 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.430+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17a67290 0x562d17a68700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40e845259e877684 server_cookie=68e4ad9156ed12c0 in_seq=0 out_seq=0 2024-05-10T10:39:51.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2722653106 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d179a7370 con 0x562d17a67290 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfc0ff9700 10 client.15222 handle_command_reply: tid=0 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 client.15222 shutdown 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 msgr2=0x562d17a56ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a56ac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80a96e0 tx=0x7fdfc80a9310 comp rx=0 tx=0).stop 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 shutdown_connections 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17a67290 0x562d17a68700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 0x562d17a57000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 0x562d17a59720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2722653106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a56ac0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 >> 172.21.15.112:0/2722653106 conn(0x562d17979b20 msgr2=0x562d17a42ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.431+0000 7fdfc0ff9700 0 client.15222 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.432+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 shutdown_connections 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.432+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2722653106 wait complete. 2024-05-10T10:39:51.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.432+0000 7fdfe9cf6b80 20 client.15222 trim_cache size 0 max 16384 2024-05-10T10:39:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 Processor -- start 2024-05-10T10:39:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 -- start start 2024-05-10T10:39:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 0x562d17a4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 0x562d17a42bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d17a66a40 con 0x562d17a53160 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562d17a67110 con 0x562d17a53a80 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d17a677e0 con 0x562d17a4dd00 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfdc9fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfdc9fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34438/0 (socket says 172.21.15.112:34438) 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.452+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2351925593 learned_addr learned my addr 172.21.15.112:0/2351925593 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2351925593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 msgr2=0x562d17a4d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2351925593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 0x562d17a4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2351925593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 msgr2=0x562d17a42bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2351925593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 0x562d17a42bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfdc9fe700 1 -- 172.21.15.112:0/2351925593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d17a53790 con 0x562d17a53160 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2351925593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a546f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80a5050 tx=0x7fdfc80dfcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a819ffe8657aac54 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:51.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfc2ffd700 1 -- 172.21.15.112:0/2351925593 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80a5610 con 0x562d17a53160 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfc2ffd700 1 -- 172.21.15.112:0/2351925593 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfc80c7020 con 0x562d17a53160 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfc2ffd700 1 -- 172.21.15.112:0/2351925593 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80910b0 con 0x562d17a53160 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2351925593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 msgr2=0x562d17a546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.453+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2351925593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a546f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80a5050 tx=0x7fdfc80dfcb0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2351925593 shutdown_connections 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2351925593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53a80 0x562d17a42bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2351925593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a4dd00 0x562d17a4d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2351925593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a53160 0x562d17a546f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2351925593 >> 172.21.15.112:0/2351925593 conn(0x562d17979b20 msgr2=0x562d17a553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2351925593 shutdown_connections 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2351925593 wait complete. 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- start start 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 Processor -- start 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a6be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a6c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a70370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d17a68580 con 0x562d17a4dd00 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562d17a67eb0 con 0x562d17a53160 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d17a68c50 con 0x562d17a53a80 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfe9cf6b80 10 client.0 _ll_register_callbacks cb 0x562d17933560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.454+0000 7fdfdd1ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a6be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a6be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34450/0 (socket says 172.21.15.112:34450) 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 -- 172.21.15.112:0/2599944761 learned_addr learned my addr 172.21.15.112:0/2599944761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a6c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdda00700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a70370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 -- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 msgr2=0x562d17a70370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a70370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 -- 172.21.15.112:0/2599944761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 msgr2=0x562d17a6c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:51.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a6c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 -- 172.21.15.112:0/2599944761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d17a708e0 con 0x562d17a4dd00 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a6c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfdd1ff700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a6be30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4008010 tx=0x7fdfd4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7cf2cb4a5f22ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2599944761 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4014a90 con 0x562d17a4dd00 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2599944761 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fdfd4028050 con 0x562d17a4dd00 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc80c7020 con 0x562d17a4dd00 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2599944761 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdfd4037050 con 0x562d17a4dd00 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfe9cf6b80 10 client.15228 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d17ad2f60 con 0x562d17a4dd00 2024-05-10T10:39:51.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.455+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2599944761 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfc80c7020 con 0x562d17a4dd00 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2599944761 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdfd4013070 con 0x562d17a4dd00 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfe9cf6b80 10 client.15228 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfe9cf6b80 10 client.15228 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfe9cf6b80 20 client.15228 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17ad7bf0 0x562d17ad6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfe9cf6b80 4 client.15228 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "status"}) -- 0x562d1781f200 con 0x562d17ad7bf0 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.456+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17ad7bf0 0x562d17ad6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfc0ff9700 10 client.15228 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfdc9fe700 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17ad7bf0 0x562d17ad6900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7110d8fe4e208b8 server_cookie=c228c2c57b76b53d in_seq=0 out_seq=0 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfc0ff9700 1 -- 172.21.15.112:0/2599944761 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562d1781f200 con 0x562d17ad7bf0 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfc0ff9700 10 client.15228 handle_command_reply: tid=0 2024-05-10T10:39:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 client.15228 shutdown 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 msgr2=0x562d17a6be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a6be30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4008010 tx=0x7fdfd4004860 comp rx=0 tx=0).stop 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 shutdown_connections 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x562d17ad7bf0 0x562d17ad6900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d17a53a80 0x562d17a70370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d17a4dd00 0x562d17a6be30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 --2- 172.21.15.112:0/2599944761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d17a53160 0x562d17a6c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfc0ff9700 0 client.15228 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 >> 172.21.15.112:0/2599944761 conn(0x562d17979b20 msgr2=0x562d17a52450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 shutdown_connections 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 1 -- 172.21.15.112:0/2599944761 wait complete. 2024-05-10T10:39:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.457+0000 7fdfe9cf6b80 20 client.15228 trim_cache size 0 max 16384 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 3, 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24445, 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 172.073887622, 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 275.59842978400002 2024-05-10T10:39:51.467 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdfc40b2900 msgr2=0x7fdfc40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdfc40b2900 0x7fdfc40b4dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4006fd0 tx=0x7fdfd4008040 comp rx=0 tx=0).stop 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8104100 msgr2=0x7fdfd8075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8104100 0x7fdfd8075ce0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800d980 tx=0x7fdfc800dd40 comp rx=0 tx=0).stop 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 shutdown_connections 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfd8108850 0x7fdfd8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdfc40b2900 0x7fdfc40b4dc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfd8103750 0x7fdfd80757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 --2- 172.21.15.112:0/2005197952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd8104100 0x7fdfd8075ce0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:51.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 >> 172.21.15.112:0/2005197952 conn(0x7fdfd807a600 msgr2=0x7fdfd8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:51.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 shutdown_connections 2024-05-10T10:39:51.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:51.458+0000 7fdfdf463700 1 -- 172.21.15.112:0/2005197952 wait complete. 2024-05-10T10:39:52.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-10T10:39:52.612 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:52.952 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:52 smithi112 ceph-mon[30100]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:53.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:52 smithi148 ceph-mon[33568]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:52 smithi195 ceph-mon[33773]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:39:54.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 -- 172.21.15.112:0/884109481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c1020c0 msgr2=0x7fcd3c1024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 --2- 172.21.15.112:0/884109481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c1020c0 0x7fcd3c1024a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b0a0 tx=0x7fcd2c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 -- 172.21.15.112:0/884109481 shutdown_connections 2024-05-10T10:39:54.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 --2- 172.21.15.112:0/884109481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3c106f60 0x7fcd3c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 --2- 172.21.15.112:0/884109481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c1020c0 0x7fcd3c1024a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 --2- 172.21.15.112:0/884109481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd3c102a70 0x7fcd3c106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 -- 172.21.15.112:0/884109481 >> 172.21.15.112:0/884109481 conn(0x7fcd3c0776c0 msgr2=0x7fcd3c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:54.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 -- 172.21.15.112:0/884109481 shutdown_connections 2024-05-10T10:39:54.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.497+0000 7fcd41f52700 1 -- 172.21.15.112:0/884109481 wait complete. 2024-05-10T10:39:54.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 Processor -- start 2024-05-10T10:39:54.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 -- start start 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd3c1020c0 0x7fcd3c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3c102a70 0x7fcd3c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c106f60 0x7fcd3c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111c50 con 0x7fcd3c106f60 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111ad0 con 0x7fcd3c102a70 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd41f52700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111dd0 con 0x7fcd3c1020c0 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c106f60 0x7fcd3c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c106f60 0x7fcd3c1a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34464/0 (socket says 172.21.15.112:34464) 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd3bfff700 1 -- 172.21.15.112:0/3678761196 learned_addr learned my addr 172.21.15.112:0/3678761196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd3c1020c0 0x7fcd3c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.498+0000 7fcd3affd700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3c102a70 0x7fcd3c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd3bfff700 1 -- 172.21.15.112:0/3678761196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd3c1020c0 msgr2=0x7fcd3c19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd3c1020c0 0x7fcd3c19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd3bfff700 1 -- 172.21.15.112:0/3678761196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3c102a70 msgr2=0x7fcd3c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3c102a70 0x7fcd3c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd3bfff700 1 -- 172.21.15.112:0/3678761196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c1a5ed0 con 0x7fcd3c106f60 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd3affd700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3c102a70 0x7fcd3c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c106f60 0x7fcd3c1a5990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c970 tx=0x7fcd3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd38ff9700 1 -- 172.21.15.112:0/3678761196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400ce60 con 0x7fcd3c106f60 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1a60a0 con 0x7fcd3c106f60 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd38ff9700 1 -- 172.21.15.112:0/3678761196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd34004830 con 0x7fcd3c106f60 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd38ff9700 1 -- 172.21.15.112:0/3678761196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd340053e0 con 0x7fcd3c106f60 2024-05-10T10:39:54.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.499+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a65e0 con 0x7fcd3c106f60 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.502+0000 7fcd38ff9700 1 -- 172.21.15.112:0/3678761196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcd34007b00 con 0x7fcd3c106f60 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.502+0000 7fcd38ff9700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fcd240b2880 0x7fcd240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fcd240b2880 0x7fcd240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd38ff9700 1 -- 172.21.15.112:0/3678761196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fcd340e2180 con 0x7fcd3c106f60 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 Processor -- start 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 -- start start 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fcd240b2880 0x7fcd240b4d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009e60 tx=0x7fcd2c0090d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd86a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86aad0 0x562ebd86e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 0x562ebd86eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ebd7b38f0 con 0x562ebd86a1b0 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ebd718d50 con 0x562ebd86ea50 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ebd7af8b0 con 0x562ebd86aad0 2024-05-10T10:39:54.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd86a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd86a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34466/0 (socket says 172.21.15.112:34466) 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2854573469 learned_addr learned my addr 172.21.15.112:0/2854573469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86aad0 0x562ebd86e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2854573469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86aad0 msgr2=0x562ebd86e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.503+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86aad0 0x562ebd86e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.504+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2854573469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 msgr2=0x562ebd86eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.504+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 0x562ebd86eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.504+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2854573469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebd700880 con 0x562ebd86a1b0 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.504+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd86a590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c015000 tx=0x7fcd2c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d78aabf245c967 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.504+0000 7fcd21ffb700 1 -- 172.21.15.112:0/2854573469 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c02c070 con 0x562ebd86a1b0 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.504+0000 7fcd21ffb700 1 -- 172.21.15.112:0/2854573469 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd2c005040 con 0x562ebd86a1b0 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.504+0000 7fcd21ffb700 1 -- 172.21.15.112:0/2854573469 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c028690 con 0x562ebd86a1b0 2024-05-10T10:39:54.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2854573469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 msgr2=0x562ebd86a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd86a590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c015000 tx=0x7fcd2c01d000 comp rx=0 tx=0).stop 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2854573469 shutdown_connections 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 0x562ebd86eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd86a590 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2854573469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86aad0 0x562ebd86e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2854573469 >> 172.21.15.112:0/2854573469 conn(0x562ebd78db20 msgr2=0x562ebd85c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2854573469 shutdown_connections 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2854573469 wait complete. 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 -- start start 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.505+0000 7fcd4c7e5b80 1 Processor -- start 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd875ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd873270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86ea50 0x562ebd8737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ebd876b50 con 0x562ebd86a1b0 2024-05-10T10:39:54.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ebd76ce80 con 0x562ebd86aad0 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ebd876cd0 con 0x562ebd86ea50 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 10 client.0 _ll_register_callbacks cb 0x562ebd747560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd875ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd875ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34476/0 (socket says 172.21.15.112:34476) 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2565521807 learned_addr learned my addr 172.21.15.112:0/2565521807 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86ea50 0x562ebd8737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd873270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2565521807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86ea50 msgr2=0x562ebd8737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86ea50 0x562ebd8737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 msgr2=0x562ebd873270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd873270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 -- 172.21.15.112:0/2565521807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebd76a710 con 0x562ebd86a1b0 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd873270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:54.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd875ed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b070 tx=0x7fcd2c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a87e1d693f9ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd1affd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd1affd700 1 -- 172.21.15.112:0/2565521807 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c037070 con 0x562ebd86a1b0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd1affd700 1 -- 172.21.15.112:0/2565521807 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fcd2c005600 con 0x562ebd86a1b0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.506+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2c005040 con 0x562ebd86a1b0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd1affd700 1 -- 172.21.15.112:0/2565521807 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd2c02c040 con 0x562ebd86a1b0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 10 client.15240 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ebd8813b0 con 0x562ebd86a1b0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd1affd700 1 -- 172.21.15.112:0/2565521807 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd2c005040 con 0x562ebd86a1b0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd1affd700 1 -- 172.21.15.112:0/2565521807 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd2c011070 con 0x562ebd86a1b0 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 10 client.15240 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 20 client.15240 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd882700 0x562ebd8848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 4 client.15240 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ebd7c12d0 con 0x562ebd882700 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.507+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd882700 0x562ebd8848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.508+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd882700 0x562ebd8848b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f5e3d76eab25d49 server_cookie=b31cb4a0f6bd5e74 in_seq=0 out_seq=0 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.508+0000 7fcd1affd700 10 client.15240 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.508+0000 7fcd1affd700 1 -- 172.21.15.112:0/2565521807 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ebd7c12d0 con 0x562ebd882700 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd1affd700 10 client.15240 handle_command_reply: tid=0 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 client.15240 shutdown 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 msgr2=0x562ebd875ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd875ed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b070 tx=0x7fcd2c007ec0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 shutdown_connections 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86ea50 0x562ebd8737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd882700 0x562ebd8848b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86a1b0 0x562ebd875ed0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2565521807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd873270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd1affd700 0 client.15240 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 >> 172.21.15.112:0/2565521807 conn(0x562ebd78db20 msgr2=0x562ebd856ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 shutdown_connections 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2565521807 wait complete. 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.509+0000 7fcd4c7e5b80 20 client.15240 trim_cache size 0 max 16384 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.529+0000 7fcd4c7e5b80 1 Processor -- start 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.529+0000 7fcd4c7e5b80 1 -- start start 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.529+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 0x562ebd882d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.529+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd856bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.529+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86ea50 0x562ebd86c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.529+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ebd882bf0 con 0x562ebd86ea50 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ebd8832c0 con 0x562ebd86aad0 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ebd883990 con 0x562ebd86a1b0 2024-05-10T10:39:54.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86ea50 0x562ebd86c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86ea50 0x562ebd86c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34482/0 (socket says 172.21.15.112:34482) 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 -- 172.21.15.112:0/2191648006 learned_addr learned my addr 172.21.15.112:0/2191648006 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 -- 172.21.15.112:0/2191648006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 msgr2=0x562ebd882d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/2191648006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 0x562ebd882d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 -- 172.21.15.112:0/2191648006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 msgr2=0x562ebd856bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/2191648006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd856bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 -- 172.21.15.112:0/2191648006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebd858c30 con 0x562ebd86ea50 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/2191648006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86ea50 0x562ebd86c410 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd340a9110 tx=0x7fcd340a8f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebf26e0b426c13b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd217fa700 1 -- 172.21.15.112:0/2191648006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3401d040 con 0x562ebd86ea50 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd217fa700 1 -- 172.21.15.112:0/2191648006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd340a9410 con 0x562ebd86ea50 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.530+0000 7fcd217fa700 1 -- 172.21.15.112:0/2191648006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd340a9670 con 0x562ebd86ea50 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2191648006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86ea50 msgr2=0x562ebd86c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2191648006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86ea50 0x562ebd86c410 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd340a9110 tx=0x7fcd340a8f20 comp rx=0 tx=0).stop 2024-05-10T10:39:54.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2191648006 shutdown_connections 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2191648006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86ea50 0x562ebd86c410 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2191648006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 0x562ebd882d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2191648006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86aad0 0x562ebd856bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2191648006 >> 172.21.15.112:0/2191648006 conn(0x562ebd78db20 msgr2=0x562ebd85c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2191648006 shutdown_connections 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2191648006 wait complete. 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 -- start start 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.531+0000 7fcd4c7e5b80 1 Processor -- start 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 0x562ebd887aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86aad0 0x562ebd887fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 0x562ebd888520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ebd884730 con 0x562ebd86aad0 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ebd884060 con 0x562ebd86ea50 2024-05-10T10:39:54.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ebd884e00 con 0x562ebd86a1b0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 10 client.0 _ll_register_callbacks cb 0x562ebd747560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86aad0 0x562ebd887fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86aad0 0x562ebd887fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34490/0 (socket says 172.21.15.112:34490) 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 -- 172.21.15.112:0/2877425908 learned_addr learned my addr 172.21.15.112:0/2877425908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 0x562ebd887aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3bfff700 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 0x562ebd888520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 -- 172.21.15.112:0/2877425908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 msgr2=0x562ebd887aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 0x562ebd887aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 -- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 msgr2=0x562ebd888520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 0x562ebd888520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 -- 172.21.15.112:0/2877425908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebd88c510 con 0x562ebd86aad0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd3affd700 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86aad0 0x562ebd887fe0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c1095e0 tx=0x7fcd30009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd275c8c91c05b19 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd1affd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:54.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd1affd700 1 -- 172.21.15.112:0/2877425908 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30016ba0 con 0x562ebd86aad0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd340a9410 con 0x562ebd86aad0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd1affd700 1 -- 172.21.15.112:0/2877425908 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fcd30016d40 con 0x562ebd86aad0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.532+0000 7fcd1affd700 1 -- 172.21.15.112:0/2877425908 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd3002b050 con 0x562ebd86aad0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 10 client.15246 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ebd8eeeb0 con 0x562ebd86aad0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd1affd700 1 -- 172.21.15.112:0/2877425908 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd340a9410 con 0x562ebd86aad0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd1affd700 1 -- 172.21.15.112:0/2877425908 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd30020020 con 0x562ebd86aad0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 10 client.15246 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 10 client.15246 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 20 client.15246 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd8f3a40 0x562ebd8f2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 4 client.15246 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "status"}) -- 0x562ebd7bb370 con 0x562ebd8f3a40 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.533+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd8f3a40 0x562ebd8f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:54.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd1affd700 10 client.15246 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd3b7fe700 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd8f3a40 0x562ebd8f2750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5df8f5c60fbd05df server_cookie=25fd88413803500e in_seq=0 out_seq=0 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd1affd700 1 -- 172.21.15.112:0/2877425908 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562ebd7bb370 con 0x562ebd8f3a40 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd1affd700 10 client.15246 handle_command_reply: tid=0 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 client.15246 shutdown 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86aad0 msgr2=0x562ebd887fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86aad0 0x562ebd887fe0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c1095e0 tx=0x7fcd30009ec0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 shutdown_connections 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ebd86ea50 0x562ebd888520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ebd86a1b0 0x562ebd887aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x562ebd8f3a40 0x562ebd8f2750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 --2- 172.21.15.112:0/2877425908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ebd86aad0 0x562ebd887fe0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 >> 172.21.15.112:0/2877425908 conn(0x562ebd78db20 msgr2=0x562ebd86d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd1affd700 0 client.15246 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:39:54.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 shutdown_connections 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 1 -- 172.21.15.112:0/2877425908 wait complete. 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.534+0000 7fcd4c7e5b80 20 client.15246 trim_cache size 0 max 16384 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24446, 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:54.661 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 176.15973558600001, 2024-05-10T10:39:54.662 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:54.662 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:54.662 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:54.662 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 259.30568707200001 2024-05-10T10:39:54.662 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:54.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fcd240b2880 msgr2=0x7fcd240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fcd240b2880 0x7fcd240b4d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009e60 tx=0x7fcd2c0090d0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c106f60 msgr2=0x7fcd3c1a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c106f60 0x7fcd3c1a5990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c970 tx=0x7fcd3400cc80 comp rx=0 tx=0).stop 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 shutdown_connections 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c106f60 0x7fcd3c1a5990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fcd240b2880 0x7fcd240b4d40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd3c1020c0 0x7fcd3c19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 --2- 172.21.15.112:0/3678761196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3c102a70 0x7fcd3c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 >> 172.21.15.112:0/3678761196 conn(0x7fcd3c0776c0 msgr2=0x7fcd3c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 shutdown_connections 2024-05-10T10:39:54.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:54.535+0000 7fcd41f52700 1 -- 172.21.15.112:0/3678761196 wait complete. 2024-05-10T10:39:54.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:54 smithi112 ceph-mon[30100]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:55.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:54 smithi148 ceph-mon[33568]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:55.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:54 smithi195 ceph-mon[33773]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:39:55.314 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-10T10:39:55.564 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:56.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:56 smithi112 ceph-mon[30100]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:57.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:56 smithi148 ceph-mon[33568]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:56 smithi195 ceph-mon[33773]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 -- 172.21.15.112:0/1859814940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 msgr2=0x7f48d4102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 --2- 172.21.15.112:0/1859814940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d4102810 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009990 tx=0x7f48bc009ca0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 -- 172.21.15.112:0/1859814940 shutdown_connections 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 --2- 172.21.15.112:0/1859814940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102d50 0x7f48d410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 --2- 172.21.15.112:0/1859814940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d4108400 0x7f48d41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 --2- 172.21.15.112:0/1859814940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d4102810 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 -- 172.21.15.112:0/1859814940 >> 172.21.15.112:0/1859814940 conn(0x7f48d4079310 msgr2=0x7f48d4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 -- 172.21.15.112:0/1859814940 shutdown_connections 2024-05-10T10:39:57.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 -- 172.21.15.112:0/1859814940 wait complete. 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 Processor -- start 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.351+0000 7f48d9b86700 1 -- start start 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d9b86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d9b86700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102d50 0x7f48d419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d9b86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d4108400 0x7f48d41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d9b86700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4112400 con 0x7f48d4102390 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d9b86700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4112280 con 0x7f48d4108400 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d9b86700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4112580 con 0x7f48d4102d50 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53784/0 (socket says 172.21.15.112:53784) 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 -- 172.21.15.112:0/871412719 learned_addr learned my addr 172.21.15.112:0/871412719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48cbfff700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102d50 0x7f48d419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d3fff700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d4108400 0x7f48d41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 -- 172.21.15.112:0/871412719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102d50 msgr2=0x7f48d419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102d50 0x7f48d419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 -- 172.21.15.112:0/871412719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d4108400 msgr2=0x7f48d41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d4108400 0x7f48d41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 -- 172.21.15.112:0/871412719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d4198fc0 con 0x7f48d4102390 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48cbfff700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102d50 0x7f48d419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d37fe700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d419e660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9a0 tx=0x7f48c400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d17fa700 1 -- 172.21.15.112:0/871412719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c400cbd0 con 0x7f48d4102390 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d17fa700 1 -- 172.21.15.112:0/871412719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c400cd70 con 0x7f48d4102390 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d17fa700 1 -- 172.21.15.112:0/871412719 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4018680 con 0x7f48d4102390 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.352+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d41992b0 con 0x7f48d4102390 2024-05-10T10:39:57.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.353+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41a54c0 con 0x7f48d4102390 2024-05-10T10:39:57.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.355+0000 7f48d17fa700 1 -- 172.21.15.112:0/871412719 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f48c4010950 con 0x7f48d4102390 2024-05-10T10:39:57.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.355+0000 7f48d17fa700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f48c00b2880 0x7f48c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48cbfff700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f48c00b2880 0x7f48c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48d17fa700 1 -- 172.21.15.112:0/871412719 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f48c40e2870 con 0x7f48d4102390 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 Processor -- start 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48cbfff700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f48c00b2880 0x7f48c00b4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b2e0 tx=0x7f48bc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 -- start start 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e535847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53584d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e534d68f0 con 0x560e53586150 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560e5343bd50 con 0x560e53586530 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48e4419b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e534d28b0 con 0x560e53585240 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e535847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.356+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e535847c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53794/0 (socket says 172.21.15.112:53794) 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d37fe700 1 -- 172.21.15.112:0/2595237446 learned_addr learned my addr 172.21.15.112:0/2595237446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48cbfff700 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53584d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d3fff700 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d37fe700 1 -- 172.21.15.112:0/2595237446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 msgr2=0x560e53594e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d37fe700 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d37fe700 1 -- 172.21.15.112:0/2595237446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 msgr2=0x560e53584d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d37fe700 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53584d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d37fe700 1 -- 172.21.15.112:0/2595237446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e53423880 con 0x560e53586150 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d3fff700 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48d37fe700 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e535847c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009230 tx=0x7f48c40a9340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91edbf3657948a7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48c9ffb700 1 -- 172.21.15.112:0/2595237446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40b22b0 con 0x560e53586150 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48c9ffb700 1 -- 172.21.15.112:0/2595237446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c40aa200 con 0x560e53586150 2024-05-10T10:39:57.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.357+0000 7f48c9ffb700 1 -- 172.21.15.112:0/2595237446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4010430 con 0x560e53586150 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 -- 172.21.15.112:0/2595237446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 msgr2=0x560e535847c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e535847c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009230 tx=0x7f48c40a9340 comp rx=0 tx=0).stop 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 -- 172.21.15.112:0/2595237446 shutdown_connections 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e535847c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 --2- 172.21.15.112:0/2595237446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53584d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 -- 172.21.15.112:0/2595237446 >> 172.21.15.112:0/2595237446 conn(0x560e534b0b20 msgr2=0x560e5357f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 -- 172.21.15.112:0/2595237446 shutdown_connections 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 -- 172.21.15.112:0/2595237446 wait complete. 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.358+0000 7f48e4419b80 1 -- start start 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 1 Processor -- start 2024-05-10T10:39:57.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e53591e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586530 0x560e535923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e535957c0 con 0x560e53586530 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560e5348fe80 con 0x560e53586150 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e53595940 con 0x560e53585240 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48e4419b80 10 client.0 _ll_register_callbacks cb 0x560e5346a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586530 0x560e535923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586530 0x560e535923d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53808/0 (socket says 172.21.15.112:53808) 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 -- 172.21.15.112:0/575972067 learned_addr learned my addr 172.21.15.112:0/575972067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48cbfff700 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e53591e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d37fe700 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 -- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 msgr2=0x560e53594af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 -- 172.21.15.112:0/575972067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 msgr2=0x560e53591e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e53591e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 -- 172.21.15.112:0/575972067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e5348d710 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48d3fff700 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586530 0x560e535923d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f48cc007870 tx=0x7f48cc007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3316f415873390 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.359+0000 7f48b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48b2ffd700 1 -- 172.21.15.112:0/575972067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc019020 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48b2ffd700 1 -- 172.21.15.112:0/575972067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f48cc010670 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c40aa200 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48b2ffd700 1 -- 172.21.15.112:0/575972067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f48cc024050 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 10 client.15258 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e535a0020 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48b2ffd700 1 -- 172.21.15.112:0/575972067 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c40aa200 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48b2ffd700 1 -- 172.21.15.112:0/575972067 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f48cc015a10 con 0x560e53586530 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:57.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 20 client.15258 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e535a26b0 0x560e535a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 4 client.15258 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.360+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e534e42d0 con 0x560e535a26b0 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.361+0000 7f48d37fe700 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e535a26b0 0x560e535a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.361+0000 7f48d37fe700 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e535a26b0 0x560e535a3b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbbc68c22e15a42 server_cookie=dd2477ff95f62354 in_seq=0 out_seq=0 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.361+0000 7f48b2ffd700 10 client.15258 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48b2ffd700 1 -- 172.21.15.112:0/575972067 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e534e42d0 con 0x560e535a26b0 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48b2ffd700 10 client.15258 handle_command_reply: tid=0 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 client.15258 shutdown 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586530 msgr2=0x560e535923d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586530 0x560e535923d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f48cc007870 tx=0x7f48cc007b80 comp rx=0 tx=0).stop 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 shutdown_connections 2024-05-10T10:39:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e535a26b0 0x560e535a3b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586530 0x560e535923d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48b2ffd700 0 client.15258 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53594af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 --2- 172.21.15.112:0/575972067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e53591e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 >> 172.21.15.112:0/575972067 conn(0x560e534b0b20 msgr2=0x560e53579ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 shutdown_connections 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 1 -- 172.21.15.112:0/575972067 wait complete. 2024-05-10T10:39:57.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.362+0000 7f48e4419b80 20 client.15258 trim_cache size 0 max 16384 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 Processor -- start 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 -- start start 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53590df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e5358d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53579bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e535a1e60 con 0x560e53586150 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560e535a2530 con 0x560e53586530 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48e4419b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e535a2c00 con 0x560e53585240 2024-05-10T10:39:57.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e5358d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e5358d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53810/0 (socket says 172.21.15.112:53810) 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 -- 172.21.15.112:0/4234058243 learned_addr learned my addr 172.21.15.112:0/4234058243 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 -- 172.21.15.112:0/4234058243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 msgr2=0x560e53590df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48d3fff700 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53579bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53590df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 -- 172.21.15.112:0/4234058243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 msgr2=0x560e53579bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53579bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.383+0000 7f48cbfff700 1 -- 172.21.15.112:0/4234058243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e5357bc30 con 0x560e53586150 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48cbfff700 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e5358d680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009990 tx=0x7f48bc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc881972a4819ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48c97fa700 1 -- 172.21.15.112:0/4234058243 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc036070 con 0x560e53586150 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48c97fa700 1 -- 172.21.15.112:0/4234058243 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48bc010e20 con 0x560e53586150 2024-05-10T10:39:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48c97fa700 1 -- 172.21.15.112:0/4234058243 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc023660 con 0x560e53586150 2024-05-10T10:39:57.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48e4419b80 1 -- 172.21.15.112:0/4234058243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 msgr2=0x560e5358d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48e4419b80 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e5358d680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009990 tx=0x7f48bc01e000 comp rx=0 tx=0).stop 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48e4419b80 1 -- 172.21.15.112:0/4234058243 shutdown_connections 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48e4419b80 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586530 0x560e53579bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48e4419b80 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53585240 0x560e53590df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48e4419b80 1 --2- 172.21.15.112:0/4234058243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53586150 0x560e5358d680 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.384+0000 7f48e4419b80 1 -- 172.21.15.112:0/4234058243 >> 172.21.15.112:0/4234058243 conn(0x560e534b0b20 msgr2=0x560e5359dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 -- 172.21.15.112:0/4234058243 shutdown_connections 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 -- 172.21.15.112:0/4234058243 wait complete. 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 -- start start 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 Processor -- start 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53585240 0x560e535ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e535a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53586530 0x560e535a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e535a39a0 con 0x560e53585240 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560e535a32d0 con 0x560e53586150 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560e535a4070 con 0x560e53586530 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.385+0000 7f48e4419b80 10 client.0 _ll_register_callbacks cb 0x560e5346a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53585240 0x560e535ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e535a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53585240 0x560e535ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53826/0 (socket says 172.21.15.112:53826) 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48d37fe700 1 -- 172.21.15.112:0/1299901996 learned_addr learned my addr 172.21.15.112:0/1299901996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48d3fff700 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53586530 0x560e535a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48cbfff700 1 -- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53586530 msgr2=0x560e535a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48cbfff700 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53586530 0x560e535a6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48cbfff700 1 -- 172.21.15.112:0/1299901996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53585240 msgr2=0x560e535ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48cbfff700 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53585240 0x560e535ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48cbfff700 1 -- 172.21.15.112:0/1299901996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e535a7750 con 0x560e53586150 2024-05-10T10:39:57.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48d37fe700 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53585240 0x560e535ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48cbfff700 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e535a6910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0098e0 tx=0x7f48bc031a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62782f2076dcee25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48b2ffd700 1 -- 172.21.15.112:0/1299901996 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc036040 con 0x560e53586150 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48bc010e20 con 0x560e53586150 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48b2ffd700 1 -- 172.21.15.112:0/1299901996 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f48bc005030 con 0x560e53586150 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.386+0000 7f48b2ffd700 1 -- 172.21.15.112:0/1299901996 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f48bc044050 con 0x560e53586150 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 10 client.24857 fetch_fsmap learned FSMap version 29 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e5360dc50 con 0x560e53586150 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48b2ffd700 1 -- 172.21.15.112:0/1299901996 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48bc010e20 con 0x560e53586150 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48b2ffd700 1 -- 172.21.15.112:0/1299901996 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f48bc022070 con 0x560e53586150 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 10 client.24857 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 10 client.24857 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 20 client.24857 populate_metadata read hostname 'smithi112' 2024-05-10T10:39:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e53612190 0x560e53610ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 4 client.24857 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "status"}) -- 0x560e534de370 con 0x560e53612190 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.387+0000 7f48d37fe700 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e53612190 0x560e53610ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48b2ffd700 10 client.24857 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48d37fe700 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e53612190 0x560e53610ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a83ade0cc0c2bf4 server_cookie=a116ca493a1c8687 in_seq=0 out_seq=0 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48b2ffd700 1 -- 172.21.15.112:0/1299901996 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560e534de370 con 0x560e53612190 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48b2ffd700 10 client.24857 handle_command_reply: tid=0 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 client.24857 shutdown 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 msgr2=0x560e535a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e535a6910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0098e0 tx=0x7f48bc031a00 comp rx=0 tx=0).stop 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 shutdown_connections 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x560e53612190 0x560e53610ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560e53586530 0x560e535a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e53585240 0x560e535ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 --2- 172.21.15.112:0/1299901996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e53586150 0x560e535a6910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 >> 172.21.15.112:0/1299901996 conn(0x560e534b0b20 msgr2=0x560e5358d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48b2ffd700 0 client.24857 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:39:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 shutdown_connections 2024-05-10T10:39:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 1 -- 172.21.15.112:0/1299901996 wait complete. 2024-05-10T10:39:57.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.388+0000 7f48e4419b80 20 client.24857 trim_cache size 0 max 16384 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24451, 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 180.021734348, 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 278.798599373 2024-05-10T10:39:57.396 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:39:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f48c00b2880 msgr2=0x7f48c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f48c00b2880 0x7f48c00b4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b2e0 tx=0x7f48bc01a040 comp rx=0 tx=0).stop 2024-05-10T10:39:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 msgr2=0x7f48d419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d419e660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9a0 tx=0x7f48c400ed60 comp rx=0 tx=0).stop 2024-05-10T10:39:57.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 shutdown_connections 2024-05-10T10:39:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d4108400 0x7f48d41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d4102390 0x7f48d419e660 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f48c00b2880 0x7f48c00b4d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 --2- 172.21.15.112:0/871412719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102d50 0x7f48d419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 >> 172.21.15.112:0/871412719 conn(0x7f48d4079310 msgr2=0x7f48d41010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 shutdown_connections 2024-05-10T10:39:57.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:39:57.389+0000 7f48d9b86700 1 -- 172.21.15.112:0/871412719 wait complete. 2024-05-10T10:39:58.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T10:39:58.450 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:39:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:39:58 smithi112 ceph-mon[30100]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:59.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:39:58 smithi148 ceph-mon[33568]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:39:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:39:58 smithi195 ceph-mon[33773]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T10:40:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.189+0000 7f92e3f42700 1 -- 172.21.15.112:0/2945895646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103f20 msgr2=0x7f92dc110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.189+0000 7f92e3f42700 1 --2- 172.21.15.112:0/2945895646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103f20 0x7f92dc110310 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b0d0 tx=0x7f92d400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 -- 172.21.15.112:0/2945895646 shutdown_connections 2024-05-10T10:40:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 --2- 172.21.15.112:0/2945895646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103f20 0x7f92dc110310 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 --2- 172.21.15.112:0/2945895646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92dc109540 0x7f92dc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 --2- 172.21.15.112:0/2945895646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92dc103560 0x7f92dc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 -- 172.21.15.112:0/2945895646 >> 172.21.15.112:0/2945895646 conn(0x7f92dc07a660 msgr2=0x7f92dc07aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 -- 172.21.15.112:0/2945895646 shutdown_connections 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 -- 172.21.15.112:0/2945895646 wait complete. 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 Processor -- start 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.190+0000 7f92e3f42700 1 -- start start 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103560 0x7f92dc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92dc103f20 0x7f92dc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92dc109540 0x7f92dc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc113550 con 0x7f92dc103560 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1133d0 con 0x7f92dc109540 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1136d0 con 0x7f92dc103f20 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103560 0x7f92dc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103560 0x7f92dc19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53838/0 (socket says 172.21.15.112:53838) 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 -- 172.21.15.112:0/413868050 learned_addr learned my addr 172.21.15.112:0/413868050 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e14dd700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92dc103f20 0x7f92dc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e24df700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92dc109540 0x7f92dc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 -- 172.21.15.112:0/413868050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92dc103f20 msgr2=0x7f92dc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92dc103f20 0x7f92dc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 -- 172.21.15.112:0/413868050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92dc109540 msgr2=0x7f92dc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92dc109540 0x7f92dc199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 -- 172.21.15.112:0/413868050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc19a030 con 0x7f92dc103560 2024-05-10T10:40:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e1cde700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103560 0x7f92dc19f6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f92d800b530 tx=0x7f92d800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92ceffd700 1 -- 172.21.15.112:0/413868050 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8015020 con 0x7f92dc103560 2024-05-10T10:40:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92ceffd700 1 -- 172.21.15.112:0/413868050 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92d8011cc0 con 0x7f92dc103560 2024-05-10T10:40:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc19a320 con 0x7f92dc103560 2024-05-10T10:40:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92ceffd700 1 -- 172.21.15.112:0/413868050 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8010dc0 con 0x7f92dc103560 2024-05-10T10:40:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.191+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1a65b0 con 0x7f92dc103560 2024-05-10T10:40:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.194+0000 7f92ceffd700 1 -- 172.21.15.112:0/413868050 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f92d8018020 con 0x7f92dc103560 2024-05-10T10:40:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.195+0000 7f92ceffd700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f92c80b2880 0x7f92c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.195+0000 7f92e14dd700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f92c80b2880 0x7f92c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.195+0000 7f92ceffd700 1 -- 172.21.15.112:0/413868050 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f92d80e1e10 con 0x7f92dc103560 2024-05-10T10:40:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.195+0000 7f92ee7d5b80 1 Processor -- start 2024-05-10T10:40:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.195+0000 7f92e14dd700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f92c80b2880 0x7f92c80b4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f92dc07acd0 tx=0x7f92d000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.195+0000 7f92ee7d5b80 1 -- start start 2024-05-10T10:40:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa6483b8f0 con 0x55fa648ef160 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa647a0d50 con 0x55fa648efa80 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa648378b0 con 0x55fa648e9d00 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648ef540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53848/0 (socket says 172.21.15.112:53848) 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 -- 172.21.15.112:0/3148722306 learned_addr learned my addr 172.21.15.112:0/3148722306 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e14dd700 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e24df700 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 -- 172.21.15.112:0/3148722306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 msgr2=0x55fa648ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ea140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 -- 172.21.15.112:0/3148722306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 msgr2=0x55fa648e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 -- 172.21.15.112:0/3148722306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa64788880 con 0x55fa648ef160 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92e1cde700 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648ef540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f92d80a8da0 tx=0x7f92d80a8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784a37673fcfbe0d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92c7fff700 1 -- 172.21.15.112:0/3148722306 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80a97a0 con 0x55fa648ef160 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92c7fff700 1 -- 172.21.15.112:0/3148722306 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92d80a90a0 con 0x55fa648ef160 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.196+0000 7f92c7fff700 1 -- 172.21.15.112:0/3148722306 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80b1c70 con 0x55fa648ef160 2024-05-10T10:40:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/3148722306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 msgr2=0x55fa648ef540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648ef540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f92d80a8da0 tx=0x7f92d80a8b20 comp rx=0 tx=0).stop 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/3148722306 shutdown_connections 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648ef540 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/3148722306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/3148722306 >> 172.21.15.112:0/3148722306 conn(0x55fa64815b20 msgr2=0x55fa648e4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/3148722306 shutdown_connections 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.197+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/3148722306 wait complete. 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 -- start start 2024-05-10T10:40:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 Processor -- start 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa648f6340 con 0x55fa648ef160 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa647f4e80 con 0x55fa648efa80 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa648f64c0 con 0x55fa648e9d00 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92ee7d5b80 10 client.0 _ll_register_callbacks cb 0x55fa647cf560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92e14dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92e14dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53852/0 (socket says 172.21.15.112:53852) 2024-05-10T10:40:00.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92e14dd700 1 -- 172.21.15.112:0/416049558 learned_addr learned my addr 172.21.15.112:0/416049558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.198+0000 7f92e1cde700 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92e24df700 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92e14dd700 1 -- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 msgr2=0x55fa648f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92e14dd700 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92e14dd700 1 -- 172.21.15.112:0/416049558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 msgr2=0x55fa648f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92e14dd700 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92e14dd700 1 -- 172.21.15.112:0/416049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa647f2710 con 0x55fa648ef160 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92e14dd700 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648f2a90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f92d0005950 tx=0x7f92d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e311c0073868d213 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92c57fa700 1 -- 172.21.15.112:0/416049558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0016070 con 0x55fa648ef160 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92c57fa700 1 -- 172.21.15.112:0/416049558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f92d0004a30 con 0x55fa648ef160 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d80a90a0 con 0x55fa648ef160 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92c57fa700 1 -- 172.21.15.112:0/416049558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92d0030050 con 0x55fa648ef160 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92ee7d5b80 10 client.15276 fetch_fsmap learned FSMap version 29 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa64900bf0 con 0x55fa648ef160 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.199+0000 7f92c57fa700 1 -- 172.21.15.112:0/416049558 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92d80a90a0 con 0x55fa648ef160 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92c57fa700 1 -- 172.21.15.112:0/416049558 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92d0024030 con 0x55fa648ef160 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92ee7d5b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92ee7d5b80 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92ee7d5b80 20 client.15276 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa649032b0 0x55fa64904720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92ee7d5b80 4 client.15276 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa648492d0 con 0x55fa649032b0 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92e1cde700 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa649032b0 0x55fa64904720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92e1cde700 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa649032b0 0x55fa64904720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebfe3bfcd0339257 server_cookie=b8efb731200c13b3 in_seq=0 out_seq=0 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.200+0000 7f92c57fa700 10 client.15276 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92c57fa700 1 -- 172.21.15.112:0/416049558 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa648492d0 con 0x55fa649032b0 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92c57fa700 10 client.15276 handle_command_reply: tid=0 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 client.15276 shutdown 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 msgr2=0x55fa648f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648f2a90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f92d0005950 tx=0x7f92d00047c0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 shutdown_connections 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa649032b0 0x55fa64904720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa648f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92c57fa700 0 client.15276 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/416049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648f2a90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.201+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 >> 172.21.15.112:0/416049558 conn(0x55fa64815b20 msgr2=0x55fa648deae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.202+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 shutdown_connections 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.202+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/416049558 wait complete. 2024-05-10T10:40:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.202+0000 7f92ee7d5b80 20 client.15276 trim_cache size 0 max 16384 2024-05-10T10:40:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.222+0000 7f92ee7d5b80 1 Processor -- start 2024-05-10T10:40:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.222+0000 7f92ee7d5b80 1 -- start start 2024-05-10T10:40:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.222+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648e9d00 0x55fa648e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.222+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.222+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648efa80 0x55fa648efe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa64902a60 con 0x55fa648ef160 2024-05-10T10:40:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa64903130 con 0x55fa648e9d00 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa64903800 con 0x55fa648efa80 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648debf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53856/0 (socket says 172.21.15.112:53856) 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 -- 172.21.15.112:0/1740493664 learned_addr learned my addr 172.21.15.112:0/1740493664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e1cde700 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648e9d00 0x55fa648e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 -- 172.21.15.112:0/1740493664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648efa80 msgr2=0x55fa648efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648efa80 0x55fa648efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 -- 172.21.15.112:0/1740493664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648e9d00 msgr2=0x55fa648e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648e9d00 0x55fa648e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 -- 172.21.15.112:0/1740493664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa648e4d20 con 0x55fa648ef160 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e1cde700 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648e9d00 0x55fa648e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92e14dd700 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648debf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f92d000a9d0 tx=0x7f92d000f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1b96b8341e104 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92c77fe700 1 -- 172.21.15.112:0/1740493664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0039070 con 0x55fa648ef160 2024-05-10T10:40:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92c77fe700 1 -- 172.21.15.112:0/1740493664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92d0004cc0 con 0x55fa648ef160 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.223+0000 7f92c77fe700 1 -- 172.21.15.112:0/1740493664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0005400 con 0x55fa648ef160 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/1740493664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 msgr2=0x55fa648debf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648debf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f92d000a9d0 tx=0x7f92d000f7d0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/1740493664 shutdown_connections 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648efa80 0x55fa648efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648e9d00 0x55fa648e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/1740493664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648debf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/1740493664 >> 172.21.15.112:0/1740493664 conn(0x55fa64815b20 msgr2=0x55fa648f1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/1740493664 shutdown_connections 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/1740493664 wait complete. 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 -- start start 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.224+0000 7f92ee7d5b80 1 Processor -- start 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92ee7d5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa64971750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa649045a0 con 0x55fa648ef160 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa64903ed0 con 0x55fa648efa80 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92ee7d5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa64904c70 con 0x55fa648e9d00 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92ee7d5b80 10 client.0 _ll_register_callbacks cb 0x55fa647cf560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648de4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53868/0 (socket says 172.21.15.112:53868) 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 -- 172.21.15.112:0/2692779706 learned_addr learned my addr 172.21.15.112:0/2692779706 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e1cde700 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e24df700 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa64971750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 -- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 msgr2=0x55fa648ddf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ddf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 -- 172.21.15.112:0/2692779706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 msgr2=0x55fa64971750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa64971750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 -- 172.21.15.112:0/2692779706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa64971c90 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92e14dd700 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648de4a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f92d0037040 tx=0x7f92d001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f8c9f781466951 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.225+0000 7f92c57fa700 1 -- 172.21.15.112:0/2692779706 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0039030 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92c57fa700 1 -- 172.21.15.112:0/2692779706 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f92d00049a0 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d0004cc0 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92c57fa700 1 -- 172.21.15.112:0/2692779706 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92d004a050 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 10 client.15282 fetch_fsmap learned FSMap version 29 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa64972e10 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92c57fa700 1 -- 172.21.15.112:0/2692779706 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92d0004cc0 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92c57fa700 1 -- 172.21.15.112:0/2692779706 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92d0024040 con 0x55fa648ef160 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 10 client.15282 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 20 client.15282 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa64977790 0x55fa649764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 4 client.15282 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "status"}) -- 0x55fa64843370 con 0x55fa64977790 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.226+0000 7f92e1cde700 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa64977790 0x55fa649764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92e1cde700 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa64977790 0x55fa649764a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78357c4c1e129584 server_cookie=2545e8350c08e7da in_seq=0 out_seq=0 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92c57fa700 10 client.15282 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92c57fa700 1 -- 172.21.15.112:0/2692779706 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fa64843370 con 0x55fa64977790 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92c57fa700 10 client.15282 handle_command_reply: tid=0 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 client.15282 shutdown 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 msgr2=0x55fa648de4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648de4a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f92d0037040 tx=0x7f92d001ba90 comp rx=0 tx=0).stop 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 shutdown_connections 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fa64977790 0x55fa649764a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa648efa80 0x55fa64971750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa648e9d00 0x55fa648ddf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92c57fa700 0 client.15282 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:00.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 --2- 172.21.15.112:0/2692779706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa648ef160 0x55fa648de4a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 >> 172.21.15.112:0/2692779706 conn(0x55fa64815b20 msgr2=0x55fa648f0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 shutdown_connections 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 1 -- 172.21.15.112:0/2692779706 wait complete. 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.227+0000 7f92ee7d5b80 20 client.15282 trim_cache size 0 max 16384 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "ccd347ac-0eb7-11ef-bc98-c7b262605968", 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24469, 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-10T10:40:00.235 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-10T10:40:00.236 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-10T10:40:00.236 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 247.29730284600001, 2024-05-10T10:40:00.236 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 29, 2024-05-10T10:40:00.236 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-05-10T10:40:00.236 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:40:00.236 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 275.66579402899998 2024-05-10T10:40:00.236 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f92c80b2880 msgr2=0x7f92c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f92c80b2880 0x7f92c80b4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f92dc07acd0 tx=0x7f92d000a300 comp rx=0 tx=0).stop 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103560 msgr2=0x7f92dc19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103560 0x7f92dc19f6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f92d800b530 tx=0x7f92d800b840 comp rx=0 tx=0).stop 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 shutdown_connections 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92dc109540 0x7f92dc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92dc103560 0x7f92dc19f6d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f92c80b2880 0x7f92c80b4d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 --2- 172.21.15.112:0/413868050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92dc103f20 0x7f92dc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 >> 172.21.15.112:0/413868050 conn(0x7f92dc07a660 msgr2=0x7f92dc1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:00.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 shutdown_connections 2024-05-10T10:40:00.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:00.228+0000 7f92e3f42700 1 -- 172.21.15.112:0/413868050 wait complete. 2024-05-10T10:40:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:00 smithi148 ceph-mon[33568]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:40:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:00 smithi148 ceph-mon[33568]: overall HEALTH_OK 2024-05-10T10:40:01.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:00 smithi112 ceph-mon[30100]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:40:01.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:00 smithi112 ceph-mon[30100]: overall HEALTH_OK 2024-05-10T10:40:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:00 smithi195 ceph-mon[33773]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T10:40:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:00 smithi195 ceph-mon[33773]: overall HEALTH_OK 2024-05-10T10:40:01.091 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T10:40:01.102 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T10:40:01.102 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T10:40:01.103 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T10:40:01.103 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-10T10:40:01.121 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-10T10:40:01.121 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-10T10:40:01.121 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-10T10:40:01.121 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T10:40:01.121 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:01.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:01.122 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-05-10T10:40:01.140 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-10T10:40:01.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:01.141 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:40:01.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:01.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-05-10T10:40:01.303 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-10T10:40:01.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T10:40:01.548 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:02.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:02 smithi112 ceph-mon[30100]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:40:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:02 smithi148 ceph-mon[33568]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:40:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:02 smithi195 ceph-mon[33773]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-10T10:40:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 -- 172.21.15.112:0/629108033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 msgr2=0x7f5f5010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 --2- 172.21.15.112:0/629108033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5010f2e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b0d0 tx=0x7f5f4800b3e0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 -- 172.21.15.112:0/629108033 shutdown_connections 2024-05-10T10:40:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 --2- 172.21.15.112:0/629108033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5010f2e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 --2- 172.21.15.112:0/629108033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f50102e00 0x7f5f501031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 --2- 172.21.15.112:0/629108033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f501037b0 0x7f5f50107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 -- 172.21.15.112:0/629108033 >> 172.21.15.112:0/629108033 conn(0x7f5f50078a10 msgr2=0x7f5f500ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 -- 172.21.15.112:0/629108033 shutdown_connections 2024-05-10T10:40:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 -- 172.21.15.112:0/629108033 wait complete. 2024-05-10T10:40:03.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.400+0000 7f5f571fb700 1 Processor -- start 2024-05-10T10:40:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f571fb700 1 -- start start 2024-05-10T10:40:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f571fb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f50102e00 0x7f5f501a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f571fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f501037b0 0x7f5f501a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f571fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f571fb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501123e0 con 0x7f5f50107760 2024-05-10T10:40:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f571fb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50112260 con 0x7f5f50102e00 2024-05-10T10:40:03.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f571fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50112560 con 0x7f5f501037b0 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53890/0 (socket says 172.21.15.112:53890) 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 -- 172.21.15.112:0/1462890216 learned_addr learned my addr 172.21.15.112:0/1462890216 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f54f97700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f50102e00 0x7f5f501a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f4ffff700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f501037b0 0x7f5f501a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 -- 172.21.15.112:0/1462890216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f501037b0 msgr2=0x7f5f501a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f501037b0 0x7f5f501a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 -- 172.21.15.112:0/1462890216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f50102e00 msgr2=0x7f5f501a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f50102e00 0x7f5f501a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 -- 172.21.15.112:0/1462890216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5019c0d0 con 0x7f5f50107760 2024-05-10T10:40:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f55798700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5019b970 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009850 tx=0x7f5f48007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.401+0000 7f5f4dffb700 1 -- 172.21.15.112:0/1462890216 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4801f030 con 0x7f5f50107760 2024-05-10T10:40:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.402+0000 7f5f4dffb700 1 -- 172.21.15.112:0/1462890216 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f4801bad0 con 0x7f5f50107760 2024-05-10T10:40:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.402+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5019c360 con 0x7f5f50107760 2024-05-10T10:40:03.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.402+0000 7f5f4dffb700 1 -- 172.21.15.112:0/1462890216 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48012bb0 con 0x7f5f50107760 2024-05-10T10:40:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.402+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f501a8680 con 0x7f5f50107760 2024-05-10T10:40:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.403+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5006a190 con 0x7f5f50107760 2024-05-10T10:40:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.404+0000 7f5f4dffb700 1 -- 172.21.15.112:0/1462890216 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5f48022030 con 0x7f5f50107760 2024-05-10T10:40:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.405+0000 7f5f4dffb700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f400b2880 0x7f5f400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.405+0000 7f5f4dffb700 1 -- 172.21.15.112:0/1462890216 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5f4800f020 con 0x7f5f50107760 2024-05-10T10:40:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.405+0000 7f5f54f97700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f400b2880 0x7f5f400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:03.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.405+0000 7f5f54f97700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f400b2880 0x7f5f400b4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f44005ea0 tx=0x7f5f44005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:03.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.408+0000 7f5f4dffb700 1 -- 172.21.15.112:0/1462890216 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5f48029020 con 0x7f5f50107760 2024-05-10T10:40:03.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.660+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f5019cea0 con 0x7f5f50107760 2024-05-10T10:40:03.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.661+0000 7f5f4dffb700 1 -- 172.21.15.112:0/1462890216 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f5f480074e0 con 0x7f5f50107760 2024-05-10T10:40:03.662 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T10:40:03.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f400b2880 msgr2=0x7f5f400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:03.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f400b2880 0x7f5f400b4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f44005ea0 tx=0x7f5f44005e30 comp rx=0 tx=0).stop 2024-05-10T10:40:03.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 msgr2=0x7f5f5019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:03.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5019b970 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009850 tx=0x7f5f48007e40 comp rx=0 tx=0).stop 2024-05-10T10:40:03.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 shutdown_connections 2024-05-10T10:40:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f50107760 0x7f5f5019b970 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f50102e00 0x7f5f501a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5f400b2880 0x7f5f400b4d40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 --2- 172.21.15.112:0/1462890216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f501037b0 0x7f5f501a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 >> 172.21.15.112:0/1462890216 conn(0x7f5f50078a10 msgr2=0x7f5f50106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 shutdown_connections 2024-05-10T10:40:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:03.663+0000 7f5f571fb700 1 -- 172.21.15.112:0/1462890216 wait complete. 2024-05-10T10:40:04.468 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T10:40:04.468 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:40:04.468 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-05-10T10:40:04.468 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:40:04.468 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:04.468 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T10:40:04.469 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T10:40:04.469 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:40:04.469 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:40:04.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:04.469 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-05-10T10:40:04.487 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:04.488 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:40:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T10:40:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42033sec preferred_lft 42033sec 2024-05-10T10:40:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-05-10T10:40:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:04.489 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T10:40:04.489 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:40:04.489 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:40:04.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-05-10T10:40:04.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-05-10T10:40:04.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-05-10T10:40:04.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T10:40:04.490 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:40:04.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:40:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:04.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:04.636 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:40:04.660 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-05-10T10:40:04.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:04.662 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-05-10T10:40:04.719 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-05-10T10:40:04.719 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:40:04.719 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T10:40:04.719 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T10:40:04.719 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:40:04.719 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:40:04.719 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:40:04.719 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T10:40:04.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T10:40:04.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T10:40:04.720 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:40:04.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:40:04.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:04 smithi112 ceph-mon[30100]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:40:04.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:04 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1462890216' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T10:40:04.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:04.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:04.847 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-05-10T10:40:04.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:04.863 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-05-10T10:40:04.919 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:40:04.919 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:40:04.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:40:04.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T10:40:04.919 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:40:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:40:05.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:05.037 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T10:40:05.037 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:40:05.037 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:40:05.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T10:40:05.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T10:40:05.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T10:40:05.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T10:40:05.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T10:40:05.038 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:40:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:04 smithi148 ceph-mon[33568]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:40:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:04 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1462890216' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T10:40:05.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:04 smithi195 ceph-mon[33773]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-10T10:40:05.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:04 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1462890216' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T10:40:05.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:40:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:05.151 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:40:05.152 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-10T10:40:05.152 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-05-10T10:40:05.152 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T10:40:05.152 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-10T10:40:05.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:40:05.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:05.229 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-10T10:40:05.230 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:40:05.230 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:05.245 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:40:05.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:40:05.245 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:05.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-05-10T10:40:05.368 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-05-10T10:40:05.425 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T10:40:05.426 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/20486a8a595afbe58cac729b72405afc9f7f9278c279f11b397cb223eac6519a/merged 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/93590821b74976f66af12e86aa7c88893f9fcfb6f6f9500aa1082194399e78a8/merged 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1bfd503ad3c3ca647e986ba54284ed75603499362df07adb930882232816fbfb/merged 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/5a44526cfeab6ba58717c3c0494d13d2dff9698e9f9cd7173f8a68c4d50a57cd/merged 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b4b77948b3d51909ef8195ab86f2ff80b1c6f5d50102df42ebb36b7d8a193811/merged 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/d4dbc60c8323acf7497abc36913b1353445ac0d7f864b1484a3300219dde14bd/merged 2024-05-10T10:40:05.427 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/3332e641574b5261511cb2cf44d0383568df36bf4996a0ede1f607f923339ea6/merged 2024-05-10T10:40:05.428 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b8898964e652a6eebbec96c2fe5cd0c9a7a2faab9969fbcaae625fcfeeaa105d/merged 2024-05-10T10:40:05.428 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/e84f7eccad36855a7c9894cc735bd11599c694daeca9bbc56be55a850e08eb54/merged 2024-05-10T10:40:05.428 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1c026c59a648379c30b280101fc69842cd4eadb169d77e0f5e552b4e874c4e01/merged 2024-05-10T10:40:05.428 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-05-10T10:40:05.428 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:40:05.428 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:40:05.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:05.428 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-05-10T10:40:05.480 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T10:40:05.481 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2 --client_fs=cephfs) 2024-05-10T10:40:05.524 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-05-10T10:40:05.548 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T10:40:05.605 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-05-10T10:40:05.605 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-05-10T10:40:05.605 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-05-10T10:40:05.605 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-05-10T10:40:05.606 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/20486a8a595afbe58cac729b72405afc9f7f9278c279f11b397cb223eac6519a/merged 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/93590821b74976f66af12e86aa7c88893f9fcfb6f6f9500aa1082194399e78a8/merged 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1bfd503ad3c3ca647e986ba54284ed75603499362df07adb930882232816fbfb/merged 2024-05-10T10:40:05.607 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/5a44526cfeab6ba58717c3c0494d13d2dff9698e9f9cd7173f8a68c4d50a57cd/merged 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b4b77948b3d51909ef8195ab86f2ff80b1c6f5d50102df42ebb36b7d8a193811/merged 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/d4dbc60c8323acf7497abc36913b1353445ac0d7f864b1484a3300219dde14bd/merged 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/3332e641574b5261511cb2cf44d0383568df36bf4996a0ede1f607f923339ea6/merged 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/b8898964e652a6eebbec96c2fe5cd0c9a7a2faab9969fbcaae625fcfeeaa105d/merged 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/e84f7eccad36855a7c9894cc735bd11599c694daeca9bbc56be55a850e08eb54/merged 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1c026c59a648379c30b280101fc69842cd4eadb169d77e0f5e552b4e874c4e01/merged 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:40:05.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:05.609 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-05-10T10:40:05.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:2024-05-10T10:40:05.604+0000 7fbddb0ca480 -1 init, newargv = 0x559c2f1bb110 newargc=15 2024-05-10T10:40:05.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[87278]: starting ceph client 2024-05-10T10:40:05.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[87278]: starting fuse 2024-05-10T10:40:05.643 INFO:teuthology.orchestra.run.smithi112.stdout:114 2024-05-10T10:40:05.644 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-10T10:40:05.644 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.0")) 2024-05-10T10:40:05.645 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-05-10T10:40:05.763 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.0.87278.asok 2024-05-10T10:40:05.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:05.774 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87278.asok 2024-05-10T10:40:05.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:05.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87278.asok status 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "d39fa08867836ac070c47cb66c782fa559fd480b", 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "0", 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "87278", 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2" 2024-05-10T10:40:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 15294, 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 15294 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-05-10T10:40:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 877771182 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 877771182 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.15294 192.168.0.1:0/877771182", 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/877771182", 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 5, 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 29, 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 90, 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-05-10T10:40:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2024-05-10T10:40:05.913 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:40:05.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:05.922 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-05-10T10:40:05.922 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:05.939 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-05-10T10:40:05.940 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:05.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:05.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:06.007 INFO:teuthology.run_tasks:Running task exec... 2024-05-10T10:40:06.018 INFO:teuthology.task.exec:Executing custom commands... 2024-05-10T10:40:06.018 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:40:06.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 -- 172.21.15.112:0/2913533480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 msgr2=0x7f3ac4102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 --2- 172.21.15.112:0/2913533480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4102e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009ca0 tx=0x7f3ab402a8c0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 -- 172.21.15.112:0/2913533480 shutdown_connections 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 --2- 172.21.15.112:0/2913533480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac41045c0 0x7f3ac410ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 --2- 172.21.15.112:0/2913533480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac4103c00 0x7f3ac4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 --2- 172.21.15.112:0/2913533480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4102e30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 -- 172.21.15.112:0/2913533480 >> 172.21.15.112:0/2913533480 conn(0x7f3ac40fdfa0 msgr2=0x7f3ac41003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 -- 172.21.15.112:0/2913533480 shutdown_connections 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.149+0000 7f3acc464640 1 -- 172.21.15.112:0/2913533480 wait complete. 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 Processor -- start 2024-05-10T10:40:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 -- start start 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac4103c00 0x7f3ac4074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac41045c0 0x7f3ac40752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac410e430 con 0x7f3ac4102a10 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac410e2b0 con 0x7f3ac4103c00 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acc464640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac410e5b0 con 0x7f3ac41045c0 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3aca200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3aca200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4076760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46342/0 (socket says 172.21.15.112:46342) 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3aca200640 1 -- 172.21.15.112:0/3895862908 learned_addr learned my addr 172.21.15.112:0/3895862908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3ac99ff640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac4103c00 0x7f3ac4074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.150+0000 7f3acaa01640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac41045c0 0x7f3ac40752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3aca200640 1 -- 172.21.15.112:0/3895862908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac41045c0 msgr2=0x7f3ac40752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3aca200640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac41045c0 0x7f3ac40752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3aca200640 1 -- 172.21.15.112:0/3895862908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac4103c00 msgr2=0x7f3ac4074da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3aca200640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac4103c00 0x7f3ac4074da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3aca200640 1 -- 172.21.15.112:0/3895862908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac4075b70 con 0x7f3ac4102a10 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3acaa01640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac41045c0 0x7f3ac40752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3aca200640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4076760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009430 tx=0x7f3ab4003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3ac37fe640 1 -- 172.21.15.112:0/3895862908 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4004040 con 0x7f3ac4102a10 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac41a2bc0 con 0x7f3ac4102a10 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3ac37fe640 1 -- 172.21.15.112:0/3895862908 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ab4004670 con 0x7f3ac4102a10 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3ac37fe640 1 -- 172.21.15.112:0/3895862908 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4030da0 con 0x7f3ac4102a10 2024-05-10T10:40:06.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.151+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac41a3160 con 0x7f3ac4102a10 2024-05-10T10:40:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.152+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac4109d30 con 0x7f3ac4102a10 2024-05-10T10:40:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.153+0000 7f3ac37fe640 1 -- 172.21.15.112:0/3895862908 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3ab40305c0 con 0x7f3ac4102a10 2024-05-10T10:40:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.154+0000 7f3ac37fe640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3aac0b28a0 0x7f3aac0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.154+0000 7f3ac99ff640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3aac0b28a0 0x7f3aac0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.154+0000 7f3ac37fe640 1 -- 172.21.15.112:0/3895862908 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3ab4103ce0 con 0x7f3ac4102a10 2024-05-10T10:40:06.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.154+0000 7f3ac99ff640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3aac0b28a0 0x7f3aac0b4d60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40764d0 tx=0x7f3ab800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:06.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.158+0000 7f3ac37fe640 1 -- 172.21.15.112:0/3895862908 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3ab4004820 con 0x7f3ac4102a10 2024-05-10T10:40:06.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.379+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f3ac40635b0 con 0x7f3ac4102a10 2024-05-10T10:40:06.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.608+0000 7f3ac37fe640 1 -- 172.21.15.112:0/3895862908 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7f3ab40cb670 con 0x7f3ac4102a10 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3aac0b28a0 msgr2=0x7f3aac0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3aac0b28a0 0x7f3aac0b4d60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40764d0 tx=0x7f3ab800b390 comp rx=0 tx=0).stop 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 msgr2=0x7f3ac4076760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4076760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009430 tx=0x7f3ab4003ca0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 shutdown_connections 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3aac0b28a0 0x7f3aac0b4d60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac41045c0 0x7f3ac40752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac4103c00 0x7f3ac4074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 --2- 172.21.15.112:0/3895862908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ac4102a10 0x7f3ac4076760 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 >> 172.21.15.112:0/3895862908 conn(0x7f3ac40fdfa0 msgr2=0x7f3ac4106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:06.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 shutdown_connections 2024-05-10T10:40:06.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.612+0000 7f3acc464640 1 -- 172.21.15.112:0/3895862908 wait complete. 2024-05-10T10:40:06.627 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-10T10:40:06.639 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-10T10:40:06.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.824+0000 7f6cbb7af640 1 -- 172.21.15.112:0/1268584101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb4103be0 msgr2=0x7f6cb4104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.824+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/1268584101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb4103be0 0x7f6cb4104060 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009900 tx=0x7f6ca802a410 comp rx=0 tx=0).stop 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.824+0000 7f6cbb7af640 1 -- 172.21.15.112:0/1268584101 shutdown_connections 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.824+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/1268584101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6cb41045a0 0x7f6cb410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.824+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/1268584101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb4103be0 0x7f6cb4104060 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.824+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/1268584101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb41029f0 0x7f6cb4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.824+0000 7f6cbb7af640 1 -- 172.21.15.112:0/1268584101 >> 172.21.15.112:0/1268584101 conn(0x7f6cb40fdf60 msgr2=0x7f6cb41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 -- 172.21.15.112:0/1268584101 shutdown_connections 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 -- 172.21.15.112:0/1268584101 wait complete. 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 Processor -- start 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 -- start start 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb41029f0 0x7f6cb4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6cb4103be0 0x7f6cb4198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb41045a0 0x7f6cb41a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410e3d0 con 0x7f6cb41045a0 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410e250 con 0x7f6cb41029f0 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cbb7af640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410e550 con 0x7f6cb4103be0 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cb9d4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb41045a0 0x7f6cb41a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cb9d4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb41045a0 0x7f6cb41a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46356/0 (socket says 172.21.15.112:46356) 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.825+0000 7f6cb9d4c640 1 -- 172.21.15.112:0/2267620397 learned_addr learned my addr 172.21.15.112:0/2267620397 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:06.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb954b640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb41029f0 0x7f6cb4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb8d4a640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6cb4103be0 0x7f6cb4198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb9d4c640 1 -- 172.21.15.112:0/2267620397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6cb4103be0 msgr2=0x7f6cb4198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb9d4c640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6cb4103be0 0x7f6cb4198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb9d4c640 1 -- 172.21.15.112:0/2267620397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb41029f0 msgr2=0x7f6cb4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb9d4c640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb41029f0 0x7f6cb4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb9d4c640 1 -- 172.21.15.112:0/2267620397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb41a0690 con 0x7f6cb41045a0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cb9d4c640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb41045a0 0x7f6cb41a0020 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00eaa0 tx=0x7f6cac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6ca67fc640 1 -- 172.21.15.112:0/2267620397 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00cdb0 con 0x7f6cb41045a0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6ca67fc640 1 -- 172.21.15.112:0/2267620397 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cac005020 con 0x7f6cb41045a0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb41a0980 con 0x7f6cb41045a0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6ca67fc640 1 -- 172.21.15.112:0/2267620397 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0116d0 con 0x7f6cb41045a0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.826+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb41a0ed0 con 0x7f6cb41045a0 2024-05-10T10:40:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.827+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb4109d90 con 0x7f6cb41045a0 2024-05-10T10:40:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.828+0000 7f6ca67fc640 1 -- 172.21.15.112:0/2267620397 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6cac0051d0 con 0x7f6cb41045a0 2024-05-10T10:40:06.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.829+0000 7f6ca67fc640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ca00b27f0 0x7f6ca00b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.829+0000 7f6cb954b640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ca00b27f0 0x7f6ca00b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.829+0000 7f6ca67fc640 1 -- 172.21.15.112:0/2267620397 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6cac0e2b20 con 0x7f6cb41045a0 2024-05-10T10:40:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.830+0000 7f6cb954b640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ca00b27f0 0x7f6ca00b4cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4103a40 tx=0x7f6cb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:06.832+0000 7f6ca67fc640 1 -- 172.21.15.112:0/2267620397 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6cac016030 con 0x7f6cb41045a0 2024-05-10T10:40:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.019+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cb402dbf0 con 0x7f6cb41045a0 2024-05-10T10:40:07.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.021+0000 7f6ca67fc640 1 -- 172.21.15.112:0/2267620397 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f6cac0aa4b0 con 0x7f6cb41045a0 2024-05-10T10:40:07.022 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:07.023 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ca00b27f0 msgr2=0x7f6ca00b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ca00b27f0 0x7f6ca00b4cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4103a40 tx=0x7f6cb0005f90 comp rx=0 tx=0).stop 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb41045a0 msgr2=0x7f6cb41a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb41045a0 0x7f6cb41a0020 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00eaa0 tx=0x7f6cac00ef60 comp rx=0 tx=0).stop 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 shutdown_connections 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ca00b27f0 0x7f6ca00b4cb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb41045a0 0x7f6cb41a0020 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6cb4103be0 0x7f6cb4198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 --2- 172.21.15.112:0/2267620397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cb41029f0 0x7f6cb4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 >> 172.21.15.112:0/2267620397 conn(0x7f6cb40fdf60 msgr2=0x7f6cb4106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 shutdown_connections 2024-05-10T10:40:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.023+0000 7f6cbb7af640 1 -- 172.21.15.112:0/2267620397 wait complete. 2024-05-10T10:40:07.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:06 smithi112 ceph-mon[30100]: pgmap v426: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:40:07.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:06 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3895862908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T10:40:07.036 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:30:55.269138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T10:35:46.170078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T10:35:49.831849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T10:35:59.956882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T10:40:07.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:40:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:06 smithi148 ceph-mon[33568]: pgmap v426: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:40:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:06 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3895862908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T10:40:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:06 smithi195 ceph-mon[33773]: pgmap v426: 129 pgs: 129 active+clean; 649 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-10T10:40:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:06 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3895862908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T10:40:07.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 -- 172.21.15.112:0/510445546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec1065a0 msgr2=0x7f28ec106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/510445546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec1065a0 0x7f28ec106a20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009900 tx=0x7f28e402a410 comp rx=0 tx=0).stop 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 -- 172.21.15.112:0/510445546 shutdown_connections 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/510445546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec1065a0 0x7f28ec106a20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/510445546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ec103bf0 0x7f28ec106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/510445546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec101280 0x7f28ec1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 -- 172.21.15.112:0/510445546 >> 172.21.15.112:0/510445546 conn(0x7f28ec0fcea0 msgr2=0x7f28ec0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 -- 172.21.15.112:0/510445546 shutdown_connections 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.139+0000 7f28f3aa0640 1 -- 172.21.15.112:0/510445546 wait complete. 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 Processor -- start 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 -- start start 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec101280 0x7f28ec19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec103bf0 0x7f28ec19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ec1065a0 0x7f28ec1a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec114a10 con 0x7f28ec101280 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec114890 con 0x7f28ec103bf0 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f3aa0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec114b90 con 0x7f28ec1065a0 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec101280 0x7f28ec19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec101280 0x7f28ec19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46376/0 (socket says 172.21.15.112:46376) 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 -- 172.21.15.112:0/3586029578 learned_addr learned my addr 172.21.15.112:0/3586029578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f103b640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec103bf0 0x7f28ec19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 -- 172.21.15.112:0/3586029578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ec1065a0 msgr2=0x7f28ec1a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ec1065a0 0x7f28ec1a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 -- 172.21.15.112:0/3586029578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec103bf0 msgr2=0x7f28ec19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec103bf0 0x7f28ec19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.140+0000 7f28f183c640 1 -- 172.21.15.112:0/3586029578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec1a6d00 con 0x7f28ec101280 2024-05-10T10:40:07.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.141+0000 7f28f183c640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec101280 0x7f28ec19f130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f28e800eab0 tx=0x7f28e800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:07.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.141+0000 7f28e2ffd640 1 -- 172.21.15.112:0/3586029578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800ce40 con 0x7f28ec101280 2024-05-10T10:40:07.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.141+0000 7f28e2ffd640 1 -- 172.21.15.112:0/3586029578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28e8011900 con 0x7f28ec101280 2024-05-10T10:40:07.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.141+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec1a6ff0 con 0x7f28ec101280 2024-05-10T10:40:07.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.141+0000 7f28e2ffd640 1 -- 172.21.15.112:0/3586029578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8010990 con 0x7f28ec101280 2024-05-10T10:40:07.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.141+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a7460 con 0x7f28ec101280 2024-05-10T10:40:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.142+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec0fe990 con 0x7f28ec101280 2024-05-10T10:40:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.143+0000 7f28e2ffd640 1 -- 172.21.15.112:0/3586029578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f28e8011470 con 0x7f28ec101280 2024-05-10T10:40:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.144+0000 7f28e2ffd640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28d80b27f0 0x7f28d80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.144+0000 7f28f103b640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28d80b27f0 0x7f28d80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.144+0000 7f28e2ffd640 1 -- 172.21.15.112:0/3586029578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f28e80e2bc0 con 0x7f28ec101280 2024-05-10T10:40:07.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.144+0000 7f28f103b640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28d80b27f0 0x7f28d80b4cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f28ec1a05f0 tx=0x7f28dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.148+0000 7f28e2ffd640 1 -- 172.21.15.112:0/3586029578 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f28e8016030 con 0x7f28ec101280 2024-05-10T10:40:07.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.328+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28ec0636b0 con 0x7f28ec101280 2024-05-10T10:40:07.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.328+0000 7f28e2ffd640 1 -- 172.21.15.112:0/3586029578 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f28e80aa550 con 0x7f28ec101280 2024-05-10T10:40:07.330 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-05-10T10:40:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28d80b27f0 msgr2=0x7f28d80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28d80b27f0 0x7f28d80b4cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f28ec1a05f0 tx=0x7f28dc00b390 comp rx=0 tx=0).stop 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec101280 msgr2=0x7f28ec19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec101280 0x7f28ec19f130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f28e800eab0 tx=0x7f28e800ef70 comp rx=0 tx=0).stop 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 shutdown_connections 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28d80b27f0 0x7f28d80b4cb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ec1065a0 0x7f28ec1a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec103bf0 0x7f28ec19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 --2- 172.21.15.112:0/3586029578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec101280 0x7f28ec19f130 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.330+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 >> 172.21.15.112:0/3586029578 conn(0x7f28ec0fcea0 msgr2=0x7f28ec104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.331+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 shutdown_connections 2024-05-10T10:40:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.331+0000 7f28f3aa0640 1 -- 172.21.15.112:0/3586029578 wait complete. 2024-05-10T10:40:07.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T10:40:07.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 -- 172.21.15.112:0/2812482232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c102b00 msgr2=0x7f5c7c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 --2- 172.21.15.112:0/2812482232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c102b00 0x7f5c7c102f80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009900 tx=0x7f5c6402a410 comp rx=0 tx=0).stop 2024-05-10T10:40:07.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 -- 172.21.15.112:0/2812482232 shutdown_connections 2024-05-10T10:40:07.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 --2- 172.21.15.112:0/2812482232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c7c1034c0 0x7f5c7c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 --2- 172.21.15.112:0/2812482232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c102b00 0x7f5c7c102f80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 --2- 172.21.15.112:0/2812482232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c7c101910 0x7f5c7c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 -- 172.21.15.112:0/2812482232 >> 172.21.15.112:0/2812482232 conn(0x7f5c7c0fce80 msgr2=0x7f5c7c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 -- 172.21.15.112:0/2812482232 shutdown_connections 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.447+0000 7f5c82609640 1 -- 172.21.15.112:0/2812482232 wait complete. 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 Processor -- start 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 -- start start 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c101910 0x7f5c7c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c7c102b00 0x7f5c7c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c7c1034c0 0x7f5c7c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c10d2d0 con 0x7f5c7c101910 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c10d150 con 0x7f5c7c1034c0 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c82609640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c10d450 con 0x7f5c7c102b00 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c101910 0x7f5c7c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c101910 0x7f5c7c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46386/0 (socket says 172.21.15.112:46386) 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 -- 172.21.15.112:0/3316862069 learned_addr learned my addr 172.21.15.112:0/3316862069 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c73fff640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c7c102b00 0x7f5c7c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c80ba6640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c7c1034c0 0x7f5c7c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 -- 172.21.15.112:0/3316862069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c7c102b00 msgr2=0x7f5c7c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c7c102b00 0x7f5c7c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 -- 172.21.15.112:0/3316862069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c7c1034c0 msgr2=0x7f5c7c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c7c1034c0 0x7f5c7c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 -- 172.21.15.112:0/3316862069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c1a0800 con 0x7f5c7c101910 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c73fff640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c7c102b00 0x7f5c7c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c80ba6640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c7c1034c0 0x7f5c7c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.448+0000 7f5c7bfff640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c101910 0x7f5c7c198be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c970 tx=0x7f5c6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.449+0000 7f5c79ffb640 1 -- 172.21.15.112:0/3316862069 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004220 con 0x7f5c7c101910 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.449+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c1a0af0 con 0x7f5c7c101910 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.449+0000 7f5c79ffb640 1 -- 172.21.15.112:0/3316862069 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c6c007e40 con 0x7f5c7c101910 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.449+0000 7f5c79ffb640 1 -- 172.21.15.112:0/3316862069 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00f450 con 0x7f5c7c101910 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.449+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c1a1040 con 0x7f5c7c101910 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.450+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c1a1210 con 0x7f5c7c101910 2024-05-10T10:40:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.451+0000 7f5c79ffb640 1 -- 172.21.15.112:0/3316862069 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5c6c003610 con 0x7f5c7c101910 2024-05-10T10:40:07.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.452+0000 7f5c79ffb640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c680b27f0 0x7f5c680b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.452+0000 7f5c73fff640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c680b27f0 0x7f5c680b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.452+0000 7f5c79ffb640 1 -- 172.21.15.112:0/3316862069 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c6c0e2790 con 0x7f5c7c101910 2024-05-10T10:40:07.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.452+0000 7f5c73fff640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c680b27f0 0x7f5c680b4cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c640096c0 tx=0x7f5c64009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.455+0000 7f5c79ffb640 1 -- 172.21.15.112:0/3316862069 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5c6c014030 con 0x7f5c7c101910 2024-05-10T10:40:07.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.637+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5c7c1a13e0 con 0x7f5c7c101910 2024-05-10T10:40:07.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.637+0000 7f5c79ffb640 1 -- 172.21.15.112:0/3316862069 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f5c6c0aa0c0 con 0x7f5c7c101910 2024-05-10T10:40:07.639 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c680b27f0 msgr2=0x7f5c680b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c680b27f0 0x7f5c680b4cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c640096c0 tx=0x7f5c64009650 comp rx=0 tx=0).stop 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c101910 msgr2=0x7f5c7c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c101910 0x7f5c7c198be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c970 tx=0x7f5c6c00ce30 comp rx=0 tx=0).stop 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 shutdown_connections 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c680b27f0 0x7f5c680b4cb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c7c1034c0 0x7f5c7c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c7c102b00 0x7f5c7c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 --2- 172.21.15.112:0/3316862069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c7c101910 0x7f5c7c198be0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.639+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 >> 172.21.15.112:0/3316862069 conn(0x7f5c7c0fce80 msgr2=0x7f5c7c06b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:07.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.640+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 shutdown_connections 2024-05-10T10:40:07.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.640+0000 7f5c82609640 1 -- 172.21.15.112:0/3316862069 wait complete. 2024-05-10T10:40:07.652 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T10:40:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.755+0000 7fda797f7640 1 -- 172.21.15.112:0/958641077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 msgr2=0x7fda74102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.755+0000 7fda797f7640 1 --2- 172.21.15.112:0/958641077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda74102f80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fda68009900 tx=0x7fda6802a410 comp rx=0 tx=0).stop 2024-05-10T10:40:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.755+0000 7fda797f7640 1 -- 172.21.15.112:0/958641077 shutdown_connections 2024-05-10T10:40:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.755+0000 7fda797f7640 1 --2- 172.21.15.112:0/958641077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda741034c0 0x7fda74109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.755+0000 7fda797f7640 1 --2- 172.21.15.112:0/958641077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda74102f80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.755+0000 7fda797f7640 1 --2- 172.21.15.112:0/958641077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda74101910 0x7fda74101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.755+0000 7fda797f7640 1 -- 172.21.15.112:0/958641077 >> 172.21.15.112:0/958641077 conn(0x7fda740fcea0 msgr2=0x7fda740ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 -- 172.21.15.112:0/958641077 shutdown_connections 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 -- 172.21.15.112:0/958641077 wait complete. 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 Processor -- start 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 -- start start 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda74101910 0x7fda7410c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda7410a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda741034c0 0x7fda7410ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda7410e410 con 0x7fda74102b00 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda7410e290 con 0x7fda741034c0 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda797f7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fda7410e590 con 0x7fda74101910 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.756+0000 7fda727fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda7410a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda7410a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46402/0 (socket says 172.21.15.112:46402) 2024-05-10T10:40:07.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 -- 172.21.15.112:0/167400750 learned_addr learned my addr 172.21.15.112:0/167400750 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda737fe640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda741034c0 0x7fda7410ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda72ffd640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda74101910 0x7fda7410c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 -- 172.21.15.112:0/167400750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda74101910 msgr2=0x7fda7410c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda74101910 0x7fda7410c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 -- 172.21.15.112:0/167400750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda741034c0 msgr2=0x7fda7410ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda741034c0 0x7fda7410ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 -- 172.21.15.112:0/167400750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7410b6a0 con 0x7fda74102b00 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda727fc640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda7410a8d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fda68009230 tx=0x7fda680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda5bfff640 1 -- 172.21.15.112:0/167400750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6803f030 con 0x7fda74102b00 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda5bfff640 1 -- 172.21.15.112:0/167400750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda68003c90 con 0x7fda74102b00 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda741a2c90 con 0x7fda74102b00 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda5bfff640 1 -- 172.21.15.112:0/167400750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68035070 con 0x7fda74102b00 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.757+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda741a3190 con 0x7fda74102b00 2024-05-10T10:40:07.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.758+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda74108d90 con 0x7fda74102b00 2024-05-10T10:40:07.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.759+0000 7fda5bfff640 1 -- 172.21.15.112:0/167400750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fda68042030 con 0x7fda74102b00 2024-05-10T10:40:07.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.760+0000 7fda5bfff640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda5c0b27f0 0x7fda5c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:07.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.761+0000 7fda72ffd640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda5c0b27f0 0x7fda5c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:07.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.761+0000 7fda5bfff640 1 -- 172.21.15.112:0/167400750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fda68103270 con 0x7fda74102b00 2024-05-10T10:40:07.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.761+0000 7fda72ffd640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda5c0b27f0 0x7fda5c0b4cb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda640058c0 tx=0x7fda6400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:07.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.764+0000 7fda5bfff640 1 -- 172.21.15.112:0/167400750 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fda680caba0 con 0x7fda74102b00 2024-05-10T10:40:07.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.947+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fda7402dce0 con 0x7fda74102b00 2024-05-10T10:40:07.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.947+0000 7fda5bfff640 1 -- 172.21.15.112:0/167400750 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fda680ca4a0 con 0x7fda74102b00 2024-05-10T10:40:07.949 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-05-10T10:40:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda5c0b27f0 msgr2=0x7fda5c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda5c0b27f0 0x7fda5c0b4cb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda640058c0 tx=0x7fda6400a4c0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 msgr2=0x7fda7410a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda7410a8d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fda68009230 tx=0x7fda680042f0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 shutdown_connections 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda5c0b27f0 0x7fda5c0b4cb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda741034c0 0x7fda7410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda74102b00 0x7fda7410a8d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 --2- 172.21.15.112:0/167400750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda74101910 0x7fda7410c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 >> 172.21.15.112:0/167400750 conn(0x7fda740fcea0 msgr2=0x7fda74105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 shutdown_connections 2024-05-10T10:40:07.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:07.949+0000 7fda797f7640 1 -- 172.21.15.112:0/167400750 wait complete. 2024-05-10T10:40:07.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:07 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3895862908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T10:40:07.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:07 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:40:07.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:07 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2267620397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:40:07.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:07 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3586029578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:40:07.962 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T10:40:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:07 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3895862908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T10:40:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:07 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:40:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:07 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2267620397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:40:08.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:07 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3586029578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:40:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 -- 172.21.15.112:0/3225933432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540100a70 msgr2=0x7fe540100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 --2- 172.21.15.112:0/3225933432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540100a70 0x7fe540100ef0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe528009900 tx=0x7fe52802a410 comp rx=0 tx=0).stop 2024-05-10T10:40:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 -- 172.21.15.112:0/3225933432 shutdown_connections 2024-05-10T10:40:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 --2- 172.21.15.112:0/3225933432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe540101430 0x7fe54010afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 --2- 172.21.15.112:0/3225933432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540100a70 0x7fe540100ef0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 --2- 172.21.15.112:0/3225933432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe540104000 0x7fe540104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 -- 172.21.15.112:0/3225933432 >> 172.21.15.112:0/3225933432 conn(0x7fe5400fc5f0 msgr2=0x7fe5400fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 -- 172.21.15.112:0/3225933432 shutdown_connections 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.064+0000 7fe544c09640 1 -- 172.21.15.112:0/3225933432 wait complete. 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 Processor -- start 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 -- start start 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe540100a70 0x7fe540198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe540101430 0x7fe5401990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540104000 0x7fe5401a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe54010e3f0 con 0x7fe540104000 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe54010e270 con 0x7fe540100a70 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe544c09640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe54010e570 con 0x7fe540101430 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540104000 0x7fe5401a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540104000 0x7fe5401a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46416/0 (socket says 172.21.15.112:46416) 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53ed9d640 1 -- 172.21.15.112:0/2049624472 learned_addr learned my addr 172.21.15.112:0/2049624472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53e59c640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe540100a70 0x7fe540198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe537fff640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe540101430 0x7fe5401990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53ed9d640 1 -- 172.21.15.112:0/2049624472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe540101430 msgr2=0x7fe5401990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53ed9d640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe540101430 0x7fe5401990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53ed9d640 1 -- 172.21.15.112:0/2049624472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe540100a70 msgr2=0x7fe540198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.065+0000 7fe53ed9d640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe540100a70 0x7fe540198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe53ed9d640 1 -- 172.21.15.112:0/2049624472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5401a07a0 con 0x7fe540104000 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe53e59c640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe540100a70 0x7fe540198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe53ed9d640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540104000 0x7fe5401a0130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe53800eaa0 tx=0x7fe53800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe5377fe640 1 -- 172.21.15.112:0/2049624472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53800cdb0 con 0x7fe540104000 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe5377fe640 1 -- 172.21.15.112:0/2049624472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe538005020 con 0x7fe540104000 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5401a0a90 con 0x7fe540104000 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe5377fe640 1 -- 172.21.15.112:0/2049624472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380116d0 con 0x7fe540104000 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.066+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5401a1030 con 0x7fe540104000 2024-05-10T10:40:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.067+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe540109fe0 con 0x7fe540104000 2024-05-10T10:40:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.068+0000 7fe5377fe640 1 -- 172.21.15.112:0/2049624472 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe5380051d0 con 0x7fe540104000 2024-05-10T10:40:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.069+0000 7fe5377fe640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe52c0b2870 0x7fe52c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.069+0000 7fe53e59c640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe52c0b2870 0x7fe52c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.069+0000 7fe5377fe640 1 -- 172.21.15.112:0/2049624472 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe5380e2b70 con 0x7fe540104000 2024-05-10T10:40:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.070+0000 7fe53e59c640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe52c0b2870 0x7fe52c0b4d30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe5300058c0 tx=0x7fe530005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:08.075 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:07 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3895862908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T10:40:08.075 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:07 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:40:08.075 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:07 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2267620397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:40:08.075 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:07 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3586029578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:40:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.073+0000 7fe5377fe640 1 -- 172.21.15.112:0/2049624472 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe538016030 con 0x7fe540104000 2024-05-10T10:40:08.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.255+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe54002db50 con 0x7fe540104000 2024-05-10T10:40:08.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.255+0000 7fe5377fe640 1 -- 172.21.15.112:0/2049624472 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fe538054e20 con 0x7fe540104000 2024-05-10T10:40:08.257 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe52c0b2870 msgr2=0x7fe52c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe52c0b2870 0x7fe52c0b4d30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe5300058c0 tx=0x7fe530005850 comp rx=0 tx=0).stop 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540104000 msgr2=0x7fe5401a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540104000 0x7fe5401a0130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe53800eaa0 tx=0x7fe53800ef60 comp rx=0 tx=0).stop 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 shutdown_connections 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe52c0b2870 0x7fe52c0b4d30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe540104000 0x7fe5401a0130 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe540101430 0x7fe5401990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 --2- 172.21.15.112:0/2049624472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe540100a70 0x7fe540198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 >> 172.21.15.112:0/2049624472 conn(0x7fe5400fc5f0 msgr2=0x7fe5401071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 shutdown_connections 2024-05-10T10:40:08.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:08.257+0000 7fe544c09640 1 -- 172.21.15.112:0/2049624472 wait complete. 2024-05-10T10:40:08.270 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-10T10:40:08.270 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:40:08.513 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:08.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:08 smithi112 ceph-mon[30100]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T10:40:08.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:08 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3316862069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T10:40:08.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:08 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/167400750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T10:40:08.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:08 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2049624472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T10:40:09.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:08 smithi148 ceph-mon[33568]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T10:40:09.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:08 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3316862069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T10:40:09.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:08 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/167400750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T10:40:09.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:08 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2049624472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T10:40:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:08 smithi195 ceph-mon[33773]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T10:40:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:08 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3316862069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T10:40:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:08 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/167400750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T10:40:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:08 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2049624472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T10:40:10.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.388+0000 7f46316d6700 1 -- 172.21.15.112:0/1951606058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c108880 msgr2=0x7f462c110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:10.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.388+0000 7f46316d6700 1 --2- 172.21.15.112:0/1951606058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c108880 0x7f462c110400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f462400b0d0 tx=0x7f462400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.388+0000 7f46316d6700 1 -- 172.21.15.112:0/1951606058 shutdown_connections 2024-05-10T10:40:10.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 --2- 172.21.15.112:0/1951606058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c108880 0x7f462c110400 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 --2- 172.21.15.112:0/1951606058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462c1037b0 0x7f462c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 --2- 172.21.15.112:0/1951606058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f462c104160 0x7f462c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 -- 172.21.15.112:0/1951606058 >> 172.21.15.112:0/1951606058 conn(0x7f462c07a580 msgr2=0x7f462c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 -- 172.21.15.112:0/1951606058 shutdown_connections 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 -- 172.21.15.112:0/1951606058 wait complete. 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 Processor -- start 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.389+0000 7f46316d6700 1 -- start start 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f46316d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462c1037b0 0x7f462c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f46316d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c104160 0x7f462c1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f46316d6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f462c108880 0x7f462c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f46316d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f462c113500 con 0x7f462c104160 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f46316d6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f462c113380 con 0x7f462c1037b0 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f46316d6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f462c113680 con 0x7f462c108880 2024-05-10T10:40:10.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c104160 0x7f462c1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c104160 0x7f462c1a1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46426/0 (socket says 172.21.15.112:46426) 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 -- 172.21.15.112:0/4120567738 learned_addr learned my addr 172.21.15.112:0/4120567738 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462b7fe700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f462c108880 0x7f462c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462affd700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462c1037b0 0x7f462c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 -- 172.21.15.112:0/4120567738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f462c108880 msgr2=0x7f462c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f462c108880 0x7f462c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 -- 172.21.15.112:0/4120567738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462c1037b0 msgr2=0x7f462c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462c1037b0 0x7f462c1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 -- 172.21.15.112:0/4120567738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c19c120 con 0x7f462c104160 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462affd700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462c1037b0 0x7f462c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.390+0000 7f462a7fc700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c104160 0x7f462c1a1d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f462000d980 tx=0x7f462000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:10.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.391+0000 7f4613fff700 1 -- 172.21.15.112:0/4120567738 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620016070 con 0x7f462c104160 2024-05-10T10:40:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.391+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c19c410 con 0x7f462c104160 2024-05-10T10:40:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.391+0000 7f4613fff700 1 -- 172.21.15.112:0/4120567738 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4620004830 con 0x7f462c104160 2024-05-10T10:40:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.391+0000 7f4613fff700 1 -- 172.21.15.112:0/4120567738 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620005440 con 0x7f462c104160 2024-05-10T10:40:10.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.391+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1a86d0 con 0x7f462c104160 2024-05-10T10:40:10.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.392+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c06a190 con 0x7f462c104160 2024-05-10T10:40:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.393+0000 7f4613fff700 1 -- 172.21.15.112:0/4120567738 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f46200055e0 con 0x7f462c104160 2024-05-10T10:40:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.394+0000 7f4613fff700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46140b2880 0x7f46140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.394+0000 7f462affd700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46140b2880 0x7f46140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.394+0000 7f4613fff700 1 -- 172.21.15.112:0/4120567738 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f46200e2390 con 0x7f462c104160 2024-05-10T10:40:10.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.394+0000 7f462affd700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46140b2880 0x7f46140b4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f462c19cb00 tx=0x7f461c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:10.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.398+0000 7f4613fff700 1 -- 172.21.15.112:0/4120567738 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f462001f020 con 0x7f462c104160 2024-05-10T10:40:10.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.644+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f462c1a8900 con 0x7f462c104160 2024-05-10T10:40:10.647 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:10 smithi112 ceph-mon[30100]: pgmap v428: 129 pgs: 129 active+clean; 650 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T10:40:10.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.645+0000 7f4613fff700 1 -- 172.21.15.112:0/4120567738 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f46200a9d20 con 0x7f462c104160 2024-05-10T10:40:10.648 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:10.648 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:40:06.380474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:40:10.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46140b2880 msgr2=0x7f46140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:10.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46140b2880 0x7f46140b4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f462c19cb00 tx=0x7f461c009260 comp rx=0 tx=0).stop 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c104160 msgr2=0x7f462c1a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c104160 0x7f462c1a1d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f462000d980 tx=0x7f462000dc90 comp rx=0 tx=0).stop 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 shutdown_connections 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f46140b2880 0x7f46140b4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f462c108880 0x7f462c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462c1037b0 0x7f462c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 --2- 172.21.15.112:0/4120567738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462c104160 0x7f462c1a1d40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 >> 172.21.15.112:0/4120567738 conn(0x7f462c07a580 msgr2=0x7f462c101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 shutdown_connections 2024-05-10T10:40:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:10.649+0000 7f46316d6700 1 -- 172.21.15.112:0/4120567738 wait complete. 2024-05-10T10:40:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-05-10T10:40:11.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:10 smithi148 ceph-mon[33568]: pgmap v428: 129 pgs: 129 active+clean; 650 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T10:40:11.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:10 smithi195 ceph-mon[33773]: pgmap v428: 129 pgs: 129 active+clean; 650 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T10:40:11.358 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-10T10:40:11.359 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-10T10:40:11.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:40:11.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 -- 172.21.15.112:0/1489275169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 msgr2=0x7fb6ec104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1489275169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec104430 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009900 tx=0x7fb6dc02a410 comp rx=0 tx=0).stop 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 -- 172.21.15.112:0/1489275169 shutdown_connections 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1489275169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ec101440 0x7fb6ec10aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1489275169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6ec100a80 0x7fb6ec100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1489275169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec104430 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 -- 172.21.15.112:0/1489275169 >> 172.21.15.112:0/1489275169 conn(0x7fb6ec0fc5e0 msgr2=0x7fb6ec0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 -- 172.21.15.112:0/1489275169 shutdown_connections 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.465+0000 7fb6f2589640 1 -- 172.21.15.112:0/1489275169 wait complete. 2024-05-10T10:40:11.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 Processor -- start 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 -- start start 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ec100a80 0x7fb6ec198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6ec101440 0x7fb6ec199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10e3c0 con 0x7fb6ec104010 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10e240 con 0x7fb6ec100a80 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f2589640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10e540 con 0x7fb6ec101440 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f0b26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f0b26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec1a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46450/0 (socket says 172.21.15.112:46450) 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.466+0000 7fb6f0b26640 1 -- 172.21.15.112:0/1447901833 learned_addr learned my addr 172.21.15.112:0/1447901833 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6ebfff640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ec100a80 0x7fb6ec198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f0b26640 1 -- 172.21.15.112:0/1447901833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6ec101440 msgr2=0x7fb6ec199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6eb7fe640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6ec101440 0x7fb6ec199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f0b26640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6ec101440 0x7fb6ec199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f0b26640 1 -- 172.21.15.112:0/1447901833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ec100a80 msgr2=0x7fb6ec198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f0b26640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ec100a80 0x7fb6ec198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f0b26640 1 -- 172.21.15.112:0/1447901833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec1a0710 con 0x7fb6ec104010 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6eb7fe640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6ec101440 0x7fb6ec199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f0b26640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec1a00a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400cce0 tx=0x7fb6e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6e97fa640 1 -- 172.21.15.112:0/1447901833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400f450 con 0x7fb6ec104010 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6e97fa640 1 -- 172.21.15.112:0/1447901833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e400fa80 con 0x7fb6ec104010 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6e97fa640 1 -- 172.21.15.112:0/1447901833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4018ad0 con 0x7fb6ec104010 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec1a0a00 con 0x7fb6ec104010 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.467+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec1a0f00 con 0x7fb6ec104010 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.468+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec109fa0 con 0x7fb6ec104010 2024-05-10T10:40:11.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.469+0000 7fb6e97fa640 1 -- 172.21.15.112:0/1447901833 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb6e400f5f0 con 0x7fb6ec104010 2024-05-10T10:40:11.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.470+0000 7fb6e97fa640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6d40b27f0 0x7fb6d40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.470+0000 7fb6ebfff640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6d40b27f0 0x7fb6d40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.470+0000 7fb6e97fa640 1 -- 172.21.15.112:0/1447901833 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb6e40e2d70 con 0x7fb6ec104010 2024-05-10T10:40:11.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.471+0000 7fb6ebfff640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6d40b27f0 0x7fb6d40b4cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0057c0 tx=0x7fb6dc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:11.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.474+0000 7fb6e97fa640 1 -- 172.21.15.112:0/1447901833 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb6e4014030 con 0x7fb6ec104010 2024-05-10T10:40:11.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.656+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6ec062df0 con 0x7fb6d40b27f0 2024-05-10T10:40:11.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.665+0000 7fb6e97fa640 1 -- 172.21.15.112:0/1447901833 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7fb6ec062df0 con 0x7fb6d40b27f0 2024-05-10T10:40:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:11.674 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6d40b27f0 msgr2=0x7fb6d40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6d40b27f0 0x7fb6d40b4cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0057c0 tx=0x7fb6dc031400 comp rx=0 tx=0).stop 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 msgr2=0x7fb6ec1a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec1a00a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400cce0 tx=0x7fb6e400a620 comp rx=0 tx=0).stop 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 shutdown_connections 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6d40b27f0 0x7fb6d40b4cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec104010 0x7fb6ec1a00a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6ec101440 0x7fb6ec199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 --2- 172.21.15.112:0/1447901833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ec100a80 0x7fb6ec198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 >> 172.21.15.112:0/1447901833 conn(0x7fb6ec0fc5e0 msgr2=0x7fb6ec107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 shutdown_connections 2024-05-10T10:40:11.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.669+0000 7fb6f2589640 1 -- 172.21.15.112:0/1447901833 wait complete. 2024-05-10T10:40:11.683 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-05-10T10:40:11.354232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667304,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2008,"num_read_kb":1112,"num_write":396,"num_write_kb":2315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2955541,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":827,"ondisk_log_size":827,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367712,"kb_used_data":14400,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574112,"statfs":{"total":1151940427776,"available":1151563890688,"internally_reserved":0,"allocated":14745600,"data_stored":8226716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":60,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003248"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295705+0000","last_change":"2024-05-10T10:35:50.924449+0000","last_active":"2024-05-10T10:38:33.295705+0000","last_peered":"2024-05-10T10:38:33.295705+0000","last_clean":"2024-05-10T10:38:33.295705+0000","last_became_active":"2024-05-10T10:35:50.924114+0000","last_became_peered":"2024-05-10T10:35:50.924114+0000","last_unstale":"2024-05-10T10:38:33.295705+0000","last_undegraded":"2024-05-10T10:38:33.295705+0000","last_fullsized":"2024-05-10T10:38:33.295705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:50.636501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292255+0000","last_change":"2024-05-10T10:35:50.928715+0000","last_active":"2024-05-10T10:38:33.292255+0000","last_peered":"2024-05-10T10:38:33.292255+0000","last_clean":"2024-05-10T10:38:33.292255+0000","last_became_active":"2024-05-10T10:35:50.928404+0000","last_became_peered":"2024-05-10T10:35:50.928404+0000","last_unstale":"2024-05-10T10:38:33.292255+0000","last_undegraded":"2024-05-10T10:38:33.292255+0000","last_fullsized":"2024-05-10T10:38:33.292255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:44.934022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295067+0000","last_change":"2024-05-10T10:35:50.925134+0000","last_active":"2024-05-10T10:38:33.295067+0000","last_peered":"2024-05-10T10:38:33.295067+0000","last_clean":"2024-05-10T10:38:33.295067+0000","last_became_active":"2024-05-10T10:35:50.924797+0000","last_became_peered":"2024-05-10T10:35:50.924797+0000","last_unstale":"2024-05-10T10:38:33.295067+0000","last_undegraded":"2024-05-10T10:38:33.295067+0000","last_fullsized":"2024-05-10T10:38:33.295067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:43:52.666519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.280643+0000","last_change":"2024-05-10T10:35:50.930916+0000","last_active":"2024-05-10T10:38:33.280643+0000","last_peered":"2024-05-10T10:38:33.280643+0000","last_clean":"2024-05-10T10:38:33.280643+0000","last_became_active":"2024-05-10T10:35:50.930378+0000","last_became_peered":"2024-05-10T10:35:50.930378+0000","last_unstale":"2024-05-10T10:38:33.280643+0000","last_undegraded":"2024-05-10T10:38:33.280643+0000","last_fullsized":"2024-05-10T10:38:33.280643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:07.165595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.269516+0000","last_change":"2024-05-10T10:35:50.925459+0000","last_active":"2024-05-10T10:38:33.269516+0000","last_peered":"2024-05-10T10:38:33.269516+0000","last_clean":"2024-05-10T10:38:33.269516+0000","last_became_active":"2024-05-10T10:35:50.925061+0000","last_became_peered":"2024-05-10T10:35:50.925061+0000","last_unstale":"2024-05-10T10:38:33.269516+0000","last_undegraded":"2024-05-10T10:38:33.269516+0000","last_fullsized":"2024-05-10T10:38:33.269516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:48:06.622369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290891+0000","last_change":"2024-05-10T10:35:50.932052+0000","last_active":"2024-05-10T10:38:33.290891+0000","last_peered":"2024-05-10T10:38:33.290891+0000","last_clean":"2024-05-10T10:38:33.290891+0000","last_became_active":"2024-05-10T10:35:50.931685+0000","last_became_peered":"2024-05-10T10:35:50.931685+0000","last_unstale":"2024-05-10T10:38:33.290891+0000","last_undegraded":"2024-05-10T10:38:33.290891+0000","last_fullsized":"2024-05-10T10:38:33.290891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:13:39.930917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289511+0000","last_change":"2024-05-10T10:35:50.930620+0000","last_active":"2024-05-10T10:38:33.289511+0000","last_peered":"2024-05-10T10:38:33.289511+0000","last_clean":"2024-05-10T10:38:33.289511+0000","last_became_active":"2024-05-10T10:35:50.929696+0000","last_became_peered":"2024-05-10T10:35:50.929696+0000","last_unstale":"2024-05-10T10:38:33.289511+0000","last_undegraded":"2024-05-10T10:38:33.289511+0000","last_fullsized":"2024-05-10T10:38:33.289511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:23.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296397+0000","last_change":"2024-05-10T10:35:50.937517+0000","last_active":"2024-05-10T10:38:33.296397+0000","last_peered":"2024-05-10T10:38:33.296397+0000","last_clean":"2024-05-10T10:38:33.296397+0000","last_became_active":"2024-05-10T10:35:50.937180+0000","last_became_peered":"2024-05-10T10:35:50.937180+0000","last_unstale":"2024-05-10T10:38:33.296397+0000","last_undegraded":"2024-05-10T10:38:33.296397+0000","last_fullsized":"2024-05-10T10:38:33.296397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.568509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293452+0000","last_change":"2024-05-10T10:35:50.937825+0000","last_active":"2024-05-10T10:38:33.293452+0000","last_peered":"2024-05-10T10:38:33.293452+0000","last_clean":"2024-05-10T10:38:33.293452+0000","last_became_active":"2024-05-10T10:35:50.935650+0000","last_became_peered":"2024-05-10T10:35:50.935650+0000","last_unstale":"2024-05-10T10:38:33.293452+0000","last_undegraded":"2024-05-10T10:38:33.293452+0000","last_fullsized":"2024-05-10T10:38:33.293452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:43:55.780266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285811+0000","last_change":"2024-05-10T10:35:50.933598+0000","last_active":"2024-05-10T10:38:33.285811+0000","last_peered":"2024-05-10T10:38:33.285811+0000","last_clean":"2024-05-10T10:38:33.285811+0000","last_became_active":"2024-05-10T10:35:50.932767+0000","last_became_peered":"2024-05-10T10:35:50.932767+0000","last_unstale":"2024-05-10T10:38:33.285811+0000","last_undegraded":"2024-05-10T10:38:33.285811+0000","last_fullsized":"2024-05-10T10:38:33.285811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:40.610376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287212+0000","last_change":"2024-05-10T10:35:50.934882+0000","last_active":"2024-05-10T10:38:33.287212+0000","last_peered":"2024-05-10T10:38:33.287212+0000","last_clean":"2024-05-10T10:38:33.287212+0000","last_became_active":"2024-05-10T10:35:50.934528+0000","last_became_peered":"2024-05-10T10:35:50.934528+0000","last_unstale":"2024-05-10T10:38:33.287212+0000","last_undegraded":"2024-05-10T10:38:33.287212+0000","last_fullsized":"2024-05-10T10:38:33.287212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:15.319765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289796+0000","last_change":"2024-05-10T10:36:35.110939+0000","last_active":"2024-05-10T10:38:33.289796+0000","last_peered":"2024-05-10T10:38:33.289796+0000","last_clean":"2024-05-10T10:38:33.289796+0000","last_became_active":"2024-05-10T10:36:35.110600+0000","last_became_peered":"2024-05-10T10:36:35.110600+0000","last_unstale":"2024-05-10T10:38:33.289796+0000","last_undegraded":"2024-05-10T10:38:33.289796+0000","last_fullsized":"2024-05-10T10:38:33.289796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:45:58.547301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.297102+0000","last_change":"2024-05-10T10:35:50.934161+0000","last_active":"2024-05-10T10:38:33.297102+0000","last_peered":"2024-05-10T10:38:33.297102+0000","last_clean":"2024-05-10T10:38:33.297102+0000","last_became_active":"2024-05-10T10:35:50.933831+0000","last_became_peered":"2024-05-10T10:35:50.933831+0000","last_unstale":"2024-05-10T10:38:33.297102+0000","last_undegraded":"2024-05-10T10:38:33.297102+0000","last_fullsized":"2024-05-10T10:38:33.297102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:09.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286140+0000","last_change":"2024-05-10T10:35:50.927404+0000","last_active":"2024-05-10T10:38:33.286140+0000","last_peered":"2024-05-10T10:38:33.286140+0000","last_clean":"2024-05-10T10:38:33.286140+0000","last_became_active":"2024-05-10T10:35:50.927094+0000","last_became_peered":"2024-05-10T10:35:50.927094+0000","last_unstale":"2024-05-10T10:38:33.286140+0000","last_undegraded":"2024-05-10T10:38:33.286140+0000","last_fullsized":"2024-05-10T10:38:33.286140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:25.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293207+0000","last_change":"2024-05-10T10:36:35.142063+0000","last_active":"2024-05-10T10:38:33.293207+0000","last_peered":"2024-05-10T10:38:33.293207+0000","last_clean":"2024-05-10T10:38:33.293207+0000","last_became_active":"2024-05-10T10:36:35.141685+0000","last_became_peered":"2024-05-10T10:36:35.141685+0000","last_unstale":"2024-05-10T10:38:33.293207+0000","last_undegraded":"2024-05-10T10:38:33.293207+0000","last_fullsized":"2024-05-10T10:38:33.293207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:47.284288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294424+0000","last_change":"2024-05-10T10:35:50.933172+0000","last_active":"2024-05-10T10:38:33.294424+0000","last_peered":"2024-05-10T10:38:33.294424+0000","last_clean":"2024-05-10T10:38:33.294424+0000","last_became_active":"2024-05-10T10:35:50.931864+0000","last_became_peered":"2024-05-10T10:35:50.931864+0000","last_unstale":"2024-05-10T10:38:33.294424+0000","last_undegraded":"2024-05-10T10:38:33.294424+0000","last_fullsized":"2024-05-10T10:38:33.294424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:50.620315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295591+0000","last_change":"2024-05-10T10:35:50.925952+0000","last_active":"2024-05-10T10:38:33.295591+0000","last_peered":"2024-05-10T10:38:33.295591+0000","last_clean":"2024-05-10T10:38:33.295591+0000","last_became_active":"2024-05-10T10:35:50.925610+0000","last_became_peered":"2024-05-10T10:35:50.925610+0000","last_unstale":"2024-05-10T10:38:33.295591+0000","last_undegraded":"2024-05-10T10:38:33.295591+0000","last_fullsized":"2024-05-10T10:38:33.295591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:38.287235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291553+0000","last_change":"2024-05-10T10:35:50.929379+0000","last_active":"2024-05-10T10:38:33.291553+0000","last_peered":"2024-05-10T10:38:33.291553+0000","last_clean":"2024-05-10T10:38:33.291553+0000","last_became_active":"2024-05-10T10:35:50.929082+0000","last_became_peered":"2024-05-10T10:35:50.929082+0000","last_unstale":"2024-05-10T10:38:33.291553+0000","last_undegraded":"2024-05-10T10:38:33.291553+0000","last_fullsized":"2024-05-10T10:38:33.291553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:54.097855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.283992+0000","last_change":"2024-05-10T10:35:50.937754+0000","last_active":"2024-05-10T10:38:33.283992+0000","last_peered":"2024-05-10T10:38:33.283992+0000","last_clean":"2024-05-10T10:38:33.283992+0000","last_became_active":"2024-05-10T10:35:50.935662+0000","last_became_peered":"2024-05-10T10:35:50.935662+0000","last_unstale":"2024-05-10T10:38:33.283992+0000","last_undegraded":"2024-05-10T10:38:33.283992+0000","last_fullsized":"2024-05-10T10:38:33.283992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:52.692279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571937+0000","last_change":"2024-05-10T10:36:01.989463+0000","last_active":"2024-05-10T10:37:35.571937+0000","last_peered":"2024-05-10T10:37:35.571937+0000","last_clean":"2024-05-10T10:37:35.571937+0000","last_became_active":"2024-05-10T10:36:01.988949+0000","last_became_peered":"2024-05-10T10:36:01.988949+0000","last_unstale":"2024-05-10T10:37:35.571937+0000","last_undegraded":"2024-05-10T10:37:35.571937+0000","last_fullsized":"2024-05-10T10:37:35.571937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:47.529845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":248,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:06.657735+0000","last_change":"2024-05-10T10:35:47.909033+0000","last_active":"2024-05-10T10:40:06.657735+0000","last_peered":"2024-05-10T10:40:06.657735+0000","last_clean":"2024-05-10T10:40:06.657735+0000","last_became_active":"2024-05-10T10:35:47.908680+0000","last_became_peered":"2024-05-10T10:35:47.908680+0000","last_unstale":"2024-05-10T10:40:06.657735+0000","last_undegraded":"2024-05-10T10:40:06.657735+0000","last_fullsized":"2024-05-10T10:40:06.657735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:46.642113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294649+0000","last_change":"2024-05-10T10:35:50.939233+0000","last_active":"2024-05-10T10:38:33.294649+0000","last_peered":"2024-05-10T10:38:33.294649+0000","last_clean":"2024-05-10T10:38:33.294649+0000","last_became_active":"2024-05-10T10:35:50.938949+0000","last_became_peered":"2024-05-10T10:35:50.938949+0000","last_unstale":"2024-05-10T10:38:33.294649+0000","last_undegraded":"2024-05-10T10:38:33.294649+0000","last_fullsized":"2024-05-10T10:38:33.294649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:06.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718685+0000","last_change":"2024-05-10T10:36:01.990982+0000","last_active":"2024-05-10T10:37:35.718685+0000","last_peered":"2024-05-10T10:37:35.718685+0000","last_clean":"2024-05-10T10:37:35.718685+0000","last_became_active":"2024-05-10T10:36:01.990325+0000","last_became_peered":"2024-05-10T10:36:01.990325+0000","last_unstale":"2024-05-10T10:37:35.718685+0000","last_undegraded":"2024-05-10T10:37:35.718685+0000","last_fullsized":"2024-05-10T10:37:35.718685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:18.424130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:48.901957+0000","last_change":"2024-05-10T10:35:47.911239+0000","last_active":"2024-05-10T10:37:48.901957+0000","last_peered":"2024-05-10T10:37:48.901957+0000","last_clean":"2024-05-10T10:37:48.901957+0000","last_became_active":"2024-05-10T10:35:47.910945+0000","last_became_peered":"2024-05-10T10:35:47.910945+0000","last_unstale":"2024-05-10T10:37:48.901957+0000","last_undegraded":"2024-05-10T10:37:48.901957+0000","last_fullsized":"2024-05-10T10:37:48.901957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:08:52.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294800+0000","last_change":"2024-05-10T10:35:50.923694+0000","last_active":"2024-05-10T10:38:33.294800+0000","last_peered":"2024-05-10T10:38:33.294800+0000","last_clean":"2024-05-10T10:38:33.294800+0000","last_became_active":"2024-05-10T10:35:50.923381+0000","last_became_peered":"2024-05-10T10:35:50.923381+0000","last_unstale":"2024-05-10T10:38:33.294800+0000","last_undegraded":"2024-05-10T10:38:33.294800+0000","last_fullsized":"2024-05-10T10:38:33.294800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:59.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718663+0000","last_change":"2024-05-10T10:36:01.989868+0000","last_active":"2024-05-10T10:37:35.718663+0000","last_peered":"2024-05-10T10:37:35.718663+0000","last_clean":"2024-05-10T10:37:35.718663+0000","last_became_active":"2024-05-10T10:36:01.989283+0000","last_became_peered":"2024-05-10T10:36:01.989283+0000","last_unstale":"2024-05-10T10:37:35.718663+0000","last_undegraded":"2024-05-10T10:37:35.718663+0000","last_fullsized":"2024-05-10T10:37:35.718663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:35.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570220+0000","last_change":"2024-05-10T10:35:47.915555+0000","last_active":"2024-05-10T10:37:35.570220+0000","last_peered":"2024-05-10T10:37:35.570220+0000","last_clean":"2024-05-10T10:37:35.570220+0000","last_became_active":"2024-05-10T10:35:47.912852+0000","last_became_peered":"2024-05-10T10:35:47.912852+0000","last_unstale":"2024-05-10T10:37:35.570220+0000","last_undegraded":"2024-05-10T10:37:35.570220+0000","last_fullsized":"2024-05-10T10:37:35.570220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:56:45.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291583+0000","last_change":"2024-05-10T10:35:50.934010+0000","last_active":"2024-05-10T10:38:33.291583+0000","last_peered":"2024-05-10T10:38:33.291583+0000","last_clean":"2024-05-10T10:38:33.291583+0000","last_became_active":"2024-05-10T10:35:50.933653+0000","last_became_peered":"2024-05-10T10:35:50.933653+0000","last_unstale":"2024-05-10T10:38:33.291583+0000","last_undegraded":"2024-05-10T10:38:33.291583+0000","last_fullsized":"2024-05-10T10:38:33.291583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:34.862734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718657+0000","last_change":"2024-05-10T10:36:01.990524+0000","last_active":"2024-05-10T10:37:35.718657+0000","last_peered":"2024-05-10T10:37:35.718657+0000","last_clean":"2024-05-10T10:37:35.718657+0000","last_became_active":"2024-05-10T10:36:01.990011+0000","last_became_peered":"2024-05-10T10:36:01.990011+0000","last_unstale":"2024-05-10T10:37:35.718657+0000","last_undegraded":"2024-05-10T10:37:35.718657+0000","last_fullsized":"2024-05-10T10:37:35.718657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:06.123095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570506+0000","last_change":"2024-05-10T10:35:47.910977+0000","last_active":"2024-05-10T10:37:35.570506+0000","last_peered":"2024-05-10T10:37:35.570506+0000","last_clean":"2024-05-10T10:37:35.570506+0000","last_became_active":"2024-05-10T10:35:47.910623+0000","last_became_peered":"2024-05-10T10:35:47.910623+0000","last_unstale":"2024-05-10T10:37:35.570506+0000","last_undegraded":"2024-05-10T10:37:35.570506+0000","last_fullsized":"2024-05-10T10:37:35.570506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:37.760666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296932+0000","last_change":"2024-05-10T10:35:50.937238+0000","last_active":"2024-05-10T10:38:33.296932+0000","last_peered":"2024-05-10T10:38:33.296932+0000","last_clean":"2024-05-10T10:38:33.296932+0000","last_became_active":"2024-05-10T10:35:50.936298+0000","last_became_peered":"2024-05-10T10:35:50.936298+0000","last_unstale":"2024-05-10T10:38:33.296932+0000","last_undegraded":"2024-05-10T10:38:33.296932+0000","last_fullsized":"2024-05-10T10:38:33.296932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:54.724452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571851+0000","last_change":"2024-05-10T10:36:01.987769+0000","last_active":"2024-05-10T10:37:35.571851+0000","last_peered":"2024-05-10T10:37:35.571851+0000","last_clean":"2024-05-10T10:37:35.571851+0000","last_became_active":"2024-05-10T10:36:01.987367+0000","last_became_peered":"2024-05-10T10:36:01.987367+0000","last_unstale":"2024-05-10T10:37:35.571851+0000","last_undegraded":"2024-05-10T10:37:35.571851+0000","last_fullsized":"2024-05-10T10:37:35.571851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:36.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661837+0000","last_change":"2024-05-10T10:35:47.915082+0000","last_active":"2024-05-10T10:40:08.661837+0000","last_peered":"2024-05-10T10:40:08.661837+0000","last_clean":"2024-05-10T10:40:08.661837+0000","last_became_active":"2024-05-10T10:35:47.914722+0000","last_became_peered":"2024-05-10T10:35:47.914722+0000","last_unstale":"2024-05-10T10:40:08.661837+0000","last_undegraded":"2024-05-10T10:40:08.661837+0000","last_fullsized":"2024-05-10T10:40:08.661837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:31.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285496+0000","last_change":"2024-05-10T10:35:50.926181+0000","last_active":"2024-05-10T10:38:33.285496+0000","last_peered":"2024-05-10T10:38:33.285496+0000","last_clean":"2024-05-10T10:38:33.285496+0000","last_became_active":"2024-05-10T10:35:50.925865+0000","last_became_peered":"2024-05-10T10:35:50.925865+0000","last_unstale":"2024-05-10T10:38:33.285496+0000","last_undegraded":"2024-05-10T10:38:33.285496+0000","last_fullsized":"2024-05-10T10:38:33.285496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:04.976908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.572140+0000","last_change":"2024-05-10T10:36:35.142412+0000","last_active":"2024-05-10T10:37:35.572140+0000","last_peered":"2024-05-10T10:37:35.572140+0000","last_clean":"2024-05-10T10:37:35.572140+0000","last_became_active":"2024-05-10T10:36:35.141959+0000","last_became_peered":"2024-05-10T10:36:35.141959+0000","last_unstale":"2024-05-10T10:37:35.572140+0000","last_undegraded":"2024-05-10T10:37:35.572140+0000","last_fullsized":"2024-05-10T10:37:35.572140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:20:34.133261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720063+0000","last_change":"2024-05-10T10:35:47.912826+0000","last_active":"2024-05-10T10:37:35.720063+0000","last_peered":"2024-05-10T10:37:35.720063+0000","last_clean":"2024-05-10T10:37:35.720063+0000","last_became_active":"2024-05-10T10:35:47.912468+0000","last_became_peered":"2024-05-10T10:35:47.912468+0000","last_unstale":"2024-05-10T10:37:35.720063+0000","last_undegraded":"2024-05-10T10:37:35.720063+0000","last_fullsized":"2024-05-10T10:37:35.720063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:56.482104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293989+0000","last_change":"2024-05-10T10:36:35.114920+0000","last_active":"2024-05-10T10:38:33.293989+0000","last_peered":"2024-05-10T10:38:33.293989+0000","last_clean":"2024-05-10T10:38:33.293989+0000","last_became_active":"2024-05-10T10:36:35.114546+0000","last_became_peered":"2024-05-10T10:36:35.114546+0000","last_unstale":"2024-05-10T10:38:33.293989+0000","last_undegraded":"2024-05-10T10:38:33.293989+0000","last_fullsized":"2024-05-10T10:38:33.293989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:52.454111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.048033+0000","last_change":"2024-05-10T10:36:01.990086+0000","last_active":"2024-05-10T10:38:06.048033+0000","last_peered":"2024-05-10T10:38:06.048033+0000","last_clean":"2024-05-10T10:38:06.048033+0000","last_became_active":"2024-05-10T10:36:01.989624+0000","last_became_peered":"2024-05-10T10:36:01.989624+0000","last_unstale":"2024-05-10T10:38:06.048033+0000","last_undegraded":"2024-05-10T10:38:06.048033+0000","last_fullsized":"2024-05-10T10:38:06.048033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:33.708069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564634+0000","last_change":"2024-05-10T10:35:47.905182+0000","last_active":"2024-05-10T10:37:35.564634+0000","last_peered":"2024-05-10T10:37:35.564634+0000","last_clean":"2024-05-10T10:37:35.564634+0000","last_became_active":"2024-05-10T10:35:47.904855+0000","last_became_peered":"2024-05-10T10:35:47.904855+0000","last_unstale":"2024-05-10T10:37:35.564634+0000","last_undegraded":"2024-05-10T10:37:35.564634+0000","last_fullsized":"2024-05-10T10:37:35.564634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:17:44.844156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292710+0000","last_change":"2024-05-10T10:35:50.935478+0000","last_active":"2024-05-10T10:38:33.292710+0000","last_peered":"2024-05-10T10:38:33.292710+0000","last_clean":"2024-05-10T10:38:33.292710+0000","last_became_active":"2024-05-10T10:35:50.933487+0000","last_became_peered":"2024-05-10T10:35:50.933487+0000","last_unstale":"2024-05-10T10:38:33.292710+0000","last_undegraded":"2024-05-10T10:38:33.292710+0000","last_fullsized":"2024-05-10T10:38:33.292710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:02.433423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:02.493132+0000","last_change":"2024-05-10T10:36:01.984285+0000","last_active":"2024-05-10T10:38:02.493132+0000","last_peered":"2024-05-10T10:38:02.493132+0000","last_clean":"2024-05-10T10:38:02.493132+0000","last_became_active":"2024-05-10T10:36:01.983900+0000","last_became_peered":"2024-05-10T10:36:01.983900+0000","last_unstale":"2024-05-10T10:38:02.493132+0000","last_undegraded":"2024-05-10T10:38:02.493132+0000","last_fullsized":"2024-05-10T10:38:02.493132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:53.406106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565788+0000","last_change":"2024-05-10T10:35:47.911123+0000","last_active":"2024-05-10T10:37:35.565788+0000","last_peered":"2024-05-10T10:37:35.565788+0000","last_clean":"2024-05-10T10:37:35.565788+0000","last_became_active":"2024-05-10T10:35:47.910762+0000","last_became_peered":"2024-05-10T10:35:47.910762+0000","last_unstale":"2024-05-10T10:37:35.565788+0000","last_undegraded":"2024-05-10T10:37:35.565788+0000","last_fullsized":"2024-05-10T10:37:35.565788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:07.045586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288933+0000","last_change":"2024-05-10T10:35:50.928093+0000","last_active":"2024-05-10T10:38:33.288933+0000","last_peered":"2024-05-10T10:38:33.288933+0000","last_clean":"2024-05-10T10:38:33.288933+0000","last_became_active":"2024-05-10T10:35:50.926945+0000","last_became_peered":"2024-05-10T10:35:50.926945+0000","last_unstale":"2024-05-10T10:38:33.288933+0000","last_undegraded":"2024-05-10T10:38:33.288933+0000","last_fullsized":"2024-05-10T10:38:33.288933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:11.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569138+0000","last_change":"2024-05-10T10:36:35.143043+0000","last_active":"2024-05-10T10:37:35.569138+0000","last_peered":"2024-05-10T10:37:35.569138+0000","last_clean":"2024-05-10T10:37:35.569138+0000","last_became_active":"2024-05-10T10:36:35.142739+0000","last_became_peered":"2024-05-10T10:36:35.142739+0000","last_unstale":"2024-05-10T10:37:35.569138+0000","last_undegraded":"2024-05-10T10:37:35.569138+0000","last_fullsized":"2024-05-10T10:37:35.569138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:10.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":427,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661413+0000","last_change":"2024-05-10T10:35:47.908687+0000","last_active":"2024-05-10T10:40:08.661413+0000","last_peered":"2024-05-10T10:40:08.661413+0000","last_clean":"2024-05-10T10:40:08.661413+0000","last_became_active":"2024-05-10T10:35:47.908349+0000","last_became_peered":"2024-05-10T10:35:47.908349+0000","last_unstale":"2024-05-10T10:40:08.661413+0000","last_undegraded":"2024-05-10T10:40:08.661413+0000","last_fullsized":"2024-05-10T10:40:08.661413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:48.696104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":376,"num_read_kb":376,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291371+0000","last_change":"2024-05-10T10:35:50.933729+0000","last_active":"2024-05-10T10:38:33.291371+0000","last_peered":"2024-05-10T10:38:33.291371+0000","last_clean":"2024-05-10T10:38:33.291371+0000","last_became_active":"2024-05-10T10:35:50.933109+0000","last_became_peered":"2024-05-10T10:35:50.933109+0000","last_unstale":"2024-05-10T10:38:33.291371+0000","last_undegraded":"2024-05-10T10:38:33.291371+0000","last_fullsized":"2024-05-10T10:38:33.291371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:10:38.305592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567135+0000","last_change":"2024-05-10T10:36:01.991969+0000","last_active":"2024-05-10T10:37:35.567135+0000","last_peered":"2024-05-10T10:37:35.567135+0000","last_clean":"2024-05-10T10:37:35.567135+0000","last_became_active":"2024-05-10T10:36:01.990278+0000","last_became_peered":"2024-05-10T10:36:01.990278+0000","last_unstale":"2024-05-10T10:37:35.567135+0000","last_undegraded":"2024-05-10T10:37:35.567135+0000","last_fullsized":"2024-05-10T10:37:35.567135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:33.070520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719303+0000","last_change":"2024-05-10T10:35:47.911682+0000","last_active":"2024-05-10T10:37:35.719303+0000","last_peered":"2024-05-10T10:37:35.719303+0000","last_clean":"2024-05-10T10:37:35.719303+0000","last_became_active":"2024-05-10T10:35:47.911328+0000","last_became_peered":"2024-05-10T10:35:47.911328+0000","last_unstale":"2024-05-10T10:37:35.719303+0000","last_undegraded":"2024-05-10T10:37:35.719303+0000","last_fullsized":"2024-05-10T10:37:35.719303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.361340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285082+0000","last_change":"2024-05-10T10:35:50.931541+0000","last_active":"2024-05-10T10:38:33.285082+0000","last_peered":"2024-05-10T10:38:33.285082+0000","last_clean":"2024-05-10T10:38:33.285082+0000","last_became_active":"2024-05-10T10:35:50.931244+0000","last_became_peered":"2024-05-10T10:35:50.931244+0000","last_unstale":"2024-05-10T10:38:33.285082+0000","last_undegraded":"2024-05-10T10:38:33.285082+0000","last_fullsized":"2024-05-10T10:38:33.285082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:38.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720689+0000","last_change":"2024-05-10T10:36:01.990390+0000","last_active":"2024-05-10T10:37:35.720689+0000","last_peered":"2024-05-10T10:37:35.720689+0000","last_clean":"2024-05-10T10:37:35.720689+0000","last_became_active":"2024-05-10T10:36:01.989434+0000","last_became_peered":"2024-05-10T10:36:01.989434+0000","last_unstale":"2024-05-10T10:37:35.720689+0000","last_undegraded":"2024-05-10T10:37:35.720689+0000","last_fullsized":"2024-05-10T10:37:35.720689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:53.129212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566392+0000","last_change":"2024-05-10T10:35:47.911590+0000","last_active":"2024-05-10T10:37:35.566392+0000","last_peered":"2024-05-10T10:37:35.566392+0000","last_clean":"2024-05-10T10:37:35.566392+0000","last_became_active":"2024-05-10T10:35:47.910950+0000","last_became_peered":"2024-05-10T10:35:47.910950+0000","last_unstale":"2024-05-10T10:37:35.566392+0000","last_undegraded":"2024-05-10T10:37:35.566392+0000","last_fullsized":"2024-05-10T10:37:35.566392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:18.844774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292200+0000","last_change":"2024-05-10T10:35:50.938174+0000","last_active":"2024-05-10T10:38:33.292200+0000","last_peered":"2024-05-10T10:38:33.292200+0000","last_clean":"2024-05-10T10:38:33.292200+0000","last_became_active":"2024-05-10T10:35:50.937618+0000","last_became_peered":"2024-05-10T10:35:50.937618+0000","last_unstale":"2024-05-10T10:38:33.292200+0000","last_undegraded":"2024-05-10T10:38:33.292200+0000","last_fullsized":"2024-05-10T10:38:33.292200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:26:48.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567141+0000","last_change":"2024-05-10T10:36:01.979226+0000","last_active":"2024-05-10T10:37:35.567141+0000","last_peered":"2024-05-10T10:37:35.567141+0000","last_clean":"2024-05-10T10:37:35.567141+0000","last_became_active":"2024-05-10T10:36:01.978847+0000","last_became_peered":"2024-05-10T10:36:01.978847+0000","last_unstale":"2024-05-10T10:37:35.567141+0000","last_undegraded":"2024-05-10T10:37:35.567141+0000","last_fullsized":"2024-05-10T10:37:35.567141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:10.629813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'42","reported_seq":490,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661398+0000","last_change":"2024-05-10T10:36:35.150814+0000","last_active":"2024-05-10T10:40:08.661398+0000","last_peered":"2024-05-10T10:40:08.661398+0000","last_clean":"2024-05-10T10:40:08.661398+0000","last_became_active":"2024-05-10T10:36:35.142980+0000","last_became_peered":"2024-05-10T10:36:35.142980+0000","last_unstale":"2024-05-10T10:40:08.661398+0000","last_undegraded":"2024-05-10T10:40:08.661398+0000","last_fullsized":"2024-05-10T10:40:08.661398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:17.907275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67776,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":387,"num_read_kb":260,"num_write":44,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290064+0000","last_change":"2024-05-10T10:35:50.925584+0000","last_active":"2024-05-10T10:38:33.290064+0000","last_peered":"2024-05-10T10:38:33.290064+0000","last_clean":"2024-05-10T10:38:33.290064+0000","last_became_active":"2024-05-10T10:35:50.925215+0000","last_became_peered":"2024-05-10T10:35:50.925215+0000","last_unstale":"2024-05-10T10:38:33.290064+0000","last_undegraded":"2024-05-10T10:38:33.290064+0000","last_fullsized":"2024-05-10T10:38:33.290064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:03.377110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566636+0000","last_change":"2024-05-10T10:36:01.985090+0000","last_active":"2024-05-10T10:37:35.566636+0000","last_peered":"2024-05-10T10:37:35.566636+0000","last_clean":"2024-05-10T10:37:35.566636+0000","last_became_active":"2024-05-10T10:36:01.984059+0000","last_became_peered":"2024-05-10T10:36:01.984059+0000","last_unstale":"2024-05-10T10:37:35.566636+0000","last_undegraded":"2024-05-10T10:37:35.566636+0000","last_fullsized":"2024-05-10T10:37:35.566636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:41.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571763+0000","last_change":"2024-05-10T10:35:47.914377+0000","last_active":"2024-05-10T10:37:35.571763+0000","last_peered":"2024-05-10T10:37:35.571763+0000","last_clean":"2024-05-10T10:37:35.571763+0000","last_became_active":"2024-05-10T10:35:47.913830+0000","last_became_peered":"2024-05-10T10:35:47.913830+0000","last_unstale":"2024-05-10T10:37:35.571763+0000","last_undegraded":"2024-05-10T10:37:35.571763+0000","last_fullsized":"2024-05-10T10:37:35.571763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:54:18.540952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284127+0000","last_change":"2024-05-10T10:35:50.932721+0000","last_active":"2024-05-10T10:38:33.284127+0000","last_peered":"2024-05-10T10:38:33.284127+0000","last_clean":"2024-05-10T10:38:33.284127+0000","last_became_active":"2024-05-10T10:35:50.932410+0000","last_became_peered":"2024-05-10T10:35:50.932410+0000","last_unstale":"2024-05-10T10:38:33.284127+0000","last_undegraded":"2024-05-10T10:38:33.284127+0000","last_fullsized":"2024-05-10T10:38:33.284127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:42.255534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569383+0000","last_change":"2024-05-10T10:36:35.143581+0000","last_active":"2024-05-10T10:37:35.569383+0000","last_peered":"2024-05-10T10:37:35.569383+0000","last_clean":"2024-05-10T10:37:35.569383+0000","last_became_active":"2024-05-10T10:36:35.142957+0000","last_became_peered":"2024-05-10T10:36:35.142957+0000","last_unstale":"2024-05-10T10:37:35.569383+0000","last_undegraded":"2024-05-10T10:37:35.569383+0000","last_fullsized":"2024-05-10T10:37:35.569383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:14:39.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566100+0000","last_change":"2024-05-10T10:35:47.908874+0000","last_active":"2024-05-10T10:37:35.566100+0000","last_peered":"2024-05-10T10:37:35.566100+0000","last_clean":"2024-05-10T10:37:35.566100+0000","last_became_active":"2024-05-10T10:35:47.908478+0000","last_became_peered":"2024-05-10T10:35:47.908478+0000","last_unstale":"2024-05-10T10:37:35.566100+0000","last_undegraded":"2024-05-10T10:37:35.566100+0000","last_fullsized":"2024-05-10T10:37:35.566100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:13.147762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285463+0000","last_change":"2024-05-10T10:35:50.925168+0000","last_active":"2024-05-10T10:38:33.285463+0000","last_peered":"2024-05-10T10:38:33.285463+0000","last_clean":"2024-05-10T10:38:33.285463+0000","last_became_active":"2024-05-10T10:35:50.924509+0000","last_became_peered":"2024-05-10T10:35:50.924509+0000","last_unstale":"2024-05-10T10:38:33.285463+0000","last_undegraded":"2024-05-10T10:38:33.285463+0000","last_fullsized":"2024-05-10T10:38:33.285463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:08.458195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569737+0000","last_change":"2024-05-10T10:36:35.144575+0000","last_active":"2024-05-10T10:37:35.569737+0000","last_peered":"2024-05-10T10:37:35.569737+0000","last_clean":"2024-05-10T10:37:35.569737+0000","last_became_active":"2024-05-10T10:36:35.144165+0000","last_became_peered":"2024-05-10T10:36:35.144165+0000","last_unstale":"2024-05-10T10:37:35.569737+0000","last_undegraded":"2024-05-10T10:37:35.569737+0000","last_fullsized":"2024-05-10T10:37:35.569737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:45.179163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":247,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.680223+0000","last_change":"2024-05-10T10:35:47.911208+0000","last_active":"2024-05-10T10:40:08.680223+0000","last_peered":"2024-05-10T10:40:08.680223+0000","last_clean":"2024-05-10T10:40:08.680223+0000","last_became_active":"2024-05-10T10:35:47.910820+0000","last_became_peered":"2024-05-10T10:35:47.910820+0000","last_unstale":"2024-05-10T10:40:08.680223+0000","last_undegraded":"2024-05-10T10:40:08.680223+0000","last_fullsized":"2024-05-10T10:40:08.680223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:04.804203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289463+0000","last_change":"2024-05-10T10:35:50.928195+0000","last_active":"2024-05-10T10:38:33.289463+0000","last_peered":"2024-05-10T10:38:33.289463+0000","last_clean":"2024-05-10T10:38:33.289463+0000","last_became_active":"2024-05-10T10:35:50.927832+0000","last_became_peered":"2024-05-10T10:35:50.927832+0000","last_unstale":"2024-05-10T10:38:33.289463+0000","last_undegraded":"2024-05-10T10:38:33.289463+0000","last_fullsized":"2024-05-10T10:38:33.289463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:03.071525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565054+0000","last_change":"2024-05-10T10:36:01.982483+0000","last_active":"2024-05-10T10:37:35.565054+0000","last_peered":"2024-05-10T10:37:35.565054+0000","last_clean":"2024-05-10T10:37:35.565054+0000","last_became_active":"2024-05-10T10:36:01.982117+0000","last_became_peered":"2024-05-10T10:36:01.982117+0000","last_unstale":"2024-05-10T10:37:35.565054+0000","last_undegraded":"2024-05-10T10:37:35.565054+0000","last_fullsized":"2024-05-10T10:37:35.565054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:08.145035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569474+0000","last_change":"2024-05-10T10:35:47.912929+0000","last_active":"2024-05-10T10:37:35.569474+0000","last_peered":"2024-05-10T10:37:35.569474+0000","last_clean":"2024-05-10T10:37:35.569474+0000","last_became_active":"2024-05-10T10:35:47.912575+0000","last_became_peered":"2024-05-10T10:35:47.912575+0000","last_unstale":"2024-05-10T10:37:35.569474+0000","last_undegraded":"2024-05-10T10:37:35.569474+0000","last_fullsized":"2024-05-10T10:37:35.569474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:03.235579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296070+0000","last_change":"2024-05-10T10:35:50.935967+0000","last_active":"2024-05-10T10:38:33.296070+0000","last_peered":"2024-05-10T10:38:33.296070+0000","last_clean":"2024-05-10T10:38:33.296070+0000","last_became_active":"2024-05-10T10:35:50.935635+0000","last_became_peered":"2024-05-10T10:35:50.935635+0000","last_unstale":"2024-05-10T10:38:33.296070+0000","last_undegraded":"2024-05-10T10:38:33.296070+0000","last_fullsized":"2024-05-10T10:38:33.296070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:02:53.681522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565799+0000","last_change":"2024-05-10T10:36:01.979488+0000","last_active":"2024-05-10T10:37:35.565799+0000","last_peered":"2024-05-10T10:37:35.565799+0000","last_clean":"2024-05-10T10:37:35.565799+0000","last_became_active":"2024-05-10T10:36:01.978839+0000","last_became_peered":"2024-05-10T10:36:01.978839+0000","last_unstale":"2024-05-10T10:37:35.565799+0000","last_undegraded":"2024-05-10T10:37:35.565799+0000","last_fullsized":"2024-05-10T10:37:35.565799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:20.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:02.476105+0000","last_change":"2024-05-10T10:35:47.913235+0000","last_active":"2024-05-10T10:38:02.476105+0000","last_peered":"2024-05-10T10:38:02.476105+0000","last_clean":"2024-05-10T10:38:02.476105+0000","last_became_active":"2024-05-10T10:35:47.912788+0000","last_became_peered":"2024-05-10T10:35:47.912788+0000","last_unstale":"2024-05-10T10:38:02.476105+0000","last_undegraded":"2024-05-10T10:38:02.476105+0000","last_fullsized":"2024-05-10T10:38:02.476105+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:56:10.275488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290951+0000","last_change":"2024-05-10T10:35:50.930269+0000","last_active":"2024-05-10T10:38:33.290951+0000","last_peered":"2024-05-10T10:38:33.290951+0000","last_clean":"2024-05-10T10:38:33.290951+0000","last_became_active":"2024-05-10T10:35:50.929918+0000","last_became_peered":"2024-05-10T10:35:50.929918+0000","last_unstale":"2024-05-10T10:38:33.290951+0000","last_undegraded":"2024-05-10T10:38:33.290951+0000","last_fullsized":"2024-05-10T10:38:33.290951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:31.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569677+0000","last_change":"2024-05-10T10:36:02.018506+0000","last_active":"2024-05-10T10:37:35.569677+0000","last_peered":"2024-05-10T10:37:35.569677+0000","last_clean":"2024-05-10T10:37:35.569677+0000","last_became_active":"2024-05-10T10:36:02.018004+0000","last_became_peered":"2024-05-10T10:36:02.018004+0000","last_unstale":"2024-05-10T10:37:35.569677+0000","last_undegraded":"2024-05-10T10:37:35.569677+0000","last_fullsized":"2024-05-10T10:37:35.569677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:37.404970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:52.729668+0000","last_change":"2024-05-10T10:35:47.910654+0000","last_active":"2024-05-10T10:37:52.729668+0000","last_peered":"2024-05-10T10:37:52.729668+0000","last_clean":"2024-05-10T10:37:52.729668+0000","last_became_active":"2024-05-10T10:35:47.910338+0000","last_became_peered":"2024-05-10T10:35:47.910338+0000","last_unstale":"2024-05-10T10:37:52.729668+0000","last_undegraded":"2024-05-10T10:37:52.729668+0000","last_fullsized":"2024-05-10T10:37:52.729668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:07.127574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290279+0000","last_change":"2024-05-10T10:35:50.940950+0000","last_active":"2024-05-10T10:38:33.290279+0000","last_peered":"2024-05-10T10:38:33.290279+0000","last_clean":"2024-05-10T10:38:33.290279+0000","last_became_active":"2024-05-10T10:35:50.940562+0000","last_became_peered":"2024-05-10T10:35:50.940562+0000","last_unstale":"2024-05-10T10:38:33.290279+0000","last_undegraded":"2024-05-10T10:38:33.290279+0000","last_fullsized":"2024-05-10T10:38:33.290279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:24.825900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564778+0000","last_change":"2024-05-10T10:36:01.989277+0000","last_active":"2024-05-10T10:37:35.564778+0000","last_peered":"2024-05-10T10:37:35.564778+0000","last_clean":"2024-05-10T10:37:35.564778+0000","last_became_active":"2024-05-10T10:36:01.988866+0000","last_became_peered":"2024-05-10T10:36:01.988866+0000","last_unstale":"2024-05-10T10:37:35.564778+0000","last_undegraded":"2024-05-10T10:37:35.564778+0000","last_fullsized":"2024-05-10T10:37:35.564778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:08:29.740044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":244,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:09.662717+0000","last_change":"2024-05-10T10:35:47.908332+0000","last_active":"2024-05-10T10:40:09.662717+0000","last_peered":"2024-05-10T10:40:09.662717+0000","last_clean":"2024-05-10T10:40:09.662717+0000","last_became_active":"2024-05-10T10:35:47.907933+0000","last_became_peered":"2024-05-10T10:35:47.907933+0000","last_unstale":"2024-05-10T10:40:09.662717+0000","last_undegraded":"2024-05-10T10:40:09.662717+0000","last_fullsized":"2024-05-10T10:40:09.662717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:47:48.702338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290317+0000","last_change":"2024-05-10T10:35:50.934638+0000","last_active":"2024-05-10T10:38:33.290317+0000","last_peered":"2024-05-10T10:38:33.290317+0000","last_clean":"2024-05-10T10:38:33.290317+0000","last_became_active":"2024-05-10T10:35:50.934285+0000","last_became_peered":"2024-05-10T10:35:50.934285+0000","last_unstale":"2024-05-10T10:38:33.290317+0000","last_undegraded":"2024-05-10T10:38:33.290317+0000","last_fullsized":"2024-05-10T10:38:33.290317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:24:31.179005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565003+0000","last_change":"2024-05-10T10:36:02.018468+0000","last_active":"2024-05-10T10:37:35.565003+0000","last_peered":"2024-05-10T10:37:35.565003+0000","last_clean":"2024-05-10T10:37:35.565003+0000","last_became_active":"2024-05-10T10:36:02.018087+0000","last_became_peered":"2024-05-10T10:36:02.018087+0000","last_unstale":"2024-05-10T10:37:35.565003+0000","last_undegraded":"2024-05-10T10:37:35.565003+0000","last_fullsized":"2024-05-10T10:37:35.565003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:55.712012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295463+0000","last_change":"2024-05-10T10:35:50.933660+0000","last_active":"2024-05-10T10:38:33.295463+0000","last_peered":"2024-05-10T10:38:33.295463+0000","last_clean":"2024-05-10T10:38:33.295463+0000","last_became_active":"2024-05-10T10:35:50.933230+0000","last_became_peered":"2024-05-10T10:35:50.933230+0000","last_unstale":"2024-05-10T10:38:33.295463+0000","last_undegraded":"2024-05-10T10:38:33.295463+0000","last_fullsized":"2024-05-10T10:38:33.295463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:34:54.272278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.679698+0000","last_change":"2024-05-10T10:35:47.910161+0000","last_active":"2024-05-10T10:40:08.679698+0000","last_peered":"2024-05-10T10:40:08.679698+0000","last_clean":"2024-05-10T10:40:08.679698+0000","last_became_active":"2024-05-10T10:35:47.909875+0000","last_became_peered":"2024-05-10T10:35:47.909875+0000","last_unstale":"2024-05-10T10:40:08.679698+0000","last_undegraded":"2024-05-10T10:40:08.679698+0000","last_fullsized":"2024-05-10T10:40:08.679698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:21.620921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.205272+0000","last_change":"2024-05-10T10:36:01.986755+0000","last_active":"2024-05-10T10:38:06.205272+0000","last_peered":"2024-05-10T10:38:06.205272+0000","last_clean":"2024-05-10T10:38:06.205272+0000","last_became_active":"2024-05-10T10:36:01.986340+0000","last_became_peered":"2024-05-10T10:36:01.986340+0000","last_unstale":"2024-05-10T10:38:06.205272+0000","last_undegraded":"2024-05-10T10:38:06.205272+0000","last_fullsized":"2024-05-10T10:38:06.205272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:38.434323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293573+0000","last_change":"2024-05-10T10:36:35.114218+0000","last_active":"2024-05-10T10:38:33.293573+0000","last_peered":"2024-05-10T10:38:33.293573+0000","last_clean":"2024-05-10T10:38:33.293573+0000","last_became_active":"2024-05-10T10:36:35.113876+0000","last_became_peered":"2024-05-10T10:36:35.113876+0000","last_unstale":"2024-05-10T10:38:33.293573+0000","last_undegraded":"2024-05-10T10:38:33.293573+0000","last_fullsized":"2024-05-10T10:38:33.293573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:21.415678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:55.105058+0000","last_change":"2024-05-10T10:35:47.909161+0000","last_active":"2024-05-10T10:37:55.105058+0000","last_peered":"2024-05-10T10:37:55.105058+0000","last_clean":"2024-05-10T10:37:55.105058+0000","last_became_active":"2024-05-10T10:35:47.908546+0000","last_became_peered":"2024-05-10T10:35:47.908546+0000","last_unstale":"2024-05-10T10:37:55.105058+0000","last_undegraded":"2024-05-10T10:37:55.105058+0000","last_fullsized":"2024-05-10T10:37:55.105058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:52.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719232+0000","last_change":"2024-05-10T10:34:20.980948+0000","last_active":"2024-05-10T10:37:35.719232+0000","last_peered":"2024-05-10T10:37:35.719232+0000","last_clean":"2024-05-10T10:37:35.719232+0000","last_became_active":"2024-05-10T10:34:20.975662+0000","last_became_peered":"2024-05-10T10:34:20.975662+0000","last_unstale":"2024-05-10T10:37:35.719232+0000","last_undegraded":"2024-05-10T10:37:35.719232+0000","last_fullsized":"2024-05-10T10:37:35.719232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_clean_scrub_stamp":"2024-05-10T10:30:55.419211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:33.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566301+0000","last_change":"2024-05-10T10:36:02.018578+0000","last_active":"2024-05-10T10:37:35.566301+0000","last_peered":"2024-05-10T10:37:35.566301+0000","last_clean":"2024-05-10T10:37:35.566301+0000","last_became_active":"2024-05-10T10:36:02.018126+0000","last_became_peered":"2024-05-10T10:36:02.018126+0000","last_unstale":"2024-05-10T10:37:35.566301+0000","last_undegraded":"2024-05-10T10:37:35.566301+0000","last_fullsized":"2024-05-10T10:37:35.566301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:18:17.112432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285918+0000","last_change":"2024-05-10T10:35:50.927472+0000","last_active":"2024-05-10T10:38:33.285918+0000","last_peered":"2024-05-10T10:38:33.285918+0000","last_clean":"2024-05-10T10:38:33.285918+0000","last_became_active":"2024-05-10T10:35:50.927030+0000","last_became_peered":"2024-05-10T10:35:50.927030+0000","last_unstale":"2024-05-10T10:38:33.285918+0000","last_undegraded":"2024-05-10T10:38:33.285918+0000","last_fullsized":"2024-05-10T10:38:33.285918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:15:03.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:51.203426+0000","last_change":"2024-05-10T10:35:47.912675+0000","last_active":"2024-05-10T10:37:51.203426+0000","last_peered":"2024-05-10T10:37:51.203426+0000","last_clean":"2024-05-10T10:37:51.203426+0000","last_became_active":"2024-05-10T10:35:47.912297+0000","last_became_peered":"2024-05-10T10:35:47.912297+0000","last_unstale":"2024-05-10T10:37:51.203426+0000","last_undegraded":"2024-05-10T10:37:51.203426+0000","last_fullsized":"2024-05-10T10:37:51.203426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:46.178221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.047984+0000","last_change":"2024-05-10T10:36:01.981501+0000","last_active":"2024-05-10T10:38:06.047984+0000","last_peered":"2024-05-10T10:38:06.047984+0000","last_clean":"2024-05-10T10:38:06.047984+0000","last_became_active":"2024-05-10T10:36:01.981029+0000","last_became_peered":"2024-05-10T10:36:01.981029+0000","last_unstale":"2024-05-10T10:38:06.047984+0000","last_undegraded":"2024-05-10T10:38:06.047984+0000","last_fullsized":"2024-05-10T10:38:06.047984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:18.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:52.474989+0000","last_change":"2024-05-10T10:35:47.910701+0000","last_active":"2024-05-10T10:37:52.474989+0000","last_peered":"2024-05-10T10:37:52.474989+0000","last_clean":"2024-05-10T10:37:52.474989+0000","last_became_active":"2024-05-10T10:35:47.910286+0000","last_became_peered":"2024-05-10T10:35:47.910286+0000","last_unstale":"2024-05-10T10:37:52.474989+0000","last_undegraded":"2024-05-10T10:37:52.474989+0000","last_fullsized":"2024-05-10T10:37:52.474989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:07.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286141+0000","last_change":"2024-05-10T10:35:50.926026+0000","last_active":"2024-05-10T10:38:33.286141+0000","last_peered":"2024-05-10T10:38:33.286141+0000","last_clean":"2024-05-10T10:38:33.286141+0000","last_became_active":"2024-05-10T10:35:50.925624+0000","last_became_peered":"2024-05-10T10:35:50.925624+0000","last_unstale":"2024-05-10T10:38:33.286141+0000","last_undegraded":"2024-05-10T10:38:33.286141+0000","last_fullsized":"2024-05-10T10:38:33.286141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:22:02.828681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:03.903648+0000","last_change":"2024-05-10T10:36:02.018403+0000","last_active":"2024-05-10T10:38:03.903648+0000","last_peered":"2024-05-10T10:38:03.903648+0000","last_clean":"2024-05-10T10:38:03.903648+0000","last_became_active":"2024-05-10T10:36:02.017980+0000","last_became_peered":"2024-05-10T10:36:02.017980+0000","last_unstale":"2024-05-10T10:38:03.903648+0000","last_undegraded":"2024-05-10T10:38:03.903648+0000","last_fullsized":"2024-05-10T10:38:03.903648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:07.264730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:05.971094+0000","last_change":"2024-05-10T10:37:35.577657+0000","last_active":"2024-05-10T10:40:05.971094+0000","last_peered":"2024-05-10T10:40:05.971094+0000","last_clean":"2024-05-10T10:40:05.971094+0000","last_became_active":"2024-05-10T10:37:35.570901+0000","last_became_peered":"2024-05-10T10:37:35.570901+0000","last_unstale":"2024-05-10T10:40:05.971094+0000","last_undegraded":"2024-05-10T10:40:05.971094+0000","last_fullsized":"2024-05-10T10:40:05.971094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:30.088160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293071+0000","last_change":"2024-05-10T10:35:50.935817+0000","last_active":"2024-05-10T10:38:33.293071+0000","last_peered":"2024-05-10T10:38:33.293071+0000","last_clean":"2024-05-10T10:38:33.293071+0000","last_became_active":"2024-05-10T10:35:50.935515+0000","last_became_peered":"2024-05-10T10:35:50.935515+0000","last_unstale":"2024-05-10T10:38:33.293071+0000","last_undegraded":"2024-05-10T10:38:33.293071+0000","last_fullsized":"2024-05-10T10:38:33.293071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:49.042193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571516+0000","last_change":"2024-05-10T10:36:01.986002+0000","last_active":"2024-05-10T10:37:35.571516+0000","last_peered":"2024-05-10T10:37:35.571516+0000","last_clean":"2024-05-10T10:37:35.571516+0000","last_became_active":"2024-05-10T10:36:01.985558+0000","last_became_peered":"2024-05-10T10:36:01.985558+0000","last_unstale":"2024-05-10T10:37:35.571516+0000","last_undegraded":"2024-05-10T10:37:35.571516+0000","last_fullsized":"2024-05-10T10:37:35.571516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:27.732316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:00.105594+0000","last_change":"2024-05-10T10:35:47.909254+0000","last_active":"2024-05-10T10:38:00.105594+0000","last_peered":"2024-05-10T10:38:00.105594+0000","last_clean":"2024-05-10T10:38:00.105594+0000","last_became_active":"2024-05-10T10:35:47.908370+0000","last_became_peered":"2024-05-10T10:35:47.908370+0000","last_unstale":"2024-05-10T10:38:00.105594+0000","last_undegraded":"2024-05-10T10:38:00.105594+0000","last_fullsized":"2024-05-10T10:38:00.105594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:44.643592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287742+0000","last_change":"2024-05-10T10:35:50.933001+0000","last_active":"2024-05-10T10:38:33.287742+0000","last_peered":"2024-05-10T10:38:33.287742+0000","last_clean":"2024-05-10T10:38:33.287742+0000","last_became_active":"2024-05-10T10:35:50.932010+0000","last_became_peered":"2024-05-10T10:35:50.932010+0000","last_unstale":"2024-05-10T10:38:33.287742+0000","last_undegraded":"2024-05-10T10:38:33.287742+0000","last_fullsized":"2024-05-10T10:38:33.287742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:01.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720023+0000","last_change":"2024-05-10T10:36:02.018650+0000","last_active":"2024-05-10T10:37:35.720023+0000","last_peered":"2024-05-10T10:37:35.720023+0000","last_clean":"2024-05-10T10:37:35.720023+0000","last_became_active":"2024-05-10T10:36:02.018258+0000","last_became_peered":"2024-05-10T10:36:02.018258+0000","last_unstale":"2024-05-10T10:37:35.720023+0000","last_undegraded":"2024-05-10T10:37:35.720023+0000","last_fullsized":"2024-05-10T10:37:35.720023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:33.590752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:57.007213+0000","last_change":"2024-05-10T10:35:47.910419+0000","last_active":"2024-05-10T10:37:57.007213+0000","last_peered":"2024-05-10T10:37:57.007213+0000","last_clean":"2024-05-10T10:37:57.007213+0000","last_became_active":"2024-05-10T10:35:47.909930+0000","last_became_peered":"2024-05-10T10:35:47.909930+0000","last_unstale":"2024-05-10T10:37:57.007213+0000","last_undegraded":"2024-05-10T10:37:57.007213+0000","last_fullsized":"2024-05-10T10:37:57.007213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:32.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.250953+0000","last_change":"2024-05-10T10:35:50.923207+0000","last_active":"2024-05-10T10:38:33.250953+0000","last_peered":"2024-05-10T10:38:33.250953+0000","last_clean":"2024-05-10T10:38:33.250953+0000","last_became_active":"2024-05-10T10:35:50.922866+0000","last_became_peered":"2024-05-10T10:35:50.922866+0000","last_unstale":"2024-05-10T10:38:33.250953+0000","last_undegraded":"2024-05-10T10:38:33.250953+0000","last_fullsized":"2024-05-10T10:38:33.250953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:48.096498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567305+0000","last_change":"2024-05-10T10:36:01.985399+0000","last_active":"2024-05-10T10:37:35.567305+0000","last_peered":"2024-05-10T10:37:35.567305+0000","last_clean":"2024-05-10T10:37:35.567305+0000","last_became_active":"2024-05-10T10:36:01.985016+0000","last_became_peered":"2024-05-10T10:36:01.985016+0000","last_unstale":"2024-05-10T10:37:35.567305+0000","last_undegraded":"2024-05-10T10:37:35.567305+0000","last_fullsized":"2024-05-10T10:37:35.567305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:12.151693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:58.902976+0000","last_change":"2024-05-10T10:35:47.908594+0000","last_active":"2024-05-10T10:37:58.902976+0000","last_peered":"2024-05-10T10:37:58.902976+0000","last_clean":"2024-05-10T10:37:58.902976+0000","last_became_active":"2024-05-10T10:35:47.908278+0000","last_became_peered":"2024-05-10T10:35:47.908278+0000","last_unstale":"2024-05-10T10:37:58.902976+0000","last_undegraded":"2024-05-10T10:37:58.902976+0000","last_fullsized":"2024-05-10T10:37:58.902976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:24.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295025+0000","last_change":"2024-05-10T10:35:50.933089+0000","last_active":"2024-05-10T10:38:33.295025+0000","last_peered":"2024-05-10T10:38:33.295025+0000","last_clean":"2024-05-10T10:38:33.295025+0000","last_became_active":"2024-05-10T10:35:50.932739+0000","last_became_peered":"2024-05-10T10:35:50.932739+0000","last_unstale":"2024-05-10T10:38:33.295025+0000","last_undegraded":"2024-05-10T10:38:33.295025+0000","last_fullsized":"2024-05-10T10:38:33.295025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:15.302832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720006+0000","last_change":"2024-05-10T10:36:01.988561+0000","last_active":"2024-05-10T10:37:35.720006+0000","last_peered":"2024-05-10T10:37:35.720006+0000","last_clean":"2024-05-10T10:37:35.720006+0000","last_became_active":"2024-05-10T10:36:01.988120+0000","last_became_peered":"2024-05-10T10:36:01.988120+0000","last_unstale":"2024-05-10T10:37:35.720006+0000","last_undegraded":"2024-05-10T10:37:35.720006+0000","last_fullsized":"2024-05-10T10:37:35.720006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:35:37.120469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:39:46.068283+0000","last_change":"2024-05-10T10:37:35.580875+0000","last_active":"2024-05-10T10:39:46.068283+0000","last_peered":"2024-05-10T10:39:46.068283+0000","last_clean":"2024-05-10T10:39:46.068283+0000","last_became_active":"2024-05-10T10:37:35.572187+0000","last_became_peered":"2024-05-10T10:37:35.572187+0000","last_unstale":"2024-05-10T10:39:46.068283+0000","last_undegraded":"2024-05-10T10:39:46.068283+0000","last_fullsized":"2024-05-10T10:39:46.068283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:06.330404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290565+0000","last_change":"2024-05-10T10:35:50.938466+0000","last_active":"2024-05-10T10:38:33.290565+0000","last_peered":"2024-05-10T10:38:33.290565+0000","last_clean":"2024-05-10T10:38:33.290565+0000","last_became_active":"2024-05-10T10:35:50.938167+0000","last_became_peered":"2024-05-10T10:35:50.938167+0000","last_unstale":"2024-05-10T10:38:33.290565+0000","last_undegraded":"2024-05-10T10:38:33.290565+0000","last_fullsized":"2024-05-10T10:38:33.290565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:18.014708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719597+0000","last_change":"2024-05-10T10:36:02.018661+0000","last_active":"2024-05-10T10:37:35.719597+0000","last_peered":"2024-05-10T10:37:35.719597+0000","last_clean":"2024-05-10T10:37:35.719597+0000","last_became_active":"2024-05-10T10:36:02.018196+0000","last_became_peered":"2024-05-10T10:36:02.018196+0000","last_unstale":"2024-05-10T10:37:35.719597+0000","last_undegraded":"2024-05-10T10:37:35.719597+0000","last_fullsized":"2024-05-10T10:37:35.719597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:11.605012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570017+0000","last_change":"2024-05-10T10:35:47.917009+0000","last_active":"2024-05-10T10:37:35.570017+0000","last_peered":"2024-05-10T10:37:35.570017+0000","last_clean":"2024-05-10T10:37:35.570017+0000","last_became_active":"2024-05-10T10:35:47.916536+0000","last_became_peered":"2024-05-10T10:35:47.916536+0000","last_unstale":"2024-05-10T10:37:35.570017+0000","last_undegraded":"2024-05-10T10:37:35.570017+0000","last_fullsized":"2024-05-10T10:37:35.570017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:17.783027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287116+0000","last_change":"2024-05-10T10:35:50.926173+0000","last_active":"2024-05-10T10:38:33.287116+0000","last_peered":"2024-05-10T10:38:33.287116+0000","last_clean":"2024-05-10T10:38:33.287116+0000","last_became_active":"2024-05-10T10:35:50.925846+0000","last_became_peered":"2024-05-10T10:35:50.925846+0000","last_unstale":"2024-05-10T10:38:33.287116+0000","last_undegraded":"2024-05-10T10:38:33.287116+0000","last_fullsized":"2024-05-10T10:38:33.287116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:29:02.587667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565679+0000","last_change":"2024-05-10T10:36:01.981312+0000","last_active":"2024-05-10T10:37:35.565679+0000","last_peered":"2024-05-10T10:37:35.565679+0000","last_clean":"2024-05-10T10:37:35.565679+0000","last_became_active":"2024-05-10T10:36:01.980861+0000","last_became_peered":"2024-05-10T10:36:01.980861+0000","last_unstale":"2024-05-10T10:37:35.565679+0000","last_undegraded":"2024-05-10T10:37:35.565679+0000","last_fullsized":"2024-05-10T10:37:35.565679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:29.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.680524+0000","last_change":"2024-05-10T10:35:47.911120+0000","last_active":"2024-05-10T10:40:08.680524+0000","last_peered":"2024-05-10T10:40:08.680524+0000","last_clean":"2024-05-10T10:40:08.680524+0000","last_became_active":"2024-05-10T10:35:47.909945+0000","last_became_peered":"2024-05-10T10:35:47.909945+0000","last_unstale":"2024-05-10T10:40:08.680524+0000","last_undegraded":"2024-05-10T10:40:08.680524+0000","last_fullsized":"2024-05-10T10:40:08.680524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:40.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287370+0000","last_change":"2024-05-10T10:35:50.933855+0000","last_active":"2024-05-10T10:38:33.287370+0000","last_peered":"2024-05-10T10:38:33.287370+0000","last_clean":"2024-05-10T10:38:33.287370+0000","last_became_active":"2024-05-10T10:35:50.933502+0000","last_became_peered":"2024-05-10T10:35:50.933502+0000","last_unstale":"2024-05-10T10:38:33.287370+0000","last_undegraded":"2024-05-10T10:38:33.287370+0000","last_fullsized":"2024-05-10T10:38:33.287370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:36.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718378+0000","last_change":"2024-05-10T10:36:01.985133+0000","last_active":"2024-05-10T10:37:35.718378+0000","last_peered":"2024-05-10T10:37:35.718378+0000","last_clean":"2024-05-10T10:37:35.718378+0000","last_became_active":"2024-05-10T10:36:01.984758+0000","last_became_peered":"2024-05-10T10:36:01.984758+0000","last_unstale":"2024-05-10T10:37:35.718378+0000","last_undegraded":"2024-05-10T10:37:35.718378+0000","last_fullsized":"2024-05-10T10:37:35.718378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:52.733063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566564+0000","last_change":"2024-05-10T10:35:47.908645+0000","last_active":"2024-05-10T10:37:35.566564+0000","last_peered":"2024-05-10T10:37:35.566564+0000","last_clean":"2024-05-10T10:37:35.566564+0000","last_became_active":"2024-05-10T10:35:47.908123+0000","last_became_peered":"2024-05-10T10:35:47.908123+0000","last_unstale":"2024-05-10T10:37:35.566564+0000","last_undegraded":"2024-05-10T10:37:35.566564+0000","last_fullsized":"2024-05-10T10:37:35.566564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:46.181724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289675+0000","last_change":"2024-05-10T10:35:50.929215+0000","last_active":"2024-05-10T10:38:33.289675+0000","last_peered":"2024-05-10T10:38:33.289675+0000","last_clean":"2024-05-10T10:38:33.289675+0000","last_became_active":"2024-05-10T10:35:50.928907+0000","last_became_peered":"2024-05-10T10:35:50.928907+0000","last_unstale":"2024-05-10T10:38:33.289675+0000","last_undegraded":"2024-05-10T10:38:33.289675+0000","last_fullsized":"2024-05-10T10:38:33.289675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:57:17.340687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565667+0000","last_change":"2024-05-10T10:36:01.988548+0000","last_active":"2024-05-10T10:37:35.565667+0000","last_peered":"2024-05-10T10:37:35.565667+0000","last_clean":"2024-05-10T10:37:35.565667+0000","last_became_active":"2024-05-10T10:36:01.987221+0000","last_became_peered":"2024-05-10T10:36:01.987221+0000","last_unstale":"2024-05-10T10:37:35.565667+0000","last_undegraded":"2024-05-10T10:37:35.565667+0000","last_fullsized":"2024-05-10T10:37:35.565667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:55:22.434536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565063+0000","last_change":"2024-05-10T10:35:47.914022+0000","last_active":"2024-05-10T10:37:35.565063+0000","last_peered":"2024-05-10T10:37:35.565063+0000","last_clean":"2024-05-10T10:37:35.565063+0000","last_became_active":"2024-05-10T10:35:47.913711+0000","last_became_peered":"2024-05-10T10:35:47.913711+0000","last_unstale":"2024-05-10T10:37:35.565063+0000","last_undegraded":"2024-05-10T10:37:35.565063+0000","last_fullsized":"2024-05-10T10:37:35.565063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:13.291307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284901+0000","last_change":"2024-05-10T10:35:50.927452+0000","last_active":"2024-05-10T10:38:33.284901+0000","last_peered":"2024-05-10T10:38:33.284901+0000","last_clean":"2024-05-10T10:38:33.284901+0000","last_became_active":"2024-05-10T10:35:50.927102+0000","last_became_peered":"2024-05-10T10:35:50.927102+0000","last_unstale":"2024-05-10T10:38:33.284901+0000","last_undegraded":"2024-05-10T10:38:33.284901+0000","last_fullsized":"2024-05-10T10:38:33.284901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:07.704196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293454+0000","last_change":"2024-05-10T10:35:50.933072+0000","last_active":"2024-05-10T10:38:33.293454+0000","last_peered":"2024-05-10T10:38:33.293454+0000","last_clean":"2024-05-10T10:38:33.293454+0000","last_became_active":"2024-05-10T10:35:50.932724+0000","last_became_peered":"2024-05-10T10:35:50.932724+0000","last_unstale":"2024-05-10T10:38:33.293454+0000","last_undegraded":"2024-05-10T10:38:33.293454+0000","last_fullsized":"2024-05-10T10:38:33.293454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:08.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287960+0000","last_change":"2024-05-10T10:35:50.931635+0000","last_active":"2024-05-10T10:38:33.287960+0000","last_peered":"2024-05-10T10:38:33.287960+0000","last_clean":"2024-05-10T10:38:33.287960+0000","last_became_active":"2024-05-10T10:35:50.930158+0000","last_became_peered":"2024-05-10T10:35:50.930158+0000","last_unstale":"2024-05-10T10:38:33.287960+0000","last_undegraded":"2024-05-10T10:38:33.287960+0000","last_fullsized":"2024-05-10T10:38:33.287960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:22.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285388+0000","last_change":"2024-05-10T10:35:50.936947+0000","last_active":"2024-05-10T10:38:33.285388+0000","last_peered":"2024-05-10T10:38:33.285388+0000","last_clean":"2024-05-10T10:38:33.285388+0000","last_became_active":"2024-05-10T10:35:50.936156+0000","last_became_peered":"2024-05-10T10:35:50.936156+0000","last_unstale":"2024-05-10T10:38:33.285388+0000","last_undegraded":"2024-05-10T10:38:33.285388+0000","last_fullsized":"2024-05-10T10:38:33.285388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:21.167787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294726+0000","last_change":"2024-05-10T10:35:50.935384+0000","last_active":"2024-05-10T10:38:33.294726+0000","last_peered":"2024-05-10T10:38:33.294726+0000","last_clean":"2024-05-10T10:38:33.294726+0000","last_became_active":"2024-05-10T10:35:50.932555+0000","last_became_peered":"2024-05-10T10:35:50.932555+0000","last_unstale":"2024-05-10T10:38:33.294726+0000","last_undegraded":"2024-05-10T10:38:33.294726+0000","last_fullsized":"2024-05-10T10:38:33.294726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:09:11.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295227+0000","last_change":"2024-05-10T10:35:50.929322+0000","last_active":"2024-05-10T10:38:33.295227+0000","last_peered":"2024-05-10T10:38:33.295227+0000","last_clean":"2024-05-10T10:38:33.295227+0000","last_became_active":"2024-05-10T10:35:50.928366+0000","last_became_peered":"2024-05-10T10:35:50.928366+0000","last_unstale":"2024-05-10T10:38:33.295227+0000","last_undegraded":"2024-05-10T10:38:33.295227+0000","last_fullsized":"2024-05-10T10:38:33.295227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:32:28.337962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292904+0000","last_change":"2024-05-10T10:36:35.110922+0000","last_active":"2024-05-10T10:38:33.292904+0000","last_peered":"2024-05-10T10:38:33.292904+0000","last_clean":"2024-05-10T10:38:33.292904+0000","last_became_active":"2024-05-10T10:36:35.110525+0000","last_became_peered":"2024-05-10T10:36:35.110525+0000","last_unstale":"2024-05-10T10:38:33.292904+0000","last_undegraded":"2024-05-10T10:38:33.292904+0000","last_fullsized":"2024-05-10T10:38:33.292904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:47.825168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288252+0000","last_change":"2024-05-10T10:35:50.927869+0000","last_active":"2024-05-10T10:38:33.288252+0000","last_peered":"2024-05-10T10:38:33.288252+0000","last_clean":"2024-05-10T10:38:33.288252+0000","last_became_active":"2024-05-10T10:35:50.927552+0000","last_became_peered":"2024-05-10T10:35:50.927552+0000","last_unstale":"2024-05-10T10:38:33.288252+0000","last_undegraded":"2024-05-10T10:38:33.288252+0000","last_fullsized":"2024-05-10T10:38:33.288252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:36.962316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285786+0000","last_change":"2024-05-10T10:35:50.933099+0000","last_active":"2024-05-10T10:38:33.285786+0000","last_peered":"2024-05-10T10:38:33.285786+0000","last_clean":"2024-05-10T10:38:33.285786+0000","last_became_active":"2024-05-10T10:35:50.932765+0000","last_became_peered":"2024-05-10T10:35:50.932765+0000","last_unstale":"2024-05-10T10:38:33.285786+0000","last_undegraded":"2024-05-10T10:38:33.285786+0000","last_fullsized":"2024-05-10T10:38:33.285786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:19.095217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294758+0000","last_change":"2024-05-10T10:35:50.937704+0000","last_active":"2024-05-10T10:38:33.294758+0000","last_peered":"2024-05-10T10:38:33.294758+0000","last_clean":"2024-05-10T10:38:33.294758+0000","last_became_active":"2024-05-10T10:35:50.937236+0000","last_became_peered":"2024-05-10T10:35:50.937236+0000","last_unstale":"2024-05-10T10:38:33.294758+0000","last_undegraded":"2024-05-10T10:38:33.294758+0000","last_fullsized":"2024-05-10T10:38:33.294758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:46.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292751+0000","last_change":"2024-05-10T10:35:50.935181+0000","last_active":"2024-05-10T10:38:33.292751+0000","last_peered":"2024-05-10T10:38:33.292751+0000","last_clean":"2024-05-10T10:38:33.292751+0000","last_became_active":"2024-05-10T10:35:50.933446+0000","last_became_peered":"2024-05-10T10:35:50.933446+0000","last_unstale":"2024-05-10T10:38:33.292751+0000","last_undegraded":"2024-05-10T10:38:33.292751+0000","last_fullsized":"2024-05-10T10:38:33.292751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:03.972410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285940+0000","last_change":"2024-05-10T10:35:50.931797+0000","last_active":"2024-05-10T10:38:33.285940+0000","last_peered":"2024-05-10T10:38:33.285940+0000","last_clean":"2024-05-10T10:38:33.285940+0000","last_became_active":"2024-05-10T10:35:50.931494+0000","last_became_peered":"2024-05-10T10:35:50.931494+0000","last_unstale":"2024-05-10T10:38:33.285940+0000","last_undegraded":"2024-05-10T10:38:33.285940+0000","last_fullsized":"2024-05-10T10:38:33.285940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:30.797462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296149+0000","last_change":"2024-05-10T10:35:50.931688+0000","last_active":"2024-05-10T10:38:33.296149+0000","last_peered":"2024-05-10T10:38:33.296149+0000","last_clean":"2024-05-10T10:38:33.296149+0000","last_became_active":"2024-05-10T10:35:50.931362+0000","last_became_peered":"2024-05-10T10:35:50.931362+0000","last_unstale":"2024-05-10T10:38:33.296149+0000","last_undegraded":"2024-05-10T10:38:33.296149+0000","last_fullsized":"2024-05-10T10:38:33.296149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:24.263752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292719+0000","last_change":"2024-05-10T10:35:50.930905+0000","last_active":"2024-05-10T10:38:33.292719+0000","last_peered":"2024-05-10T10:38:33.292719+0000","last_clean":"2024-05-10T10:38:33.292719+0000","last_became_active":"2024-05-10T10:35:50.930448+0000","last_became_peered":"2024-05-10T10:35:50.930448+0000","last_unstale":"2024-05-10T10:38:33.292719+0000","last_undegraded":"2024-05-10T10:38:33.292719+0000","last_fullsized":"2024-05-10T10:38:33.292719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:52:24.468600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76702,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1932,"num_read_kb":1048,"num_write":271,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3701,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":234420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678085,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":521483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874313,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1176607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037838,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234068,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430298,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1119415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626527,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":516917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":585723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":518373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":518660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":516132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":593613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1113479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:40:11.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 -- 172.21.15.112:0/542951339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301045a0 msgr2=0x7fbf3010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 --2- 172.21.15.112:0/542951339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301045a0 0x7fbf3010ae30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009ca0 tx=0x7fbf2802a8c0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 -- 172.21.15.112:0/542951339 shutdown_connections 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 --2- 172.21.15.112:0/542951339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301045a0 0x7fbf3010ae30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 --2- 172.21.15.112:0/542951339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30103be0 0x7fbf30104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 --2- 172.21.15.112:0/542951339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf301029f0 0x7fbf30102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 -- 172.21.15.112:0/542951339 >> 172.21.15.112:0/542951339 conn(0x7fbf300fdf60 msgr2=0x7fbf301003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 -- 172.21.15.112:0/542951339 shutdown_connections 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.823+0000 7fbf35eda640 1 -- 172.21.15.112:0/542951339 wait complete. 2024-05-10T10:40:11.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 Processor -- start 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 -- start start 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301029f0 0x7fbf30198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30103be0 0x7fbf30198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf301045a0 0x7fbf301a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3010e400 con 0x7fbf301029f0 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3010e280 con 0x7fbf301045a0 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf35eda640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3010e580 con 0x7fbf30103be0 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301029f0 0x7fbf30198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301029f0 0x7fbf30198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46470/0 (socket says 172.21.15.112:46470) 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2f7fe640 1 -- 172.21.15.112:0/4048578965 learned_addr learned my addr 172.21.15.112:0/4048578965 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2ffff640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf301045a0 0x7fbf301a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2effd640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30103be0 0x7fbf30198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2f7fe640 1 -- 172.21.15.112:0/4048578965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30103be0 msgr2=0x7fbf30198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2f7fe640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30103be0 0x7fbf30198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.824+0000 7fbf2f7fe640 1 -- 172.21.15.112:0/4048578965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf301045a0 msgr2=0x7fbf301a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2f7fe640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf301045a0 0x7fbf301a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2f7fe640 1 -- 172.21.15.112:0/4048578965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf301a06c0 con 0x7fbf301029f0 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2effd640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30103be0 0x7fbf30198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2ffff640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf301045a0 0x7fbf301a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2f7fe640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301029f0 0x7fbf30198aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000e9c0 tx=0x7fbf2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2cff9640 1 -- 172.21.15.112:0/4048578965 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000cd80 con 0x7fbf301029f0 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2cff9640 1 -- 172.21.15.112:0/4048578965 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf200044b0 con 0x7fbf301029f0 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf301a09b0 con 0x7fbf301029f0 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf2cff9640 1 -- 172.21.15.112:0/4048578965 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20010430 con 0x7fbf301029f0 2024-05-10T10:40:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.825+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf301a0f00 con 0x7fbf301029f0 2024-05-10T10:40:11.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.826+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf30069c90 con 0x7fbf301029f0 2024-05-10T10:40:11.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.827+0000 7fbf2cff9640 1 -- 172.21.15.112:0/4048578965 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbf20003f00 con 0x7fbf301029f0 2024-05-10T10:40:11.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.828+0000 7fbf2cff9640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbf180b27f0 0x7fbf180b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:11.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.828+0000 7fbf2effd640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbf180b27f0 0x7fbf180b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:11.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.828+0000 7fbf2cff9640 1 -- 172.21.15.112:0/4048578965 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbf200e2b80 con 0x7fbf301029f0 2024-05-10T10:40:11.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.828+0000 7fbf2effd640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbf180b27f0 0x7fbf180b4cb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbf30199fb0 tx=0x7fbf24005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:11.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:11.831+0000 7fbf2cff9640 1 -- 172.21.15.112:0/4048578965 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbf200aa5a0 con 0x7fbf301029f0 2024-05-10T10:40:12.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.012+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf30062df0 con 0x7fbf180b27f0 2024-05-10T10:40:12.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.022+0000 7fbf2cff9640 1 -- 172.21.15.112:0/4048578965 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7fbf30062df0 con 0x7fbf180b27f0 2024-05-10T10:40:12.026 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:12.028 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-10T10:40:12.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbf180b27f0 msgr2=0x7fbf180b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbf180b27f0 0x7fbf180b4cb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbf30199fb0 tx=0x7fbf24005e50 comp rx=0 tx=0).stop 2024-05-10T10:40:12.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301029f0 msgr2=0x7fbf30198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301029f0 0x7fbf30198aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000e9c0 tx=0x7fbf2000ee80 comp rx=0 tx=0).stop 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 shutdown_connections 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbf180b27f0 0x7fbf180b4cb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf301045a0 0x7fbf301a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30103be0 0x7fbf30198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 --2- 172.21.15.112:0/4048578965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf301029f0 0x7fbf30198aa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 >> 172.21.15.112:0/4048578965 conn(0x7fbf300fdf60 msgr2=0x7fbf30106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 shutdown_connections 2024-05-10T10:40:12.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.025+0000 7fbf35eda640 1 -- 172.21.15.112:0/4048578965 wait complete. 2024-05-10T10:40:12.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:11 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4120567738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:40:12.038 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-05-10T10:40:11.354232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667304,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2008,"num_read_kb":1112,"num_write":396,"num_write_kb":2315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2955541,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":827,"ondisk_log_size":827,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367712,"kb_used_data":14400,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574112,"statfs":{"total":1151940427776,"available":1151563890688,"internally_reserved":0,"allocated":14745600,"data_stored":8226716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":60,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003248"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295705+0000","last_change":"2024-05-10T10:35:50.924449+0000","last_active":"2024-05-10T10:38:33.295705+0000","last_peered":"2024-05-10T10:38:33.295705+0000","last_clean":"2024-05-10T10:38:33.295705+0000","last_became_active":"2024-05-10T10:35:50.924114+0000","last_became_peered":"2024-05-10T10:35:50.924114+0000","last_unstale":"2024-05-10T10:38:33.295705+0000","last_undegraded":"2024-05-10T10:38:33.295705+0000","last_fullsized":"2024-05-10T10:38:33.295705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:50.636501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292255+0000","last_change":"2024-05-10T10:35:50.928715+0000","last_active":"2024-05-10T10:38:33.292255+0000","last_peered":"2024-05-10T10:38:33.292255+0000","last_clean":"2024-05-10T10:38:33.292255+0000","last_became_active":"2024-05-10T10:35:50.928404+0000","last_became_peered":"2024-05-10T10:35:50.928404+0000","last_unstale":"2024-05-10T10:38:33.292255+0000","last_undegraded":"2024-05-10T10:38:33.292255+0000","last_fullsized":"2024-05-10T10:38:33.292255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:44.934022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295067+0000","last_change":"2024-05-10T10:35:50.925134+0000","last_active":"2024-05-10T10:38:33.295067+0000","last_peered":"2024-05-10T10:38:33.295067+0000","last_clean":"2024-05-10T10:38:33.295067+0000","last_became_active":"2024-05-10T10:35:50.924797+0000","last_became_peered":"2024-05-10T10:35:50.924797+0000","last_unstale":"2024-05-10T10:38:33.295067+0000","last_undegraded":"2024-05-10T10:38:33.295067+0000","last_fullsized":"2024-05-10T10:38:33.295067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:43:52.666519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.280643+0000","last_change":"2024-05-10T10:35:50.930916+0000","last_active":"2024-05-10T10:38:33.280643+0000","last_peered":"2024-05-10T10:38:33.280643+0000","last_clean":"2024-05-10T10:38:33.280643+0000","last_became_active":"2024-05-10T10:35:50.930378+0000","last_became_peered":"2024-05-10T10:35:50.930378+0000","last_unstale":"2024-05-10T10:38:33.280643+0000","last_undegraded":"2024-05-10T10:38:33.280643+0000","last_fullsized":"2024-05-10T10:38:33.280643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:07.165595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.269516+0000","last_change":"2024-05-10T10:35:50.925459+0000","last_active":"2024-05-10T10:38:33.269516+0000","last_peered":"2024-05-10T10:38:33.269516+0000","last_clean":"2024-05-10T10:38:33.269516+0000","last_became_active":"2024-05-10T10:35:50.925061+0000","last_became_peered":"2024-05-10T10:35:50.925061+0000","last_unstale":"2024-05-10T10:38:33.269516+0000","last_undegraded":"2024-05-10T10:38:33.269516+0000","last_fullsized":"2024-05-10T10:38:33.269516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:48:06.622369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290891+0000","last_change":"2024-05-10T10:35:50.932052+0000","last_active":"2024-05-10T10:38:33.290891+0000","last_peered":"2024-05-10T10:38:33.290891+0000","last_clean":"2024-05-10T10:38:33.290891+0000","last_became_active":"2024-05-10T10:35:50.931685+0000","last_became_peered":"2024-05-10T10:35:50.931685+0000","last_unstale":"2024-05-10T10:38:33.290891+0000","last_undegraded":"2024-05-10T10:38:33.290891+0000","last_fullsized":"2024-05-10T10:38:33.290891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:13:39.930917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289511+0000","last_change":"2024-05-10T10:35:50.930620+0000","last_active":"2024-05-10T10:38:33.289511+0000","last_peered":"2024-05-10T10:38:33.289511+0000","last_clean":"2024-05-10T10:38:33.289511+0000","last_became_active":"2024-05-10T10:35:50.929696+0000","last_became_peered":"2024-05-10T10:35:50.929696+0000","last_unstale":"2024-05-10T10:38:33.289511+0000","last_undegraded":"2024-05-10T10:38:33.289511+0000","last_fullsized":"2024-05-10T10:38:33.289511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:23.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296397+0000","last_change":"2024-05-10T10:35:50.937517+0000","last_active":"2024-05-10T10:38:33.296397+0000","last_peered":"2024-05-10T10:38:33.296397+0000","last_clean":"2024-05-10T10:38:33.296397+0000","last_became_active":"2024-05-10T10:35:50.937180+0000","last_became_peered":"2024-05-10T10:35:50.937180+0000","last_unstale":"2024-05-10T10:38:33.296397+0000","last_undegraded":"2024-05-10T10:38:33.296397+0000","last_fullsized":"2024-05-10T10:38:33.296397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.568509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293452+0000","last_change":"2024-05-10T10:35:50.937825+0000","last_active":"2024-05-10T10:38:33.293452+0000","last_peered":"2024-05-10T10:38:33.293452+0000","last_clean":"2024-05-10T10:38:33.293452+0000","last_became_active":"2024-05-10T10:35:50.935650+0000","last_became_peered":"2024-05-10T10:35:50.935650+0000","last_unstale":"2024-05-10T10:38:33.293452+0000","last_undegraded":"2024-05-10T10:38:33.293452+0000","last_fullsized":"2024-05-10T10:38:33.293452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:43:55.780266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285811+0000","last_change":"2024-05-10T10:35:50.933598+0000","last_active":"2024-05-10T10:38:33.285811+0000","last_peered":"2024-05-10T10:38:33.285811+0000","last_clean":"2024-05-10T10:38:33.285811+0000","last_became_active":"2024-05-10T10:35:50.932767+0000","last_became_peered":"2024-05-10T10:35:50.932767+0000","last_unstale":"2024-05-10T10:38:33.285811+0000","last_undegraded":"2024-05-10T10:38:33.285811+0000","last_fullsized":"2024-05-10T10:38:33.285811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:40.610376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287212+0000","last_change":"2024-05-10T10:35:50.934882+0000","last_active":"2024-05-10T10:38:33.287212+0000","last_peered":"2024-05-10T10:38:33.287212+0000","last_clean":"2024-05-10T10:38:33.287212+0000","last_became_active":"2024-05-10T10:35:50.934528+0000","last_became_peered":"2024-05-10T10:35:50.934528+0000","last_unstale":"2024-05-10T10:38:33.287212+0000","last_undegraded":"2024-05-10T10:38:33.287212+0000","last_fullsized":"2024-05-10T10:38:33.287212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:15.319765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289796+0000","last_change":"2024-05-10T10:36:35.110939+0000","last_active":"2024-05-10T10:38:33.289796+0000","last_peered":"2024-05-10T10:38:33.289796+0000","last_clean":"2024-05-10T10:38:33.289796+0000","last_became_active":"2024-05-10T10:36:35.110600+0000","last_became_peered":"2024-05-10T10:36:35.110600+0000","last_unstale":"2024-05-10T10:38:33.289796+0000","last_undegraded":"2024-05-10T10:38:33.289796+0000","last_fullsized":"2024-05-10T10:38:33.289796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:45:58.547301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.297102+0000","last_change":"2024-05-10T10:35:50.934161+0000","last_active":"2024-05-10T10:38:33.297102+0000","last_peered":"2024-05-10T10:38:33.297102+0000","last_clean":"2024-05-10T10:38:33.297102+0000","last_became_active":"2024-05-10T10:35:50.933831+0000","last_became_peered":"2024-05-10T10:35:50.933831+0000","last_unstale":"2024-05-10T10:38:33.297102+0000","last_undegraded":"2024-05-10T10:38:33.297102+0000","last_fullsized":"2024-05-10T10:38:33.297102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:09.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286140+0000","last_change":"2024-05-10T10:35:50.927404+0000","last_active":"2024-05-10T10:38:33.286140+0000","last_peered":"2024-05-10T10:38:33.286140+0000","last_clean":"2024-05-10T10:38:33.286140+0000","last_became_active":"2024-05-10T10:35:50.927094+0000","last_became_peered":"2024-05-10T10:35:50.927094+0000","last_unstale":"2024-05-10T10:38:33.286140+0000","last_undegraded":"2024-05-10T10:38:33.286140+0000","last_fullsized":"2024-05-10T10:38:33.286140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:25.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293207+0000","last_change":"2024-05-10T10:36:35.142063+0000","last_active":"2024-05-10T10:38:33.293207+0000","last_peered":"2024-05-10T10:38:33.293207+0000","last_clean":"2024-05-10T10:38:33.293207+0000","last_became_active":"2024-05-10T10:36:35.141685+0000","last_became_peered":"2024-05-10T10:36:35.141685+0000","last_unstale":"2024-05-10T10:38:33.293207+0000","last_undegraded":"2024-05-10T10:38:33.293207+0000","last_fullsized":"2024-05-10T10:38:33.293207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:47.284288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294424+0000","last_change":"2024-05-10T10:35:50.933172+0000","last_active":"2024-05-10T10:38:33.294424+0000","last_peered":"2024-05-10T10:38:33.294424+0000","last_clean":"2024-05-10T10:38:33.294424+0000","last_became_active":"2024-05-10T10:35:50.931864+0000","last_became_peered":"2024-05-10T10:35:50.931864+0000","last_unstale":"2024-05-10T10:38:33.294424+0000","last_undegraded":"2024-05-10T10:38:33.294424+0000","last_fullsized":"2024-05-10T10:38:33.294424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:50.620315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295591+0000","last_change":"2024-05-10T10:35:50.925952+0000","last_active":"2024-05-10T10:38:33.295591+0000","last_peered":"2024-05-10T10:38:33.295591+0000","last_clean":"2024-05-10T10:38:33.295591+0000","last_became_active":"2024-05-10T10:35:50.925610+0000","last_became_peered":"2024-05-10T10:35:50.925610+0000","last_unstale":"2024-05-10T10:38:33.295591+0000","last_undegraded":"2024-05-10T10:38:33.295591+0000","last_fullsized":"2024-05-10T10:38:33.295591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:38.287235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291553+0000","last_change":"2024-05-10T10:35:50.929379+0000","last_active":"2024-05-10T10:38:33.291553+0000","last_peered":"2024-05-10T10:38:33.291553+0000","last_clean":"2024-05-10T10:38:33.291553+0000","last_became_active":"2024-05-10T10:35:50.929082+0000","last_became_peered":"2024-05-10T10:35:50.929082+0000","last_unstale":"2024-05-10T10:38:33.291553+0000","last_undegraded":"2024-05-10T10:38:33.291553+0000","last_fullsized":"2024-05-10T10:38:33.291553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:54.097855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.283992+0000","last_change":"2024-05-10T10:35:50.937754+0000","last_active":"2024-05-10T10:38:33.283992+0000","last_peered":"2024-05-10T10:38:33.283992+0000","last_clean":"2024-05-10T10:38:33.283992+0000","last_became_active":"2024-05-10T10:35:50.935662+0000","last_became_peered":"2024-05-10T10:35:50.935662+0000","last_unstale":"2024-05-10T10:38:33.283992+0000","last_undegraded":"2024-05-10T10:38:33.283992+0000","last_fullsized":"2024-05-10T10:38:33.283992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:52.692279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571937+0000","last_change":"2024-05-10T10:36:01.989463+0000","last_active":"2024-05-10T10:37:35.571937+0000","last_peered":"2024-05-10T10:37:35.571937+0000","last_clean":"2024-05-10T10:37:35.571937+0000","last_became_active":"2024-05-10T10:36:01.988949+0000","last_became_peered":"2024-05-10T10:36:01.988949+0000","last_unstale":"2024-05-10T10:37:35.571937+0000","last_undegraded":"2024-05-10T10:37:35.571937+0000","last_fullsized":"2024-05-10T10:37:35.571937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:47.529845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":248,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:06.657735+0000","last_change":"2024-05-10T10:35:47.909033+0000","last_active":"2024-05-10T10:40:06.657735+0000","last_peered":"2024-05-10T10:40:06.657735+0000","last_clean":"2024-05-10T10:40:06.657735+0000","last_became_active":"2024-05-10T10:35:47.908680+0000","last_became_peered":"2024-05-10T10:35:47.908680+0000","last_unstale":"2024-05-10T10:40:06.657735+0000","last_undegraded":"2024-05-10T10:40:06.657735+0000","last_fullsized":"2024-05-10T10:40:06.657735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:46.642113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294649+0000","last_change":"2024-05-10T10:35:50.939233+0000","last_active":"2024-05-10T10:38:33.294649+0000","last_peered":"2024-05-10T10:38:33.294649+0000","last_clean":"2024-05-10T10:38:33.294649+0000","last_became_active":"2024-05-10T10:35:50.938949+0000","last_became_peered":"2024-05-10T10:35:50.938949+0000","last_unstale":"2024-05-10T10:38:33.294649+0000","last_undegraded":"2024-05-10T10:38:33.294649+0000","last_fullsized":"2024-05-10T10:38:33.294649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:06.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718685+0000","last_change":"2024-05-10T10:36:01.990982+0000","last_active":"2024-05-10T10:37:35.718685+0000","last_peered":"2024-05-10T10:37:35.718685+0000","last_clean":"2024-05-10T10:37:35.718685+0000","last_became_active":"2024-05-10T10:36:01.990325+0000","last_became_peered":"2024-05-10T10:36:01.990325+0000","last_unstale":"2024-05-10T10:37:35.718685+0000","last_undegraded":"2024-05-10T10:37:35.718685+0000","last_fullsized":"2024-05-10T10:37:35.718685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:18.424130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:48.901957+0000","last_change":"2024-05-10T10:35:47.911239+0000","last_active":"2024-05-10T10:37:48.901957+0000","last_peered":"2024-05-10T10:37:48.901957+0000","last_clean":"2024-05-10T10:37:48.901957+0000","last_became_active":"2024-05-10T10:35:47.910945+0000","last_became_peered":"2024-05-10T10:35:47.910945+0000","last_unstale":"2024-05-10T10:37:48.901957+0000","last_undegraded":"2024-05-10T10:37:48.901957+0000","last_fullsized":"2024-05-10T10:37:48.901957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:08:52.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294800+0000","last_change":"2024-05-10T10:35:50.923694+0000","last_active":"2024-05-10T10:38:33.294800+0000","last_peered":"2024-05-10T10:38:33.294800+0000","last_clean":"2024-05-10T10:38:33.294800+0000","last_became_active":"2024-05-10T10:35:50.923381+0000","last_became_peered":"2024-05-10T10:35:50.923381+0000","last_unstale":"2024-05-10T10:38:33.294800+0000","last_undegraded":"2024-05-10T10:38:33.294800+0000","last_fullsized":"2024-05-10T10:38:33.294800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:59.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718663+0000","last_change":"2024-05-10T10:36:01.989868+0000","last_active":"2024-05-10T10:37:35.718663+0000","last_peered":"2024-05-10T10:37:35.718663+0000","last_clean":"2024-05-10T10:37:35.718663+0000","last_became_active":"2024-05-10T10:36:01.989283+0000","last_became_peered":"2024-05-10T10:36:01.989283+0000","last_unstale":"2024-05-10T10:37:35.718663+0000","last_undegraded":"2024-05-10T10:37:35.718663+0000","last_fullsized":"2024-05-10T10:37:35.718663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:35.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570220+0000","last_change":"2024-05-10T10:35:47.915555+0000","last_active":"2024-05-10T10:37:35.570220+0000","last_peered":"2024-05-10T10:37:35.570220+0000","last_clean":"2024-05-10T10:37:35.570220+0000","last_became_active":"2024-05-10T10:35:47.912852+0000","last_became_peered":"2024-05-10T10:35:47.912852+0000","last_unstale":"2024-05-10T10:37:35.570220+0000","last_undegraded":"2024-05-10T10:37:35.570220+0000","last_fullsized":"2024-05-10T10:37:35.570220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:56:45.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291583+0000","last_change":"2024-05-10T10:35:50.934010+0000","last_active":"2024-05-10T10:38:33.291583+0000","last_peered":"2024-05-10T10:38:33.291583+0000","last_clean":"2024-05-10T10:38:33.291583+0000","last_became_active":"2024-05-10T10:35:50.933653+0000","last_became_peered":"2024-05-10T10:35:50.933653+0000","last_unstale":"2024-05-10T10:38:33.291583+0000","last_undegraded":"2024-05-10T10:38:33.291583+0000","last_fullsized":"2024-05-10T10:38:33.291583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:34.862734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718657+0000","last_change":"2024-05-10T10:36:01.990524+0000","last_active":"2024-05-10T10:37:35.718657+0000","last_peered":"2024-05-10T10:37:35.718657+0000","last_clean":"2024-05-10T10:37:35.718657+0000","last_became_active":"2024-05-10T10:36:01.990011+0000","last_became_peered":"2024-05-10T10:36:01.990011+0000","last_unstale":"2024-05-10T10:37:35.718657+0000","last_undegraded":"2024-05-10T10:37:35.718657+0000","last_fullsized":"2024-05-10T10:37:35.718657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:06.123095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570506+0000","last_change":"2024-05-10T10:35:47.910977+0000","last_active":"2024-05-10T10:37:35.570506+0000","last_peered":"2024-05-10T10:37:35.570506+0000","last_clean":"2024-05-10T10:37:35.570506+0000","last_became_active":"2024-05-10T10:35:47.910623+0000","last_became_peered":"2024-05-10T10:35:47.910623+0000","last_unstale":"2024-05-10T10:37:35.570506+0000","last_undegraded":"2024-05-10T10:37:35.570506+0000","last_fullsized":"2024-05-10T10:37:35.570506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:37.760666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296932+0000","last_change":"2024-05-10T10:35:50.937238+0000","last_active":"2024-05-10T10:38:33.296932+0000","last_peered":"2024-05-10T10:38:33.296932+0000","last_clean":"2024-05-10T10:38:33.296932+0000","last_became_active":"2024-05-10T10:35:50.936298+0000","last_became_peered":"2024-05-10T10:35:50.936298+0000","last_unstale":"2024-05-10T10:38:33.296932+0000","last_undegraded":"2024-05-10T10:38:33.296932+0000","last_fullsized":"2024-05-10T10:38:33.296932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:54.724452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571851+0000","last_change":"2024-05-10T10:36:01.987769+0000","last_active":"2024-05-10T10:37:35.571851+0000","last_peered":"2024-05-10T10:37:35.571851+0000","last_clean":"2024-05-10T10:37:35.571851+0000","last_became_active":"2024-05-10T10:36:01.987367+0000","last_became_peered":"2024-05-10T10:36:01.987367+0000","last_unstale":"2024-05-10T10:37:35.571851+0000","last_undegraded":"2024-05-10T10:37:35.571851+0000","last_fullsized":"2024-05-10T10:37:35.571851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:36.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661837+0000","last_change":"2024-05-10T10:35:47.915082+0000","last_active":"2024-05-10T10:40:08.661837+0000","last_peered":"2024-05-10T10:40:08.661837+0000","last_clean":"2024-05-10T10:40:08.661837+0000","last_became_active":"2024-05-10T10:35:47.914722+0000","last_became_peered":"2024-05-10T10:35:47.914722+0000","last_unstale":"2024-05-10T10:40:08.661837+0000","last_undegraded":"2024-05-10T10:40:08.661837+0000","last_fullsized":"2024-05-10T10:40:08.661837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:31.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285496+0000","last_change":"2024-05-10T10:35:50.926181+0000","last_active":"2024-05-10T10:38:33.285496+0000","last_peered":"2024-05-10T10:38:33.285496+0000","last_clean":"2024-05-10T10:38:33.285496+0000","last_became_active":"2024-05-10T10:35:50.925865+0000","last_became_peered":"2024-05-10T10:35:50.925865+0000","last_unstale":"2024-05-10T10:38:33.285496+0000","last_undegraded":"2024-05-10T10:38:33.285496+0000","last_fullsized":"2024-05-10T10:38:33.285496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:04.976908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.572140+0000","last_change":"2024-05-10T10:36:35.142412+0000","last_active":"2024-05-10T10:37:35.572140+0000","last_peered":"2024-05-10T10:37:35.572140+0000","last_clean":"2024-05-10T10:37:35.572140+0000","last_became_active":"2024-05-10T10:36:35.141959+0000","last_became_peered":"2024-05-10T10:36:35.141959+0000","last_unstale":"2024-05-10T10:37:35.572140+0000","last_undegraded":"2024-05-10T10:37:35.572140+0000","last_fullsized":"2024-05-10T10:37:35.572140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:20:34.133261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720063+0000","last_change":"2024-05-10T10:35:47.912826+0000","last_active":"2024-05-10T10:37:35.720063+0000","last_peered":"2024-05-10T10:37:35.720063+0000","last_clean":"2024-05-10T10:37:35.720063+0000","last_became_active":"2024-05-10T10:35:47.912468+0000","last_became_peered":"2024-05-10T10:35:47.912468+0000","last_unstale":"2024-05-10T10:37:35.720063+0000","last_undegraded":"2024-05-10T10:37:35.720063+0000","last_fullsized":"2024-05-10T10:37:35.720063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:56.482104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293989+0000","last_change":"2024-05-10T10:36:35.114920+0000","last_active":"2024-05-10T10:38:33.293989+0000","last_peered":"2024-05-10T10:38:33.293989+0000","last_clean":"2024-05-10T10:38:33.293989+0000","last_became_active":"2024-05-10T10:36:35.114546+0000","last_became_peered":"2024-05-10T10:36:35.114546+0000","last_unstale":"2024-05-10T10:38:33.293989+0000","last_undegraded":"2024-05-10T10:38:33.293989+0000","last_fullsized":"2024-05-10T10:38:33.293989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:52.454111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.048033+0000","last_change":"2024-05-10T10:36:01.990086+0000","last_active":"2024-05-10T10:38:06.048033+0000","last_peered":"2024-05-10T10:38:06.048033+0000","last_clean":"2024-05-10T10:38:06.048033+0000","last_became_active":"2024-05-10T10:36:01.989624+0000","last_became_peered":"2024-05-10T10:36:01.989624+0000","last_unstale":"2024-05-10T10:38:06.048033+0000","last_undegraded":"2024-05-10T10:38:06.048033+0000","last_fullsized":"2024-05-10T10:38:06.048033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:33.708069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564634+0000","last_change":"2024-05-10T10:35:47.905182+0000","last_active":"2024-05-10T10:37:35.564634+0000","last_peered":"2024-05-10T10:37:35.564634+0000","last_clean":"2024-05-10T10:37:35.564634+0000","last_became_active":"2024-05-10T10:35:47.904855+0000","last_became_peered":"2024-05-10T10:35:47.904855+0000","last_unstale":"2024-05-10T10:37:35.564634+0000","last_undegraded":"2024-05-10T10:37:35.564634+0000","last_fullsized":"2024-05-10T10:37:35.564634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:17:44.844156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292710+0000","last_change":"2024-05-10T10:35:50.935478+0000","last_active":"2024-05-10T10:38:33.292710+0000","last_peered":"2024-05-10T10:38:33.292710+0000","last_clean":"2024-05-10T10:38:33.292710+0000","last_became_active":"2024-05-10T10:35:50.933487+0000","last_became_peered":"2024-05-10T10:35:50.933487+0000","last_unstale":"2024-05-10T10:38:33.292710+0000","last_undegraded":"2024-05-10T10:38:33.292710+0000","last_fullsized":"2024-05-10T10:38:33.292710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:02.433423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:02.493132+0000","last_change":"2024-05-10T10:36:01.984285+0000","last_active":"2024-05-10T10:38:02.493132+0000","last_peered":"2024-05-10T10:38:02.493132+0000","last_clean":"2024-05-10T10:38:02.493132+0000","last_became_active":"2024-05-10T10:36:01.983900+0000","last_became_peered":"2024-05-10T10:36:01.983900+0000","last_unstale":"2024-05-10T10:38:02.493132+0000","last_undegraded":"2024-05-10T10:38:02.493132+0000","last_fullsized":"2024-05-10T10:38:02.493132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:53.406106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565788+0000","last_change":"2024-05-10T10:35:47.911123+0000","last_active":"2024-05-10T10:37:35.565788+0000","last_peered":"2024-05-10T10:37:35.565788+0000","last_clean":"2024-05-10T10:37:35.565788+0000","last_became_active":"2024-05-10T10:35:47.910762+0000","last_became_peered":"2024-05-10T10:35:47.910762+0000","last_unstale":"2024-05-10T10:37:35.565788+0000","last_undegraded":"2024-05-10T10:37:35.565788+0000","last_fullsized":"2024-05-10T10:37:35.565788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:07.045586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288933+0000","last_change":"2024-05-10T10:35:50.928093+0000","last_active":"2024-05-10T10:38:33.288933+0000","last_peered":"2024-05-10T10:38:33.288933+0000","last_clean":"2024-05-10T10:38:33.288933+0000","last_became_active":"2024-05-10T10:35:50.926945+0000","last_became_peered":"2024-05-10T10:35:50.926945+0000","last_unstale":"2024-05-10T10:38:33.288933+0000","last_undegraded":"2024-05-10T10:38:33.288933+0000","last_fullsized":"2024-05-10T10:38:33.288933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:11.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569138+0000","last_change":"2024-05-10T10:36:35.143043+0000","last_active":"2024-05-10T10:37:35.569138+0000","last_peered":"2024-05-10T10:37:35.569138+0000","last_clean":"2024-05-10T10:37:35.569138+0000","last_became_active":"2024-05-10T10:36:35.142739+0000","last_became_peered":"2024-05-10T10:36:35.142739+0000","last_unstale":"2024-05-10T10:37:35.569138+0000","last_undegraded":"2024-05-10T10:37:35.569138+0000","last_fullsized":"2024-05-10T10:37:35.569138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:10.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":427,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661413+0000","last_change":"2024-05-10T10:35:47.908687+0000","last_active":"2024-05-10T10:40:08.661413+0000","last_peered":"2024-05-10T10:40:08.661413+0000","last_clean":"2024-05-10T10:40:08.661413+0000","last_became_active":"2024-05-10T10:35:47.908349+0000","last_became_peered":"2024-05-10T10:35:47.908349+0000","last_unstale":"2024-05-10T10:40:08.661413+0000","last_undegraded":"2024-05-10T10:40:08.661413+0000","last_fullsized":"2024-05-10T10:40:08.661413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:48.696104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":376,"num_read_kb":376,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291371+0000","last_change":"2024-05-10T10:35:50.933729+0000","last_active":"2024-05-10T10:38:33.291371+0000","last_peered":"2024-05-10T10:38:33.291371+0000","last_clean":"2024-05-10T10:38:33.291371+0000","last_became_active":"2024-05-10T10:35:50.933109+0000","last_became_peered":"2024-05-10T10:35:50.933109+0000","last_unstale":"2024-05-10T10:38:33.291371+0000","last_undegraded":"2024-05-10T10:38:33.291371+0000","last_fullsized":"2024-05-10T10:38:33.291371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:10:38.305592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567135+0000","last_change":"2024-05-10T10:36:01.991969+0000","last_active":"2024-05-10T10:37:35.567135+0000","last_peered":"2024-05-10T10:37:35.567135+0000","last_clean":"2024-05-10T10:37:35.567135+0000","last_became_active":"2024-05-10T10:36:01.990278+0000","last_became_peered":"2024-05-10T10:36:01.990278+0000","last_unstale":"2024-05-10T10:37:35.567135+0000","last_undegraded":"2024-05-10T10:37:35.567135+0000","last_fullsized":"2024-05-10T10:37:35.567135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:33.070520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719303+0000","last_change":"2024-05-10T10:35:47.911682+0000","last_active":"2024-05-10T10:37:35.719303+0000","last_peered":"2024-05-10T10:37:35.719303+0000","last_clean":"2024-05-10T10:37:35.719303+0000","last_became_active":"2024-05-10T10:35:47.911328+0000","last_became_peered":"2024-05-10T10:35:47.911328+0000","last_unstale":"2024-05-10T10:37:35.719303+0000","last_undegraded":"2024-05-10T10:37:35.719303+0000","last_fullsized":"2024-05-10T10:37:35.719303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.361340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285082+0000","last_change":"2024-05-10T10:35:50.931541+0000","last_active":"2024-05-10T10:38:33.285082+0000","last_peered":"2024-05-10T10:38:33.285082+0000","last_clean":"2024-05-10T10:38:33.285082+0000","last_became_active":"2024-05-10T10:35:50.931244+0000","last_became_peered":"2024-05-10T10:35:50.931244+0000","last_unstale":"2024-05-10T10:38:33.285082+0000","last_undegraded":"2024-05-10T10:38:33.285082+0000","last_fullsized":"2024-05-10T10:38:33.285082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:38.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720689+0000","last_change":"2024-05-10T10:36:01.990390+0000","last_active":"2024-05-10T10:37:35.720689+0000","last_peered":"2024-05-10T10:37:35.720689+0000","last_clean":"2024-05-10T10:37:35.720689+0000","last_became_active":"2024-05-10T10:36:01.989434+0000","last_became_peered":"2024-05-10T10:36:01.989434+0000","last_unstale":"2024-05-10T10:37:35.720689+0000","last_undegraded":"2024-05-10T10:37:35.720689+0000","last_fullsized":"2024-05-10T10:37:35.720689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:53.129212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566392+0000","last_change":"2024-05-10T10:35:47.911590+0000","last_active":"2024-05-10T10:37:35.566392+0000","last_peered":"2024-05-10T10:37:35.566392+0000","last_clean":"2024-05-10T10:37:35.566392+0000","last_became_active":"2024-05-10T10:35:47.910950+0000","last_became_peered":"2024-05-10T10:35:47.910950+0000","last_unstale":"2024-05-10T10:37:35.566392+0000","last_undegraded":"2024-05-10T10:37:35.566392+0000","last_fullsized":"2024-05-10T10:37:35.566392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:18.844774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292200+0000","last_change":"2024-05-10T10:35:50.938174+0000","last_active":"2024-05-10T10:38:33.292200+0000","last_peered":"2024-05-10T10:38:33.292200+0000","last_clean":"2024-05-10T10:38:33.292200+0000","last_became_active":"2024-05-10T10:35:50.937618+0000","last_became_peered":"2024-05-10T10:35:50.937618+0000","last_unstale":"2024-05-10T10:38:33.292200+0000","last_undegraded":"2024-05-10T10:38:33.292200+0000","last_fullsized":"2024-05-10T10:38:33.292200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:26:48.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567141+0000","last_change":"2024-05-10T10:36:01.979226+0000","last_active":"2024-05-10T10:37:35.567141+0000","last_peered":"2024-05-10T10:37:35.567141+0000","last_clean":"2024-05-10T10:37:35.567141+0000","last_became_active":"2024-05-10T10:36:01.978847+0000","last_became_peered":"2024-05-10T10:36:01.978847+0000","last_unstale":"2024-05-10T10:37:35.567141+0000","last_undegraded":"2024-05-10T10:37:35.567141+0000","last_fullsized":"2024-05-10T10:37:35.567141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:10.629813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'42","reported_seq":490,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661398+0000","last_change":"2024-05-10T10:36:35.150814+0000","last_active":"2024-05-10T10:40:08.661398+0000","last_peered":"2024-05-10T10:40:08.661398+0000","last_clean":"2024-05-10T10:40:08.661398+0000","last_became_active":"2024-05-10T10:36:35.142980+0000","last_became_peered":"2024-05-10T10:36:35.142980+0000","last_unstale":"2024-05-10T10:40:08.661398+0000","last_undegraded":"2024-05-10T10:40:08.661398+0000","last_fullsized":"2024-05-10T10:40:08.661398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:17.907275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67776,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":387,"num_read_kb":260,"num_write":44,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290064+0000","last_change":"2024-05-10T10:35:50.925584+0000","last_active":"2024-05-10T10:38:33.290064+0000","last_peered":"2024-05-10T10:38:33.290064+0000","last_clean":"2024-05-10T10:38:33.290064+0000","last_became_active":"2024-05-10T10:35:50.925215+0000","last_became_peered":"2024-05-10T10:35:50.925215+0000","last_unstale":"2024-05-10T10:38:33.290064+0000","last_undegraded":"2024-05-10T10:38:33.290064+0000","last_fullsized":"2024-05-10T10:38:33.290064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:03.377110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566636+0000","last_change":"2024-05-10T10:36:01.985090+0000","last_active":"2024-05-10T10:37:35.566636+0000","last_peered":"2024-05-10T10:37:35.566636+0000","last_clean":"2024-05-10T10:37:35.566636+0000","last_became_active":"2024-05-10T10:36:01.984059+0000","last_became_peered":"2024-05-10T10:36:01.984059+0000","last_unstale":"2024-05-10T10:37:35.566636+0000","last_undegraded":"2024-05-10T10:37:35.566636+0000","last_fullsized":"2024-05-10T10:37:35.566636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:41.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571763+0000","last_change":"2024-05-10T10:35:47.914377+0000","last_active":"2024-05-10T10:37:35.571763+0000","last_peered":"2024-05-10T10:37:35.571763+0000","last_clean":"2024-05-10T10:37:35.571763+0000","last_became_active":"2024-05-10T10:35:47.913830+0000","last_became_peered":"2024-05-10T10:35:47.913830+0000","last_unstale":"2024-05-10T10:37:35.571763+0000","last_undegraded":"2024-05-10T10:37:35.571763+0000","last_fullsized":"2024-05-10T10:37:35.571763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:54:18.540952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284127+0000","last_change":"2024-05-10T10:35:50.932721+0000","last_active":"2024-05-10T10:38:33.284127+0000","last_peered":"2024-05-10T10:38:33.284127+0000","last_clean":"2024-05-10T10:38:33.284127+0000","last_became_active":"2024-05-10T10:35:50.932410+0000","last_became_peered":"2024-05-10T10:35:50.932410+0000","last_unstale":"2024-05-10T10:38:33.284127+0000","last_undegraded":"2024-05-10T10:38:33.284127+0000","last_fullsized":"2024-05-10T10:38:33.284127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:42.255534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569383+0000","last_change":"2024-05-10T10:36:35.143581+0000","last_active":"2024-05-10T10:37:35.569383+0000","last_peered":"2024-05-10T10:37:35.569383+0000","last_clean":"2024-05-10T10:37:35.569383+0000","last_became_active":"2024-05-10T10:36:35.142957+0000","last_became_peered":"2024-05-10T10:36:35.142957+0000","last_unstale":"2024-05-10T10:37:35.569383+0000","last_undegraded":"2024-05-10T10:37:35.569383+0000","last_fullsized":"2024-05-10T10:37:35.569383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:14:39.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566100+0000","last_change":"2024-05-10T10:35:47.908874+0000","last_active":"2024-05-10T10:37:35.566100+0000","last_peered":"2024-05-10T10:37:35.566100+0000","last_clean":"2024-05-10T10:37:35.566100+0000","last_became_active":"2024-05-10T10:35:47.908478+0000","last_became_peered":"2024-05-10T10:35:47.908478+0000","last_unstale":"2024-05-10T10:37:35.566100+0000","last_undegraded":"2024-05-10T10:37:35.566100+0000","last_fullsized":"2024-05-10T10:37:35.566100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:13.147762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285463+0000","last_change":"2024-05-10T10:35:50.925168+0000","last_active":"2024-05-10T10:38:33.285463+0000","last_peered":"2024-05-10T10:38:33.285463+0000","last_clean":"2024-05-10T10:38:33.285463+0000","last_became_active":"2024-05-10T10:35:50.924509+0000","last_became_peered":"2024-05-10T10:35:50.924509+0000","last_unstale":"2024-05-10T10:38:33.285463+0000","last_undegraded":"2024-05-10T10:38:33.285463+0000","last_fullsized":"2024-05-10T10:38:33.285463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:08.458195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569737+0000","last_change":"2024-05-10T10:36:35.144575+0000","last_active":"2024-05-10T10:37:35.569737+0000","last_peered":"2024-05-10T10:37:35.569737+0000","last_clean":"2024-05-10T10:37:35.569737+0000","last_became_active":"2024-05-10T10:36:35.144165+0000","last_became_peered":"2024-05-10T10:36:35.144165+0000","last_unstale":"2024-05-10T10:37:35.569737+0000","last_undegraded":"2024-05-10T10:37:35.569737+0000","last_fullsized":"2024-05-10T10:37:35.569737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:45.179163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":247,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.680223+0000","last_change":"2024-05-10T10:35:47.911208+0000","last_active":"2024-05-10T10:40:08.680223+0000","last_peered":"2024-05-10T10:40:08.680223+0000","last_clean":"2024-05-10T10:40:08.680223+0000","last_became_active":"2024-05-10T10:35:47.910820+0000","last_became_peered":"2024-05-10T10:35:47.910820+0000","last_unstale":"2024-05-10T10:40:08.680223+0000","last_undegraded":"2024-05-10T10:40:08.680223+0000","last_fullsized":"2024-05-10T10:40:08.680223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:04.804203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289463+0000","last_change":"2024-05-10T10:35:50.928195+0000","last_active":"2024-05-10T10:38:33.289463+0000","last_peered":"2024-05-10T10:38:33.289463+0000","last_clean":"2024-05-10T10:38:33.289463+0000","last_became_active":"2024-05-10T10:35:50.927832+0000","last_became_peered":"2024-05-10T10:35:50.927832+0000","last_unstale":"2024-05-10T10:38:33.289463+0000","last_undegraded":"2024-05-10T10:38:33.289463+0000","last_fullsized":"2024-05-10T10:38:33.289463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:03.071525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565054+0000","last_change":"2024-05-10T10:36:01.982483+0000","last_active":"2024-05-10T10:37:35.565054+0000","last_peered":"2024-05-10T10:37:35.565054+0000","last_clean":"2024-05-10T10:37:35.565054+0000","last_became_active":"2024-05-10T10:36:01.982117+0000","last_became_peered":"2024-05-10T10:36:01.982117+0000","last_unstale":"2024-05-10T10:37:35.565054+0000","last_undegraded":"2024-05-10T10:37:35.565054+0000","last_fullsized":"2024-05-10T10:37:35.565054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:08.145035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569474+0000","last_change":"2024-05-10T10:35:47.912929+0000","last_active":"2024-05-10T10:37:35.569474+0000","last_peered":"2024-05-10T10:37:35.569474+0000","last_clean":"2024-05-10T10:37:35.569474+0000","last_became_active":"2024-05-10T10:35:47.912575+0000","last_became_peered":"2024-05-10T10:35:47.912575+0000","last_unstale":"2024-05-10T10:37:35.569474+0000","last_undegraded":"2024-05-10T10:37:35.569474+0000","last_fullsized":"2024-05-10T10:37:35.569474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:03.235579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296070+0000","last_change":"2024-05-10T10:35:50.935967+0000","last_active":"2024-05-10T10:38:33.296070+0000","last_peered":"2024-05-10T10:38:33.296070+0000","last_clean":"2024-05-10T10:38:33.296070+0000","last_became_active":"2024-05-10T10:35:50.935635+0000","last_became_peered":"2024-05-10T10:35:50.935635+0000","last_unstale":"2024-05-10T10:38:33.296070+0000","last_undegraded":"2024-05-10T10:38:33.296070+0000","last_fullsized":"2024-05-10T10:38:33.296070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:02:53.681522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565799+0000","last_change":"2024-05-10T10:36:01.979488+0000","last_active":"2024-05-10T10:37:35.565799+0000","last_peered":"2024-05-10T10:37:35.565799+0000","last_clean":"2024-05-10T10:37:35.565799+0000","last_became_active":"2024-05-10T10:36:01.978839+0000","last_became_peered":"2024-05-10T10:36:01.978839+0000","last_unstale":"2024-05-10T10:37:35.565799+0000","last_undegraded":"2024-05-10T10:37:35.565799+0000","last_fullsized":"2024-05-10T10:37:35.565799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:20.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:02.476105+0000","last_change":"2024-05-10T10:35:47.913235+0000","last_active":"2024-05-10T10:38:02.476105+0000","last_peered":"2024-05-10T10:38:02.476105+0000","last_clean":"2024-05-10T10:38:02.476105+0000","last_became_active":"2024-05-10T10:35:47.912788+0000","last_became_peered":"2024-05-10T10:35:47.912788+0000","last_unstale":"2024-05-10T10:38:02.476105+0000","last_undegraded":"2024-05-10T10:38:02.476105+0000","last_fullsized":"2024-05-10T10:38:02.476105+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:56:10.275488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290951+0000","last_change":"2024-05-10T10:35:50.930269+0000","last_active":"2024-05-10T10:38:33.290951+0000","last_peered":"2024-05-10T10:38:33.290951+0000","last_clean":"2024-05-10T10:38:33.290951+0000","last_became_active":"2024-05-10T10:35:50.929918+0000","last_became_peered":"2024-05-10T10:35:50.929918+0000","last_unstale":"2024-05-10T10:38:33.290951+0000","last_undegraded":"2024-05-10T10:38:33.290951+0000","last_fullsized":"2024-05-10T10:38:33.290951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:31.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569677+0000","last_change":"2024-05-10T10:36:02.018506+0000","last_active":"2024-05-10T10:37:35.569677+0000","last_peered":"2024-05-10T10:37:35.569677+0000","last_clean":"2024-05-10T10:37:35.569677+0000","last_became_active":"2024-05-10T10:36:02.018004+0000","last_became_peered":"2024-05-10T10:36:02.018004+0000","last_unstale":"2024-05-10T10:37:35.569677+0000","last_undegraded":"2024-05-10T10:37:35.569677+0000","last_fullsized":"2024-05-10T10:37:35.569677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:37.404970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:52.729668+0000","last_change":"2024-05-10T10:35:47.910654+0000","last_active":"2024-05-10T10:37:52.729668+0000","last_peered":"2024-05-10T10:37:52.729668+0000","last_clean":"2024-05-10T10:37:52.729668+0000","last_became_active":"2024-05-10T10:35:47.910338+0000","last_became_peered":"2024-05-10T10:35:47.910338+0000","last_unstale":"2024-05-10T10:37:52.729668+0000","last_undegraded":"2024-05-10T10:37:52.729668+0000","last_fullsized":"2024-05-10T10:37:52.729668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:07.127574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290279+0000","last_change":"2024-05-10T10:35:50.940950+0000","last_active":"2024-05-10T10:38:33.290279+0000","last_peered":"2024-05-10T10:38:33.290279+0000","last_clean":"2024-05-10T10:38:33.290279+0000","last_became_active":"2024-05-10T10:35:50.940562+0000","last_became_peered":"2024-05-10T10:35:50.940562+0000","last_unstale":"2024-05-10T10:38:33.290279+0000","last_undegraded":"2024-05-10T10:38:33.290279+0000","last_fullsized":"2024-05-10T10:38:33.290279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:24.825900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564778+0000","last_change":"2024-05-10T10:36:01.989277+0000","last_active":"2024-05-10T10:37:35.564778+0000","last_peered":"2024-05-10T10:37:35.564778+0000","last_clean":"2024-05-10T10:37:35.564778+0000","last_became_active":"2024-05-10T10:36:01.988866+0000","last_became_peered":"2024-05-10T10:36:01.988866+0000","last_unstale":"2024-05-10T10:37:35.564778+0000","last_undegraded":"2024-05-10T10:37:35.564778+0000","last_fullsized":"2024-05-10T10:37:35.564778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:08:29.740044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":244,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:09.662717+0000","last_change":"2024-05-10T10:35:47.908332+0000","last_active":"2024-05-10T10:40:09.662717+0000","last_peered":"2024-05-10T10:40:09.662717+0000","last_clean":"2024-05-10T10:40:09.662717+0000","last_became_active":"2024-05-10T10:35:47.907933+0000","last_became_peered":"2024-05-10T10:35:47.907933+0000","last_unstale":"2024-05-10T10:40:09.662717+0000","last_undegraded":"2024-05-10T10:40:09.662717+0000","last_fullsized":"2024-05-10T10:40:09.662717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:47:48.702338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290317+0000","last_change":"2024-05-10T10:35:50.934638+0000","last_active":"2024-05-10T10:38:33.290317+0000","last_peered":"2024-05-10T10:38:33.290317+0000","last_clean":"2024-05-10T10:38:33.290317+0000","last_became_active":"2024-05-10T10:35:50.934285+0000","last_became_peered":"2024-05-10T10:35:50.934285+0000","last_unstale":"2024-05-10T10:38:33.290317+0000","last_undegraded":"2024-05-10T10:38:33.290317+0000","last_fullsized":"2024-05-10T10:38:33.290317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:24:31.179005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565003+0000","last_change":"2024-05-10T10:36:02.018468+0000","last_active":"2024-05-10T10:37:35.565003+0000","last_peered":"2024-05-10T10:37:35.565003+0000","last_clean":"2024-05-10T10:37:35.565003+0000","last_became_active":"2024-05-10T10:36:02.018087+0000","last_became_peered":"2024-05-10T10:36:02.018087+0000","last_unstale":"2024-05-10T10:37:35.565003+0000","last_undegraded":"2024-05-10T10:37:35.565003+0000","last_fullsized":"2024-05-10T10:37:35.565003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:55.712012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295463+0000","last_change":"2024-05-10T10:35:50.933660+0000","last_active":"2024-05-10T10:38:33.295463+0000","last_peered":"2024-05-10T10:38:33.295463+0000","last_clean":"2024-05-10T10:38:33.295463+0000","last_became_active":"2024-05-10T10:35:50.933230+0000","last_became_peered":"2024-05-10T10:35:50.933230+0000","last_unstale":"2024-05-10T10:38:33.295463+0000","last_undegraded":"2024-05-10T10:38:33.295463+0000","last_fullsized":"2024-05-10T10:38:33.295463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:34:54.272278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.679698+0000","last_change":"2024-05-10T10:35:47.910161+0000","last_active":"2024-05-10T10:40:08.679698+0000","last_peered":"2024-05-10T10:40:08.679698+0000","last_clean":"2024-05-10T10:40:08.679698+0000","last_became_active":"2024-05-10T10:35:47.909875+0000","last_became_peered":"2024-05-10T10:35:47.909875+0000","last_unstale":"2024-05-10T10:40:08.679698+0000","last_undegraded":"2024-05-10T10:40:08.679698+0000","last_fullsized":"2024-05-10T10:40:08.679698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:21.620921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.205272+0000","last_change":"2024-05-10T10:36:01.986755+0000","last_active":"2024-05-10T10:38:06.205272+0000","last_peered":"2024-05-10T10:38:06.205272+0000","last_clean":"2024-05-10T10:38:06.205272+0000","last_became_active":"2024-05-10T10:36:01.986340+0000","last_became_peered":"2024-05-10T10:36:01.986340+0000","last_unstale":"2024-05-10T10:38:06.205272+0000","last_undegraded":"2024-05-10T10:38:06.205272+0000","last_fullsized":"2024-05-10T10:38:06.205272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:38.434323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293573+0000","last_change":"2024-05-10T10:36:35.114218+0000","last_active":"2024-05-10T10:38:33.293573+0000","last_peered":"2024-05-10T10:38:33.293573+0000","last_clean":"2024-05-10T10:38:33.293573+0000","last_became_active":"2024-05-10T10:36:35.113876+0000","last_became_peered":"2024-05-10T10:36:35.113876+0000","last_unstale":"2024-05-10T10:38:33.293573+0000","last_undegraded":"2024-05-10T10:38:33.293573+0000","last_fullsized":"2024-05-10T10:38:33.293573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:21.415678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:55.105058+0000","last_change":"2024-05-10T10:35:47.909161+0000","last_active":"2024-05-10T10:37:55.105058+0000","last_peered":"2024-05-10T10:37:55.105058+0000","last_clean":"2024-05-10T10:37:55.105058+0000","last_became_active":"2024-05-10T10:35:47.908546+0000","last_became_peered":"2024-05-10T10:35:47.908546+0000","last_unstale":"2024-05-10T10:37:55.105058+0000","last_undegraded":"2024-05-10T10:37:55.105058+0000","last_fullsized":"2024-05-10T10:37:55.105058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:52.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719232+0000","last_change":"2024-05-10T10:34:20.980948+0000","last_active":"2024-05-10T10:37:35.719232+0000","last_peered":"2024-05-10T10:37:35.719232+0000","last_clean":"2024-05-10T10:37:35.719232+0000","last_became_active":"2024-05-10T10:34:20.975662+0000","last_became_peered":"2024-05-10T10:34:20.975662+0000","last_unstale":"2024-05-10T10:37:35.719232+0000","last_undegraded":"2024-05-10T10:37:35.719232+0000","last_fullsized":"2024-05-10T10:37:35.719232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_clean_scrub_stamp":"2024-05-10T10:30:55.419211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:33.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566301+0000","last_change":"2024-05-10T10:36:02.018578+0000","last_active":"2024-05-10T10:37:35.566301+0000","last_peered":"2024-05-10T10:37:35.566301+0000","last_clean":"2024-05-10T10:37:35.566301+0000","last_became_active":"2024-05-10T10:36:02.018126+0000","last_became_peered":"2024-05-10T10:36:02.018126+0000","last_unstale":"2024-05-10T10:37:35.566301+0000","last_undegraded":"2024-05-10T10:37:35.566301+0000","last_fullsized":"2024-05-10T10:37:35.566301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:18:17.112432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285918+0000","last_change":"2024-05-10T10:35:50.927472+0000","last_active":"2024-05-10T10:38:33.285918+0000","last_peered":"2024-05-10T10:38:33.285918+0000","last_clean":"2024-05-10T10:38:33.285918+0000","last_became_active":"2024-05-10T10:35:50.927030+0000","last_became_peered":"2024-05-10T10:35:50.927030+0000","last_unstale":"2024-05-10T10:38:33.285918+0000","last_undegraded":"2024-05-10T10:38:33.285918+0000","last_fullsized":"2024-05-10T10:38:33.285918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:15:03.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:51.203426+0000","last_change":"2024-05-10T10:35:47.912675+0000","last_active":"2024-05-10T10:37:51.203426+0000","last_peered":"2024-05-10T10:37:51.203426+0000","last_clean":"2024-05-10T10:37:51.203426+0000","last_became_active":"2024-05-10T10:35:47.912297+0000","last_became_peered":"2024-05-10T10:35:47.912297+0000","last_unstale":"2024-05-10T10:37:51.203426+0000","last_undegraded":"2024-05-10T10:37:51.203426+0000","last_fullsized":"2024-05-10T10:37:51.203426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:46.178221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.047984+0000","last_change":"2024-05-10T10:36:01.981501+0000","last_active":"2024-05-10T10:38:06.047984+0000","last_peered":"2024-05-10T10:38:06.047984+0000","last_clean":"2024-05-10T10:38:06.047984+0000","last_became_active":"2024-05-10T10:36:01.981029+0000","last_became_peered":"2024-05-10T10:36:01.981029+0000","last_unstale":"2024-05-10T10:38:06.047984+0000","last_undegraded":"2024-05-10T10:38:06.047984+0000","last_fullsized":"2024-05-10T10:38:06.047984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:18.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:52.474989+0000","last_change":"2024-05-10T10:35:47.910701+0000","last_active":"2024-05-10T10:37:52.474989+0000","last_peered":"2024-05-10T10:37:52.474989+0000","last_clean":"2024-05-10T10:37:52.474989+0000","last_became_active":"2024-05-10T10:35:47.910286+0000","last_became_peered":"2024-05-10T10:35:47.910286+0000","last_unstale":"2024-05-10T10:37:52.474989+0000","last_undegraded":"2024-05-10T10:37:52.474989+0000","last_fullsized":"2024-05-10T10:37:52.474989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:07.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286141+0000","last_change":"2024-05-10T10:35:50.926026+0000","last_active":"2024-05-10T10:38:33.286141+0000","last_peered":"2024-05-10T10:38:33.286141+0000","last_clean":"2024-05-10T10:38:33.286141+0000","last_became_active":"2024-05-10T10:35:50.925624+0000","last_became_peered":"2024-05-10T10:35:50.925624+0000","last_unstale":"2024-05-10T10:38:33.286141+0000","last_undegraded":"2024-05-10T10:38:33.286141+0000","last_fullsized":"2024-05-10T10:38:33.286141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:22:02.828681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:03.903648+0000","last_change":"2024-05-10T10:36:02.018403+0000","last_active":"2024-05-10T10:38:03.903648+0000","last_peered":"2024-05-10T10:38:03.903648+0000","last_clean":"2024-05-10T10:38:03.903648+0000","last_became_active":"2024-05-10T10:36:02.017980+0000","last_became_peered":"2024-05-10T10:36:02.017980+0000","last_unstale":"2024-05-10T10:38:03.903648+0000","last_undegraded":"2024-05-10T10:38:03.903648+0000","last_fullsized":"2024-05-10T10:38:03.903648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:07.264730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:05.971094+0000","last_change":"2024-05-10T10:37:35.577657+0000","last_active":"2024-05-10T10:40:05.971094+0000","last_peered":"2024-05-10T10:40:05.971094+0000","last_clean":"2024-05-10T10:40:05.971094+0000","last_became_active":"2024-05-10T10:37:35.570901+0000","last_became_peered":"2024-05-10T10:37:35.570901+0000","last_unstale":"2024-05-10T10:40:05.971094+0000","last_undegraded":"2024-05-10T10:40:05.971094+0000","last_fullsized":"2024-05-10T10:40:05.971094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:30.088160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293071+0000","last_change":"2024-05-10T10:35:50.935817+0000","last_active":"2024-05-10T10:38:33.293071+0000","last_peered":"2024-05-10T10:38:33.293071+0000","last_clean":"2024-05-10T10:38:33.293071+0000","last_became_active":"2024-05-10T10:35:50.935515+0000","last_became_peered":"2024-05-10T10:35:50.935515+0000","last_unstale":"2024-05-10T10:38:33.293071+0000","last_undegraded":"2024-05-10T10:38:33.293071+0000","last_fullsized":"2024-05-10T10:38:33.293071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:49.042193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571516+0000","last_change":"2024-05-10T10:36:01.986002+0000","last_active":"2024-05-10T10:37:35.571516+0000","last_peered":"2024-05-10T10:37:35.571516+0000","last_clean":"2024-05-10T10:37:35.571516+0000","last_became_active":"2024-05-10T10:36:01.985558+0000","last_became_peered":"2024-05-10T10:36:01.985558+0000","last_unstale":"2024-05-10T10:37:35.571516+0000","last_undegraded":"2024-05-10T10:37:35.571516+0000","last_fullsized":"2024-05-10T10:37:35.571516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:27.732316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:00.105594+0000","last_change":"2024-05-10T10:35:47.909254+0000","last_active":"2024-05-10T10:38:00.105594+0000","last_peered":"2024-05-10T10:38:00.105594+0000","last_clean":"2024-05-10T10:38:00.105594+0000","last_became_active":"2024-05-10T10:35:47.908370+0000","last_became_peered":"2024-05-10T10:35:47.908370+0000","last_unstale":"2024-05-10T10:38:00.105594+0000","last_undegraded":"2024-05-10T10:38:00.105594+0000","last_fullsized":"2024-05-10T10:38:00.105594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:44.643592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287742+0000","last_change":"2024-05-10T10:35:50.933001+0000","last_active":"2024-05-10T10:38:33.287742+0000","last_peered":"2024-05-10T10:38:33.287742+0000","last_clean":"2024-05-10T10:38:33.287742+0000","last_became_active":"2024-05-10T10:35:50.932010+0000","last_became_peered":"2024-05-10T10:35:50.932010+0000","last_unstale":"2024-05-10T10:38:33.287742+0000","last_undegraded":"2024-05-10T10:38:33.287742+0000","last_fullsized":"2024-05-10T10:38:33.287742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:01.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720023+0000","last_change":"2024-05-10T10:36:02.018650+0000","last_active":"2024-05-10T10:37:35.720023+0000","last_peered":"2024-05-10T10:37:35.720023+0000","last_clean":"2024-05-10T10:37:35.720023+0000","last_became_active":"2024-05-10T10:36:02.018258+0000","last_became_peered":"2024-05-10T10:36:02.018258+0000","last_unstale":"2024-05-10T10:37:35.720023+0000","last_undegraded":"2024-05-10T10:37:35.720023+0000","last_fullsized":"2024-05-10T10:37:35.720023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:33.590752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:57.007213+0000","last_change":"2024-05-10T10:35:47.910419+0000","last_active":"2024-05-10T10:37:57.007213+0000","last_peered":"2024-05-10T10:37:57.007213+0000","last_clean":"2024-05-10T10:37:57.007213+0000","last_became_active":"2024-05-10T10:35:47.909930+0000","last_became_peered":"2024-05-10T10:35:47.909930+0000","last_unstale":"2024-05-10T10:37:57.007213+0000","last_undegraded":"2024-05-10T10:37:57.007213+0000","last_fullsized":"2024-05-10T10:37:57.007213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:32.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.250953+0000","last_change":"2024-05-10T10:35:50.923207+0000","last_active":"2024-05-10T10:38:33.250953+0000","last_peered":"2024-05-10T10:38:33.250953+0000","last_clean":"2024-05-10T10:38:33.250953+0000","last_became_active":"2024-05-10T10:35:50.922866+0000","last_became_peered":"2024-05-10T10:35:50.922866+0000","last_unstale":"2024-05-10T10:38:33.250953+0000","last_undegraded":"2024-05-10T10:38:33.250953+0000","last_fullsized":"2024-05-10T10:38:33.250953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:48.096498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567305+0000","last_change":"2024-05-10T10:36:01.985399+0000","last_active":"2024-05-10T10:37:35.567305+0000","last_peered":"2024-05-10T10:37:35.567305+0000","last_clean":"2024-05-10T10:37:35.567305+0000","last_became_active":"2024-05-10T10:36:01.985016+0000","last_became_peered":"2024-05-10T10:36:01.985016+0000","last_unstale":"2024-05-10T10:37:35.567305+0000","last_undegraded":"2024-05-10T10:37:35.567305+0000","last_fullsized":"2024-05-10T10:37:35.567305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:12.151693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:58.902976+0000","last_change":"2024-05-10T10:35:47.908594+0000","last_active":"2024-05-10T10:37:58.902976+0000","last_peered":"2024-05-10T10:37:58.902976+0000","last_clean":"2024-05-10T10:37:58.902976+0000","last_became_active":"2024-05-10T10:35:47.908278+0000","last_became_peered":"2024-05-10T10:35:47.908278+0000","last_unstale":"2024-05-10T10:37:58.902976+0000","last_undegraded":"2024-05-10T10:37:58.902976+0000","last_fullsized":"2024-05-10T10:37:58.902976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:24.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295025+0000","last_change":"2024-05-10T10:35:50.933089+0000","last_active":"2024-05-10T10:38:33.295025+0000","last_peered":"2024-05-10T10:38:33.295025+0000","last_clean":"2024-05-10T10:38:33.295025+0000","last_became_active":"2024-05-10T10:35:50.932739+0000","last_became_peered":"2024-05-10T10:35:50.932739+0000","last_unstale":"2024-05-10T10:38:33.295025+0000","last_undegraded":"2024-05-10T10:38:33.295025+0000","last_fullsized":"2024-05-10T10:38:33.295025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:15.302832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720006+0000","last_change":"2024-05-10T10:36:01.988561+0000","last_active":"2024-05-10T10:37:35.720006+0000","last_peered":"2024-05-10T10:37:35.720006+0000","last_clean":"2024-05-10T10:37:35.720006+0000","last_became_active":"2024-05-10T10:36:01.988120+0000","last_became_peered":"2024-05-10T10:36:01.988120+0000","last_unstale":"2024-05-10T10:37:35.720006+0000","last_undegraded":"2024-05-10T10:37:35.720006+0000","last_fullsized":"2024-05-10T10:37:35.720006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:35:37.120469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:39:46.068283+0000","last_change":"2024-05-10T10:37:35.580875+0000","last_active":"2024-05-10T10:39:46.068283+0000","last_peered":"2024-05-10T10:39:46.068283+0000","last_clean":"2024-05-10T10:39:46.068283+0000","last_became_active":"2024-05-10T10:37:35.572187+0000","last_became_peered":"2024-05-10T10:37:35.572187+0000","last_unstale":"2024-05-10T10:39:46.068283+0000","last_undegraded":"2024-05-10T10:39:46.068283+0000","last_fullsized":"2024-05-10T10:39:46.068283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:06.330404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290565+0000","last_change":"2024-05-10T10:35:50.938466+0000","last_active":"2024-05-10T10:38:33.290565+0000","last_peered":"2024-05-10T10:38:33.290565+0000","last_clean":"2024-05-10T10:38:33.290565+0000","last_became_active":"2024-05-10T10:35:50.938167+0000","last_became_peered":"2024-05-10T10:35:50.938167+0000","last_unstale":"2024-05-10T10:38:33.290565+0000","last_undegraded":"2024-05-10T10:38:33.290565+0000","last_fullsized":"2024-05-10T10:38:33.290565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:18.014708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719597+0000","last_change":"2024-05-10T10:36:02.018661+0000","last_active":"2024-05-10T10:37:35.719597+0000","last_peered":"2024-05-10T10:37:35.719597+0000","last_clean":"2024-05-10T10:37:35.719597+0000","last_became_active":"2024-05-10T10:36:02.018196+0000","last_became_peered":"2024-05-10T10:36:02.018196+0000","last_unstale":"2024-05-10T10:37:35.719597+0000","last_undegraded":"2024-05-10T10:37:35.719597+0000","last_fullsized":"2024-05-10T10:37:35.719597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:11.605012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570017+0000","last_change":"2024-05-10T10:35:47.917009+0000","last_active":"2024-05-10T10:37:35.570017+0000","last_peered":"2024-05-10T10:37:35.570017+0000","last_clean":"2024-05-10T10:37:35.570017+0000","last_became_active":"2024-05-10T10:35:47.916536+0000","last_became_peered":"2024-05-10T10:35:47.916536+0000","last_unstale":"2024-05-10T10:37:35.570017+0000","last_undegraded":"2024-05-10T10:37:35.570017+0000","last_fullsized":"2024-05-10T10:37:35.570017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:17.783027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287116+0000","last_change":"2024-05-10T10:35:50.926173+0000","last_active":"2024-05-10T10:38:33.287116+0000","last_peered":"2024-05-10T10:38:33.287116+0000","last_clean":"2024-05-10T10:38:33.287116+0000","last_became_active":"2024-05-10T10:35:50.925846+0000","last_became_peered":"2024-05-10T10:35:50.925846+0000","last_unstale":"2024-05-10T10:38:33.287116+0000","last_undegraded":"2024-05-10T10:38:33.287116+0000","last_fullsized":"2024-05-10T10:38:33.287116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:29:02.587667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565679+0000","last_change":"2024-05-10T10:36:01.981312+0000","last_active":"2024-05-10T10:37:35.565679+0000","last_peered":"2024-05-10T10:37:35.565679+0000","last_clean":"2024-05-10T10:37:35.565679+0000","last_became_active":"2024-05-10T10:36:01.980861+0000","last_became_peered":"2024-05-10T10:36:01.980861+0000","last_unstale":"2024-05-10T10:37:35.565679+0000","last_undegraded":"2024-05-10T10:37:35.565679+0000","last_fullsized":"2024-05-10T10:37:35.565679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:29.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.680524+0000","last_change":"2024-05-10T10:35:47.911120+0000","last_active":"2024-05-10T10:40:08.680524+0000","last_peered":"2024-05-10T10:40:08.680524+0000","last_clean":"2024-05-10T10:40:08.680524+0000","last_became_active":"2024-05-10T10:35:47.909945+0000","last_became_peered":"2024-05-10T10:35:47.909945+0000","last_unstale":"2024-05-10T10:40:08.680524+0000","last_undegraded":"2024-05-10T10:40:08.680524+0000","last_fullsized":"2024-05-10T10:40:08.680524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:40.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287370+0000","last_change":"2024-05-10T10:35:50.933855+0000","last_active":"2024-05-10T10:38:33.287370+0000","last_peered":"2024-05-10T10:38:33.287370+0000","last_clean":"2024-05-10T10:38:33.287370+0000","last_became_active":"2024-05-10T10:35:50.933502+0000","last_became_peered":"2024-05-10T10:35:50.933502+0000","last_unstale":"2024-05-10T10:38:33.287370+0000","last_undegraded":"2024-05-10T10:38:33.287370+0000","last_fullsized":"2024-05-10T10:38:33.287370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:36.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718378+0000","last_change":"2024-05-10T10:36:01.985133+0000","last_active":"2024-05-10T10:37:35.718378+0000","last_peered":"2024-05-10T10:37:35.718378+0000","last_clean":"2024-05-10T10:37:35.718378+0000","last_became_active":"2024-05-10T10:36:01.984758+0000","last_became_peered":"2024-05-10T10:36:01.984758+0000","last_unstale":"2024-05-10T10:37:35.718378+0000","last_undegraded":"2024-05-10T10:37:35.718378+0000","last_fullsized":"2024-05-10T10:37:35.718378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:52.733063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566564+0000","last_change":"2024-05-10T10:35:47.908645+0000","last_active":"2024-05-10T10:37:35.566564+0000","last_peered":"2024-05-10T10:37:35.566564+0000","last_clean":"2024-05-10T10:37:35.566564+0000","last_became_active":"2024-05-10T10:35:47.908123+0000","last_became_peered":"2024-05-10T10:35:47.908123+0000","last_unstale":"2024-05-10T10:37:35.566564+0000","last_undegraded":"2024-05-10T10:37:35.566564+0000","last_fullsized":"2024-05-10T10:37:35.566564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:46.181724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289675+0000","last_change":"2024-05-10T10:35:50.929215+0000","last_active":"2024-05-10T10:38:33.289675+0000","last_peered":"2024-05-10T10:38:33.289675+0000","last_clean":"2024-05-10T10:38:33.289675+0000","last_became_active":"2024-05-10T10:35:50.928907+0000","last_became_peered":"2024-05-10T10:35:50.928907+0000","last_unstale":"2024-05-10T10:38:33.289675+0000","last_undegraded":"2024-05-10T10:38:33.289675+0000","last_fullsized":"2024-05-10T10:38:33.289675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:57:17.340687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565667+0000","last_change":"2024-05-10T10:36:01.988548+0000","last_active":"2024-05-10T10:37:35.565667+0000","last_peered":"2024-05-10T10:37:35.565667+0000","last_clean":"2024-05-10T10:37:35.565667+0000","last_became_active":"2024-05-10T10:36:01.987221+0000","last_became_peered":"2024-05-10T10:36:01.987221+0000","last_unstale":"2024-05-10T10:37:35.565667+0000","last_undegraded":"2024-05-10T10:37:35.565667+0000","last_fullsized":"2024-05-10T10:37:35.565667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:55:22.434536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565063+0000","last_change":"2024-05-10T10:35:47.914022+0000","last_active":"2024-05-10T10:37:35.565063+0000","last_peered":"2024-05-10T10:37:35.565063+0000","last_clean":"2024-05-10T10:37:35.565063+0000","last_became_active":"2024-05-10T10:35:47.913711+0000","last_became_peered":"2024-05-10T10:35:47.913711+0000","last_unstale":"2024-05-10T10:37:35.565063+0000","last_undegraded":"2024-05-10T10:37:35.565063+0000","last_fullsized":"2024-05-10T10:37:35.565063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:13.291307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284901+0000","last_change":"2024-05-10T10:35:50.927452+0000","last_active":"2024-05-10T10:38:33.284901+0000","last_peered":"2024-05-10T10:38:33.284901+0000","last_clean":"2024-05-10T10:38:33.284901+0000","last_became_active":"2024-05-10T10:35:50.927102+0000","last_became_peered":"2024-05-10T10:35:50.927102+0000","last_unstale":"2024-05-10T10:38:33.284901+0000","last_undegraded":"2024-05-10T10:38:33.284901+0000","last_fullsized":"2024-05-10T10:38:33.284901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:07.704196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293454+0000","last_change":"2024-05-10T10:35:50.933072+0000","last_active":"2024-05-10T10:38:33.293454+0000","last_peered":"2024-05-10T10:38:33.293454+0000","last_clean":"2024-05-10T10:38:33.293454+0000","last_became_active":"2024-05-10T10:35:50.932724+0000","last_became_peered":"2024-05-10T10:35:50.932724+0000","last_unstale":"2024-05-10T10:38:33.293454+0000","last_undegraded":"2024-05-10T10:38:33.293454+0000","last_fullsized":"2024-05-10T10:38:33.293454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:08.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287960+0000","last_change":"2024-05-10T10:35:50.931635+0000","last_active":"2024-05-10T10:38:33.287960+0000","last_peered":"2024-05-10T10:38:33.287960+0000","last_clean":"2024-05-10T10:38:33.287960+0000","last_became_active":"2024-05-10T10:35:50.930158+0000","last_became_peered":"2024-05-10T10:35:50.930158+0000","last_unstale":"2024-05-10T10:38:33.287960+0000","last_undegraded":"2024-05-10T10:38:33.287960+0000","last_fullsized":"2024-05-10T10:38:33.287960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:22.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285388+0000","last_change":"2024-05-10T10:35:50.936947+0000","last_active":"2024-05-10T10:38:33.285388+0000","last_peered":"2024-05-10T10:38:33.285388+0000","last_clean":"2024-05-10T10:38:33.285388+0000","last_became_active":"2024-05-10T10:35:50.936156+0000","last_became_peered":"2024-05-10T10:35:50.936156+0000","last_unstale":"2024-05-10T10:38:33.285388+0000","last_undegraded":"2024-05-10T10:38:33.285388+0000","last_fullsized":"2024-05-10T10:38:33.285388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:21.167787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294726+0000","last_change":"2024-05-10T10:35:50.935384+0000","last_active":"2024-05-10T10:38:33.294726+0000","last_peered":"2024-05-10T10:38:33.294726+0000","last_clean":"2024-05-10T10:38:33.294726+0000","last_became_active":"2024-05-10T10:35:50.932555+0000","last_became_peered":"2024-05-10T10:35:50.932555+0000","last_unstale":"2024-05-10T10:38:33.294726+0000","last_undegraded":"2024-05-10T10:38:33.294726+0000","last_fullsized":"2024-05-10T10:38:33.294726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:09:11.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295227+0000","last_change":"2024-05-10T10:35:50.929322+0000","last_active":"2024-05-10T10:38:33.295227+0000","last_peered":"2024-05-10T10:38:33.295227+0000","last_clean":"2024-05-10T10:38:33.295227+0000","last_became_active":"2024-05-10T10:35:50.928366+0000","last_became_peered":"2024-05-10T10:35:50.928366+0000","last_unstale":"2024-05-10T10:38:33.295227+0000","last_undegraded":"2024-05-10T10:38:33.295227+0000","last_fullsized":"2024-05-10T10:38:33.295227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:32:28.337962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292904+0000","last_change":"2024-05-10T10:36:35.110922+0000","last_active":"2024-05-10T10:38:33.292904+0000","last_peered":"2024-05-10T10:38:33.292904+0000","last_clean":"2024-05-10T10:38:33.292904+0000","last_became_active":"2024-05-10T10:36:35.110525+0000","last_became_peered":"2024-05-10T10:36:35.110525+0000","last_unstale":"2024-05-10T10:38:33.292904+0000","last_undegraded":"2024-05-10T10:38:33.292904+0000","last_fullsized":"2024-05-10T10:38:33.292904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:47.825168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288252+0000","last_change":"2024-05-10T10:35:50.927869+0000","last_active":"2024-05-10T10:38:33.288252+0000","last_peered":"2024-05-10T10:38:33.288252+0000","last_clean":"2024-05-10T10:38:33.288252+0000","last_became_active":"2024-05-10T10:35:50.927552+0000","last_became_peered":"2024-05-10T10:35:50.927552+0000","last_unstale":"2024-05-10T10:38:33.288252+0000","last_undegraded":"2024-05-10T10:38:33.288252+0000","last_fullsized":"2024-05-10T10:38:33.288252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:36.962316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285786+0000","last_change":"2024-05-10T10:35:50.933099+0000","last_active":"2024-05-10T10:38:33.285786+0000","last_peered":"2024-05-10T10:38:33.285786+0000","last_clean":"2024-05-10T10:38:33.285786+0000","last_became_active":"2024-05-10T10:35:50.932765+0000","last_became_peered":"2024-05-10T10:35:50.932765+0000","last_unstale":"2024-05-10T10:38:33.285786+0000","last_undegraded":"2024-05-10T10:38:33.285786+0000","last_fullsized":"2024-05-10T10:38:33.285786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:19.095217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294758+0000","last_change":"2024-05-10T10:35:50.937704+0000","last_active":"2024-05-10T10:38:33.294758+0000","last_peered":"2024-05-10T10:38:33.294758+0000","last_clean":"2024-05-10T10:38:33.294758+0000","last_became_active":"2024-05-10T10:35:50.937236+0000","last_became_peered":"2024-05-10T10:35:50.937236+0000","last_unstale":"2024-05-10T10:38:33.294758+0000","last_undegraded":"2024-05-10T10:38:33.294758+0000","last_fullsized":"2024-05-10T10:38:33.294758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:46.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292751+0000","last_change":"2024-05-10T10:35:50.935181+0000","last_active":"2024-05-10T10:38:33.292751+0000","last_peered":"2024-05-10T10:38:33.292751+0000","last_clean":"2024-05-10T10:38:33.292751+0000","last_became_active":"2024-05-10T10:35:50.933446+0000","last_became_peered":"2024-05-10T10:35:50.933446+0000","last_unstale":"2024-05-10T10:38:33.292751+0000","last_undegraded":"2024-05-10T10:38:33.292751+0000","last_fullsized":"2024-05-10T10:38:33.292751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:03.972410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285940+0000","last_change":"2024-05-10T10:35:50.931797+0000","last_active":"2024-05-10T10:38:33.285940+0000","last_peered":"2024-05-10T10:38:33.285940+0000","last_clean":"2024-05-10T10:38:33.285940+0000","last_became_active":"2024-05-10T10:35:50.931494+0000","last_became_peered":"2024-05-10T10:35:50.931494+0000","last_unstale":"2024-05-10T10:38:33.285940+0000","last_undegraded":"2024-05-10T10:38:33.285940+0000","last_fullsized":"2024-05-10T10:38:33.285940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:30.797462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296149+0000","last_change":"2024-05-10T10:35:50.931688+0000","last_active":"2024-05-10T10:38:33.296149+0000","last_peered":"2024-05-10T10:38:33.296149+0000","last_clean":"2024-05-10T10:38:33.296149+0000","last_became_active":"2024-05-10T10:35:50.931362+0000","last_became_peered":"2024-05-10T10:35:50.931362+0000","last_unstale":"2024-05-10T10:38:33.296149+0000","last_undegraded":"2024-05-10T10:38:33.296149+0000","last_fullsized":"2024-05-10T10:38:33.296149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:24.263752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292719+0000","last_change":"2024-05-10T10:35:50.930905+0000","last_active":"2024-05-10T10:38:33.292719+0000","last_peered":"2024-05-10T10:38:33.292719+0000","last_clean":"2024-05-10T10:38:33.292719+0000","last_became_active":"2024-05-10T10:35:50.930448+0000","last_became_peered":"2024-05-10T10:35:50.930448+0000","last_unstale":"2024-05-10T10:38:33.292719+0000","last_undegraded":"2024-05-10T10:38:33.292719+0000","last_fullsized":"2024-05-10T10:38:33.292719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:52:24.468600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76702,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1932,"num_read_kb":1048,"num_write":271,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3701,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":234420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678085,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":521483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874313,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1176607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037838,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234068,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430298,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1119415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626527,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":516917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":585723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":518373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":518660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":516132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":593613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1113479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:40:12.044 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-10T10:40:12.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:40:12.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:11 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4120567738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:40:12.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:11 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4120567738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 -- 172.21.15.112:0/3670939344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c1018d0 msgr2=0x7fd29c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3670939344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c1018d0 0x7fd29c101cf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd294009900 tx=0x7fd29402a410 comp rx=0 tx=0).stop 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 -- 172.21.15.112:0/3670939344 shutdown_connections 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3670939344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd29c103480 0x7fd29c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3670939344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c102ac0 0x7fd29c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3670939344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c1018d0 0x7fd29c101cf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 -- 172.21.15.112:0/3670939344 >> 172.21.15.112:0/3670939344 conn(0x7fd29c0fce40 msgr2=0x7fd29c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 -- 172.21.15.112:0/3670939344 shutdown_connections 2024-05-10T10:40:12.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.177+0000 7fd2a171c640 1 -- 172.21.15.112:0/3670939344 wait complete. 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 Processor -- start 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 -- start start 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd29c1018d0 0x7fd29c06a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c102ac0 0x7fd29c19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c103480 0x7fd29c19fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c10d2a0 con 0x7fd29c102ac0 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c10d120 con 0x7fd29c1018d0 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd2a171c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c10d420 con 0x7fd29c103480 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd29a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c102ac0 0x7fd29c19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd29a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c102ac0 0x7fd29c19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46488/0 (socket says 172.21.15.112:46488) 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd29a7fc640 1 -- 172.21.15.112:0/3156554800 learned_addr learned my addr 172.21.15.112:0/3156554800 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.178+0000 7fd29affd640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd29c1018d0 0x7fd29c06a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd29b7fe640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c103480 0x7fd29c19fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd29a7fc640 1 -- 172.21.15.112:0/3156554800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c103480 msgr2=0x7fd29c19fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd29a7fc640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c103480 0x7fd29c19fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd29a7fc640 1 -- 172.21.15.112:0/3156554800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd29c1018d0 msgr2=0x7fd29c06a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd29a7fc640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd29c1018d0 0x7fd29c06a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd29a7fc640 1 -- 172.21.15.112:0/3156554800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c1a07e0 con 0x7fd29c102ac0 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd29a7fc640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c102ac0 0x7fd29c19d8b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd29000d770 tx=0x7fd29000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd283fff640 1 -- 172.21.15.112:0/3156554800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290017070 con 0x7fd29c102ac0 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29c1a0a70 con 0x7fd29c102ac0 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd283fff640 1 -- 172.21.15.112:0/3156554800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2900054b0 con 0x7fd29c102ac0 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd283fff640 1 -- 172.21.15.112:0/3156554800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290004c60 con 0x7fd29c102ac0 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.179+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29c1a0f70 con 0x7fd29c102ac0 2024-05-10T10:40:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.180+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29c069840 con 0x7fd29c102ac0 2024-05-10T10:40:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.181+0000 7fd283fff640 1 -- 172.21.15.112:0/3156554800 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd290004e00 con 0x7fd29c102ac0 2024-05-10T10:40:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.182+0000 7fd283fff640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd2840b27f0 0x7fd2840b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.182+0000 7fd29affd640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd2840b27f0 0x7fd2840b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.182+0000 7fd283fff640 1 -- 172.21.15.112:0/3156554800 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd2900e2350 con 0x7fd29c102ac0 2024-05-10T10:40:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.182+0000 7fd29affd640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd2840b27f0 0x7fd2840b4cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd2940057c0 tx=0x7fd294031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:12.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.185+0000 7fd283fff640 1 -- 172.21.15.112:0/3156554800 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd2900a9d80 con 0x7fd29c102ac0 2024-05-10T10:40:12.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.366+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 --> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd29c062df0 con 0x7fd2840b27f0 2024-05-10T10:40:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.375+0000 7fd283fff640 1 -- 172.21.15.112:0/3156554800 <== mgr.14150 v2:172.21.15.112:6800/1898750131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7fd29c062df0 con 0x7fd2840b27f0 2024-05-10T10:40:12.381 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:12.383 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-10T10:40:12.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd2840b27f0 msgr2=0x7fd2840b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:12.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd2840b27f0 0x7fd2840b4cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd2940057c0 tx=0x7fd294031400 comp rx=0 tx=0).stop 2024-05-10T10:40:12.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c102ac0 msgr2=0x7fd29c19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:12.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c102ac0 0x7fd29c19d8b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd29000d770 tx=0x7fd29000dc30 comp rx=0 tx=0).stop 2024-05-10T10:40:12.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 shutdown_connections 2024-05-10T10:40:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd2840b27f0 0x7fd2840b4cb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd29c103480 0x7fd29c19fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd29c102ac0 0x7fd29c19d8b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 --2- 172.21.15.112:0/3156554800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd29c1018d0 0x7fd29c06a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 >> 172.21.15.112:0/3156554800 conn(0x7fd29c0fce40 msgr2=0x7fd29c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 shutdown_connections 2024-05-10T10:40:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:12.379+0000 7fd2a171c640 1 -- 172.21.15.112:0/3156554800 wait complete. 2024-05-10T10:40:12.391 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-05-10T10:40:11.354232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667304,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2008,"num_read_kb":1112,"num_write":396,"num_write_kb":2315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2955541,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":827,"ondisk_log_size":827,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367712,"kb_used_data":14400,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574112,"statfs":{"total":1151940427776,"available":1151563890688,"internally_reserved":0,"allocated":14745600,"data_stored":8226716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":60,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003248"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295705+0000","last_change":"2024-05-10T10:35:50.924449+0000","last_active":"2024-05-10T10:38:33.295705+0000","last_peered":"2024-05-10T10:38:33.295705+0000","last_clean":"2024-05-10T10:38:33.295705+0000","last_became_active":"2024-05-10T10:35:50.924114+0000","last_became_peered":"2024-05-10T10:35:50.924114+0000","last_unstale":"2024-05-10T10:38:33.295705+0000","last_undegraded":"2024-05-10T10:38:33.295705+0000","last_fullsized":"2024-05-10T10:38:33.295705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:50.636501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292255+0000","last_change":"2024-05-10T10:35:50.928715+0000","last_active":"2024-05-10T10:38:33.292255+0000","last_peered":"2024-05-10T10:38:33.292255+0000","last_clean":"2024-05-10T10:38:33.292255+0000","last_became_active":"2024-05-10T10:35:50.928404+0000","last_became_peered":"2024-05-10T10:35:50.928404+0000","last_unstale":"2024-05-10T10:38:33.292255+0000","last_undegraded":"2024-05-10T10:38:33.292255+0000","last_fullsized":"2024-05-10T10:38:33.292255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:44.934022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295067+0000","last_change":"2024-05-10T10:35:50.925134+0000","last_active":"2024-05-10T10:38:33.295067+0000","last_peered":"2024-05-10T10:38:33.295067+0000","last_clean":"2024-05-10T10:38:33.295067+0000","last_became_active":"2024-05-10T10:35:50.924797+0000","last_became_peered":"2024-05-10T10:35:50.924797+0000","last_unstale":"2024-05-10T10:38:33.295067+0000","last_undegraded":"2024-05-10T10:38:33.295067+0000","last_fullsized":"2024-05-10T10:38:33.295067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:43:52.666519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.280643+0000","last_change":"2024-05-10T10:35:50.930916+0000","last_active":"2024-05-10T10:38:33.280643+0000","last_peered":"2024-05-10T10:38:33.280643+0000","last_clean":"2024-05-10T10:38:33.280643+0000","last_became_active":"2024-05-10T10:35:50.930378+0000","last_became_peered":"2024-05-10T10:35:50.930378+0000","last_unstale":"2024-05-10T10:38:33.280643+0000","last_undegraded":"2024-05-10T10:38:33.280643+0000","last_fullsized":"2024-05-10T10:38:33.280643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:07.165595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.269516+0000","last_change":"2024-05-10T10:35:50.925459+0000","last_active":"2024-05-10T10:38:33.269516+0000","last_peered":"2024-05-10T10:38:33.269516+0000","last_clean":"2024-05-10T10:38:33.269516+0000","last_became_active":"2024-05-10T10:35:50.925061+0000","last_became_peered":"2024-05-10T10:35:50.925061+0000","last_unstale":"2024-05-10T10:38:33.269516+0000","last_undegraded":"2024-05-10T10:38:33.269516+0000","last_fullsized":"2024-05-10T10:38:33.269516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:48:06.622369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290891+0000","last_change":"2024-05-10T10:35:50.932052+0000","last_active":"2024-05-10T10:38:33.290891+0000","last_peered":"2024-05-10T10:38:33.290891+0000","last_clean":"2024-05-10T10:38:33.290891+0000","last_became_active":"2024-05-10T10:35:50.931685+0000","last_became_peered":"2024-05-10T10:35:50.931685+0000","last_unstale":"2024-05-10T10:38:33.290891+0000","last_undegraded":"2024-05-10T10:38:33.290891+0000","last_fullsized":"2024-05-10T10:38:33.290891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:13:39.930917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289511+0000","last_change":"2024-05-10T10:35:50.930620+0000","last_active":"2024-05-10T10:38:33.289511+0000","last_peered":"2024-05-10T10:38:33.289511+0000","last_clean":"2024-05-10T10:38:33.289511+0000","last_became_active":"2024-05-10T10:35:50.929696+0000","last_became_peered":"2024-05-10T10:35:50.929696+0000","last_unstale":"2024-05-10T10:38:33.289511+0000","last_undegraded":"2024-05-10T10:38:33.289511+0000","last_fullsized":"2024-05-10T10:38:33.289511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:23.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296397+0000","last_change":"2024-05-10T10:35:50.937517+0000","last_active":"2024-05-10T10:38:33.296397+0000","last_peered":"2024-05-10T10:38:33.296397+0000","last_clean":"2024-05-10T10:38:33.296397+0000","last_became_active":"2024-05-10T10:35:50.937180+0000","last_became_peered":"2024-05-10T10:35:50.937180+0000","last_unstale":"2024-05-10T10:38:33.296397+0000","last_undegraded":"2024-05-10T10:38:33.296397+0000","last_fullsized":"2024-05-10T10:38:33.296397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.568509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293452+0000","last_change":"2024-05-10T10:35:50.937825+0000","last_active":"2024-05-10T10:38:33.293452+0000","last_peered":"2024-05-10T10:38:33.293452+0000","last_clean":"2024-05-10T10:38:33.293452+0000","last_became_active":"2024-05-10T10:35:50.935650+0000","last_became_peered":"2024-05-10T10:35:50.935650+0000","last_unstale":"2024-05-10T10:38:33.293452+0000","last_undegraded":"2024-05-10T10:38:33.293452+0000","last_fullsized":"2024-05-10T10:38:33.293452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:43:55.780266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285811+0000","last_change":"2024-05-10T10:35:50.933598+0000","last_active":"2024-05-10T10:38:33.285811+0000","last_peered":"2024-05-10T10:38:33.285811+0000","last_clean":"2024-05-10T10:38:33.285811+0000","last_became_active":"2024-05-10T10:35:50.932767+0000","last_became_peered":"2024-05-10T10:35:50.932767+0000","last_unstale":"2024-05-10T10:38:33.285811+0000","last_undegraded":"2024-05-10T10:38:33.285811+0000","last_fullsized":"2024-05-10T10:38:33.285811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:40.610376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287212+0000","last_change":"2024-05-10T10:35:50.934882+0000","last_active":"2024-05-10T10:38:33.287212+0000","last_peered":"2024-05-10T10:38:33.287212+0000","last_clean":"2024-05-10T10:38:33.287212+0000","last_became_active":"2024-05-10T10:35:50.934528+0000","last_became_peered":"2024-05-10T10:35:50.934528+0000","last_unstale":"2024-05-10T10:38:33.287212+0000","last_undegraded":"2024-05-10T10:38:33.287212+0000","last_fullsized":"2024-05-10T10:38:33.287212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:15.319765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289796+0000","last_change":"2024-05-10T10:36:35.110939+0000","last_active":"2024-05-10T10:38:33.289796+0000","last_peered":"2024-05-10T10:38:33.289796+0000","last_clean":"2024-05-10T10:38:33.289796+0000","last_became_active":"2024-05-10T10:36:35.110600+0000","last_became_peered":"2024-05-10T10:36:35.110600+0000","last_unstale":"2024-05-10T10:38:33.289796+0000","last_undegraded":"2024-05-10T10:38:33.289796+0000","last_fullsized":"2024-05-10T10:38:33.289796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:45:58.547301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.297102+0000","last_change":"2024-05-10T10:35:50.934161+0000","last_active":"2024-05-10T10:38:33.297102+0000","last_peered":"2024-05-10T10:38:33.297102+0000","last_clean":"2024-05-10T10:38:33.297102+0000","last_became_active":"2024-05-10T10:35:50.933831+0000","last_became_peered":"2024-05-10T10:35:50.933831+0000","last_unstale":"2024-05-10T10:38:33.297102+0000","last_undegraded":"2024-05-10T10:38:33.297102+0000","last_fullsized":"2024-05-10T10:38:33.297102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:09.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286140+0000","last_change":"2024-05-10T10:35:50.927404+0000","last_active":"2024-05-10T10:38:33.286140+0000","last_peered":"2024-05-10T10:38:33.286140+0000","last_clean":"2024-05-10T10:38:33.286140+0000","last_became_active":"2024-05-10T10:35:50.927094+0000","last_became_peered":"2024-05-10T10:35:50.927094+0000","last_unstale":"2024-05-10T10:38:33.286140+0000","last_undegraded":"2024-05-10T10:38:33.286140+0000","last_fullsized":"2024-05-10T10:38:33.286140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:25.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293207+0000","last_change":"2024-05-10T10:36:35.142063+0000","last_active":"2024-05-10T10:38:33.293207+0000","last_peered":"2024-05-10T10:38:33.293207+0000","last_clean":"2024-05-10T10:38:33.293207+0000","last_became_active":"2024-05-10T10:36:35.141685+0000","last_became_peered":"2024-05-10T10:36:35.141685+0000","last_unstale":"2024-05-10T10:38:33.293207+0000","last_undegraded":"2024-05-10T10:38:33.293207+0000","last_fullsized":"2024-05-10T10:38:33.293207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:47.284288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294424+0000","last_change":"2024-05-10T10:35:50.933172+0000","last_active":"2024-05-10T10:38:33.294424+0000","last_peered":"2024-05-10T10:38:33.294424+0000","last_clean":"2024-05-10T10:38:33.294424+0000","last_became_active":"2024-05-10T10:35:50.931864+0000","last_became_peered":"2024-05-10T10:35:50.931864+0000","last_unstale":"2024-05-10T10:38:33.294424+0000","last_undegraded":"2024-05-10T10:38:33.294424+0000","last_fullsized":"2024-05-10T10:38:33.294424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:50.620315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295591+0000","last_change":"2024-05-10T10:35:50.925952+0000","last_active":"2024-05-10T10:38:33.295591+0000","last_peered":"2024-05-10T10:38:33.295591+0000","last_clean":"2024-05-10T10:38:33.295591+0000","last_became_active":"2024-05-10T10:35:50.925610+0000","last_became_peered":"2024-05-10T10:35:50.925610+0000","last_unstale":"2024-05-10T10:38:33.295591+0000","last_undegraded":"2024-05-10T10:38:33.295591+0000","last_fullsized":"2024-05-10T10:38:33.295591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:38.287235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291553+0000","last_change":"2024-05-10T10:35:50.929379+0000","last_active":"2024-05-10T10:38:33.291553+0000","last_peered":"2024-05-10T10:38:33.291553+0000","last_clean":"2024-05-10T10:38:33.291553+0000","last_became_active":"2024-05-10T10:35:50.929082+0000","last_became_peered":"2024-05-10T10:35:50.929082+0000","last_unstale":"2024-05-10T10:38:33.291553+0000","last_undegraded":"2024-05-10T10:38:33.291553+0000","last_fullsized":"2024-05-10T10:38:33.291553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:54.097855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.283992+0000","last_change":"2024-05-10T10:35:50.937754+0000","last_active":"2024-05-10T10:38:33.283992+0000","last_peered":"2024-05-10T10:38:33.283992+0000","last_clean":"2024-05-10T10:38:33.283992+0000","last_became_active":"2024-05-10T10:35:50.935662+0000","last_became_peered":"2024-05-10T10:35:50.935662+0000","last_unstale":"2024-05-10T10:38:33.283992+0000","last_undegraded":"2024-05-10T10:38:33.283992+0000","last_fullsized":"2024-05-10T10:38:33.283992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:52.692279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571937+0000","last_change":"2024-05-10T10:36:01.989463+0000","last_active":"2024-05-10T10:37:35.571937+0000","last_peered":"2024-05-10T10:37:35.571937+0000","last_clean":"2024-05-10T10:37:35.571937+0000","last_became_active":"2024-05-10T10:36:01.988949+0000","last_became_peered":"2024-05-10T10:36:01.988949+0000","last_unstale":"2024-05-10T10:37:35.571937+0000","last_undegraded":"2024-05-10T10:37:35.571937+0000","last_fullsized":"2024-05-10T10:37:35.571937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:47.529845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":248,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:06.657735+0000","last_change":"2024-05-10T10:35:47.909033+0000","last_active":"2024-05-10T10:40:06.657735+0000","last_peered":"2024-05-10T10:40:06.657735+0000","last_clean":"2024-05-10T10:40:06.657735+0000","last_became_active":"2024-05-10T10:35:47.908680+0000","last_became_peered":"2024-05-10T10:35:47.908680+0000","last_unstale":"2024-05-10T10:40:06.657735+0000","last_undegraded":"2024-05-10T10:40:06.657735+0000","last_fullsized":"2024-05-10T10:40:06.657735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:46.642113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294649+0000","last_change":"2024-05-10T10:35:50.939233+0000","last_active":"2024-05-10T10:38:33.294649+0000","last_peered":"2024-05-10T10:38:33.294649+0000","last_clean":"2024-05-10T10:38:33.294649+0000","last_became_active":"2024-05-10T10:35:50.938949+0000","last_became_peered":"2024-05-10T10:35:50.938949+0000","last_unstale":"2024-05-10T10:38:33.294649+0000","last_undegraded":"2024-05-10T10:38:33.294649+0000","last_fullsized":"2024-05-10T10:38:33.294649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:06.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718685+0000","last_change":"2024-05-10T10:36:01.990982+0000","last_active":"2024-05-10T10:37:35.718685+0000","last_peered":"2024-05-10T10:37:35.718685+0000","last_clean":"2024-05-10T10:37:35.718685+0000","last_became_active":"2024-05-10T10:36:01.990325+0000","last_became_peered":"2024-05-10T10:36:01.990325+0000","last_unstale":"2024-05-10T10:37:35.718685+0000","last_undegraded":"2024-05-10T10:37:35.718685+0000","last_fullsized":"2024-05-10T10:37:35.718685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:18.424130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:48.901957+0000","last_change":"2024-05-10T10:35:47.911239+0000","last_active":"2024-05-10T10:37:48.901957+0000","last_peered":"2024-05-10T10:37:48.901957+0000","last_clean":"2024-05-10T10:37:48.901957+0000","last_became_active":"2024-05-10T10:35:47.910945+0000","last_became_peered":"2024-05-10T10:35:47.910945+0000","last_unstale":"2024-05-10T10:37:48.901957+0000","last_undegraded":"2024-05-10T10:37:48.901957+0000","last_fullsized":"2024-05-10T10:37:48.901957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:08:52.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294800+0000","last_change":"2024-05-10T10:35:50.923694+0000","last_active":"2024-05-10T10:38:33.294800+0000","last_peered":"2024-05-10T10:38:33.294800+0000","last_clean":"2024-05-10T10:38:33.294800+0000","last_became_active":"2024-05-10T10:35:50.923381+0000","last_became_peered":"2024-05-10T10:35:50.923381+0000","last_unstale":"2024-05-10T10:38:33.294800+0000","last_undegraded":"2024-05-10T10:38:33.294800+0000","last_fullsized":"2024-05-10T10:38:33.294800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:59.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718663+0000","last_change":"2024-05-10T10:36:01.989868+0000","last_active":"2024-05-10T10:37:35.718663+0000","last_peered":"2024-05-10T10:37:35.718663+0000","last_clean":"2024-05-10T10:37:35.718663+0000","last_became_active":"2024-05-10T10:36:01.989283+0000","last_became_peered":"2024-05-10T10:36:01.989283+0000","last_unstale":"2024-05-10T10:37:35.718663+0000","last_undegraded":"2024-05-10T10:37:35.718663+0000","last_fullsized":"2024-05-10T10:37:35.718663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:35.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570220+0000","last_change":"2024-05-10T10:35:47.915555+0000","last_active":"2024-05-10T10:37:35.570220+0000","last_peered":"2024-05-10T10:37:35.570220+0000","last_clean":"2024-05-10T10:37:35.570220+0000","last_became_active":"2024-05-10T10:35:47.912852+0000","last_became_peered":"2024-05-10T10:35:47.912852+0000","last_unstale":"2024-05-10T10:37:35.570220+0000","last_undegraded":"2024-05-10T10:37:35.570220+0000","last_fullsized":"2024-05-10T10:37:35.570220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:56:45.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291583+0000","last_change":"2024-05-10T10:35:50.934010+0000","last_active":"2024-05-10T10:38:33.291583+0000","last_peered":"2024-05-10T10:38:33.291583+0000","last_clean":"2024-05-10T10:38:33.291583+0000","last_became_active":"2024-05-10T10:35:50.933653+0000","last_became_peered":"2024-05-10T10:35:50.933653+0000","last_unstale":"2024-05-10T10:38:33.291583+0000","last_undegraded":"2024-05-10T10:38:33.291583+0000","last_fullsized":"2024-05-10T10:38:33.291583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:34.862734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718657+0000","last_change":"2024-05-10T10:36:01.990524+0000","last_active":"2024-05-10T10:37:35.718657+0000","last_peered":"2024-05-10T10:37:35.718657+0000","last_clean":"2024-05-10T10:37:35.718657+0000","last_became_active":"2024-05-10T10:36:01.990011+0000","last_became_peered":"2024-05-10T10:36:01.990011+0000","last_unstale":"2024-05-10T10:37:35.718657+0000","last_undegraded":"2024-05-10T10:37:35.718657+0000","last_fullsized":"2024-05-10T10:37:35.718657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:06.123095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570506+0000","last_change":"2024-05-10T10:35:47.910977+0000","last_active":"2024-05-10T10:37:35.570506+0000","last_peered":"2024-05-10T10:37:35.570506+0000","last_clean":"2024-05-10T10:37:35.570506+0000","last_became_active":"2024-05-10T10:35:47.910623+0000","last_became_peered":"2024-05-10T10:35:47.910623+0000","last_unstale":"2024-05-10T10:37:35.570506+0000","last_undegraded":"2024-05-10T10:37:35.570506+0000","last_fullsized":"2024-05-10T10:37:35.570506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:37.760666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296932+0000","last_change":"2024-05-10T10:35:50.937238+0000","last_active":"2024-05-10T10:38:33.296932+0000","last_peered":"2024-05-10T10:38:33.296932+0000","last_clean":"2024-05-10T10:38:33.296932+0000","last_became_active":"2024-05-10T10:35:50.936298+0000","last_became_peered":"2024-05-10T10:35:50.936298+0000","last_unstale":"2024-05-10T10:38:33.296932+0000","last_undegraded":"2024-05-10T10:38:33.296932+0000","last_fullsized":"2024-05-10T10:38:33.296932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:54.724452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571851+0000","last_change":"2024-05-10T10:36:01.987769+0000","last_active":"2024-05-10T10:37:35.571851+0000","last_peered":"2024-05-10T10:37:35.571851+0000","last_clean":"2024-05-10T10:37:35.571851+0000","last_became_active":"2024-05-10T10:36:01.987367+0000","last_became_peered":"2024-05-10T10:36:01.987367+0000","last_unstale":"2024-05-10T10:37:35.571851+0000","last_undegraded":"2024-05-10T10:37:35.571851+0000","last_fullsized":"2024-05-10T10:37:35.571851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:36.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661837+0000","last_change":"2024-05-10T10:35:47.915082+0000","last_active":"2024-05-10T10:40:08.661837+0000","last_peered":"2024-05-10T10:40:08.661837+0000","last_clean":"2024-05-10T10:40:08.661837+0000","last_became_active":"2024-05-10T10:35:47.914722+0000","last_became_peered":"2024-05-10T10:35:47.914722+0000","last_unstale":"2024-05-10T10:40:08.661837+0000","last_undegraded":"2024-05-10T10:40:08.661837+0000","last_fullsized":"2024-05-10T10:40:08.661837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:31.005710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285496+0000","last_change":"2024-05-10T10:35:50.926181+0000","last_active":"2024-05-10T10:38:33.285496+0000","last_peered":"2024-05-10T10:38:33.285496+0000","last_clean":"2024-05-10T10:38:33.285496+0000","last_became_active":"2024-05-10T10:35:50.925865+0000","last_became_peered":"2024-05-10T10:35:50.925865+0000","last_unstale":"2024-05-10T10:38:33.285496+0000","last_undegraded":"2024-05-10T10:38:33.285496+0000","last_fullsized":"2024-05-10T10:38:33.285496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:04.976908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.572140+0000","last_change":"2024-05-10T10:36:35.142412+0000","last_active":"2024-05-10T10:37:35.572140+0000","last_peered":"2024-05-10T10:37:35.572140+0000","last_clean":"2024-05-10T10:37:35.572140+0000","last_became_active":"2024-05-10T10:36:35.141959+0000","last_became_peered":"2024-05-10T10:36:35.141959+0000","last_unstale":"2024-05-10T10:37:35.572140+0000","last_undegraded":"2024-05-10T10:37:35.572140+0000","last_fullsized":"2024-05-10T10:37:35.572140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:20:34.133261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720063+0000","last_change":"2024-05-10T10:35:47.912826+0000","last_active":"2024-05-10T10:37:35.720063+0000","last_peered":"2024-05-10T10:37:35.720063+0000","last_clean":"2024-05-10T10:37:35.720063+0000","last_became_active":"2024-05-10T10:35:47.912468+0000","last_became_peered":"2024-05-10T10:35:47.912468+0000","last_unstale":"2024-05-10T10:37:35.720063+0000","last_undegraded":"2024-05-10T10:37:35.720063+0000","last_fullsized":"2024-05-10T10:37:35.720063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:56.482104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293989+0000","last_change":"2024-05-10T10:36:35.114920+0000","last_active":"2024-05-10T10:38:33.293989+0000","last_peered":"2024-05-10T10:38:33.293989+0000","last_clean":"2024-05-10T10:38:33.293989+0000","last_became_active":"2024-05-10T10:36:35.114546+0000","last_became_peered":"2024-05-10T10:36:35.114546+0000","last_unstale":"2024-05-10T10:38:33.293989+0000","last_undegraded":"2024-05-10T10:38:33.293989+0000","last_fullsized":"2024-05-10T10:38:33.293989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:52.454111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.048033+0000","last_change":"2024-05-10T10:36:01.990086+0000","last_active":"2024-05-10T10:38:06.048033+0000","last_peered":"2024-05-10T10:38:06.048033+0000","last_clean":"2024-05-10T10:38:06.048033+0000","last_became_active":"2024-05-10T10:36:01.989624+0000","last_became_peered":"2024-05-10T10:36:01.989624+0000","last_unstale":"2024-05-10T10:38:06.048033+0000","last_undegraded":"2024-05-10T10:38:06.048033+0000","last_fullsized":"2024-05-10T10:38:06.048033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:33.708069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564634+0000","last_change":"2024-05-10T10:35:47.905182+0000","last_active":"2024-05-10T10:37:35.564634+0000","last_peered":"2024-05-10T10:37:35.564634+0000","last_clean":"2024-05-10T10:37:35.564634+0000","last_became_active":"2024-05-10T10:35:47.904855+0000","last_became_peered":"2024-05-10T10:35:47.904855+0000","last_unstale":"2024-05-10T10:37:35.564634+0000","last_undegraded":"2024-05-10T10:37:35.564634+0000","last_fullsized":"2024-05-10T10:37:35.564634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:17:44.844156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292710+0000","last_change":"2024-05-10T10:35:50.935478+0000","last_active":"2024-05-10T10:38:33.292710+0000","last_peered":"2024-05-10T10:38:33.292710+0000","last_clean":"2024-05-10T10:38:33.292710+0000","last_became_active":"2024-05-10T10:35:50.933487+0000","last_became_peered":"2024-05-10T10:35:50.933487+0000","last_unstale":"2024-05-10T10:38:33.292710+0000","last_undegraded":"2024-05-10T10:38:33.292710+0000","last_fullsized":"2024-05-10T10:38:33.292710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:02.433423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:02.493132+0000","last_change":"2024-05-10T10:36:01.984285+0000","last_active":"2024-05-10T10:38:02.493132+0000","last_peered":"2024-05-10T10:38:02.493132+0000","last_clean":"2024-05-10T10:38:02.493132+0000","last_became_active":"2024-05-10T10:36:01.983900+0000","last_became_peered":"2024-05-10T10:36:01.983900+0000","last_unstale":"2024-05-10T10:38:02.493132+0000","last_undegraded":"2024-05-10T10:38:02.493132+0000","last_fullsized":"2024-05-10T10:38:02.493132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:53.406106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565788+0000","last_change":"2024-05-10T10:35:47.911123+0000","last_active":"2024-05-10T10:37:35.565788+0000","last_peered":"2024-05-10T10:37:35.565788+0000","last_clean":"2024-05-10T10:37:35.565788+0000","last_became_active":"2024-05-10T10:35:47.910762+0000","last_became_peered":"2024-05-10T10:35:47.910762+0000","last_unstale":"2024-05-10T10:37:35.565788+0000","last_undegraded":"2024-05-10T10:37:35.565788+0000","last_fullsized":"2024-05-10T10:37:35.565788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:07.045586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288933+0000","last_change":"2024-05-10T10:35:50.928093+0000","last_active":"2024-05-10T10:38:33.288933+0000","last_peered":"2024-05-10T10:38:33.288933+0000","last_clean":"2024-05-10T10:38:33.288933+0000","last_became_active":"2024-05-10T10:35:50.926945+0000","last_became_peered":"2024-05-10T10:35:50.926945+0000","last_unstale":"2024-05-10T10:38:33.288933+0000","last_undegraded":"2024-05-10T10:38:33.288933+0000","last_fullsized":"2024-05-10T10:38:33.288933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:11.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569138+0000","last_change":"2024-05-10T10:36:35.143043+0000","last_active":"2024-05-10T10:37:35.569138+0000","last_peered":"2024-05-10T10:37:35.569138+0000","last_clean":"2024-05-10T10:37:35.569138+0000","last_became_active":"2024-05-10T10:36:35.142739+0000","last_became_peered":"2024-05-10T10:36:35.142739+0000","last_unstale":"2024-05-10T10:37:35.569138+0000","last_undegraded":"2024-05-10T10:37:35.569138+0000","last_fullsized":"2024-05-10T10:37:35.569138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:10.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":427,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661413+0000","last_change":"2024-05-10T10:35:47.908687+0000","last_active":"2024-05-10T10:40:08.661413+0000","last_peered":"2024-05-10T10:40:08.661413+0000","last_clean":"2024-05-10T10:40:08.661413+0000","last_became_active":"2024-05-10T10:35:47.908349+0000","last_became_peered":"2024-05-10T10:35:47.908349+0000","last_unstale":"2024-05-10T10:40:08.661413+0000","last_undegraded":"2024-05-10T10:40:08.661413+0000","last_fullsized":"2024-05-10T10:40:08.661413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:48.696104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":376,"num_read_kb":376,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.291371+0000","last_change":"2024-05-10T10:35:50.933729+0000","last_active":"2024-05-10T10:38:33.291371+0000","last_peered":"2024-05-10T10:38:33.291371+0000","last_clean":"2024-05-10T10:38:33.291371+0000","last_became_active":"2024-05-10T10:35:50.933109+0000","last_became_peered":"2024-05-10T10:35:50.933109+0000","last_unstale":"2024-05-10T10:38:33.291371+0000","last_undegraded":"2024-05-10T10:38:33.291371+0000","last_fullsized":"2024-05-10T10:38:33.291371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:10:38.305592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567135+0000","last_change":"2024-05-10T10:36:01.991969+0000","last_active":"2024-05-10T10:37:35.567135+0000","last_peered":"2024-05-10T10:37:35.567135+0000","last_clean":"2024-05-10T10:37:35.567135+0000","last_became_active":"2024-05-10T10:36:01.990278+0000","last_became_peered":"2024-05-10T10:36:01.990278+0000","last_unstale":"2024-05-10T10:37:35.567135+0000","last_undegraded":"2024-05-10T10:37:35.567135+0000","last_fullsized":"2024-05-10T10:37:35.567135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:33.070520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719303+0000","last_change":"2024-05-10T10:35:47.911682+0000","last_active":"2024-05-10T10:37:35.719303+0000","last_peered":"2024-05-10T10:37:35.719303+0000","last_clean":"2024-05-10T10:37:35.719303+0000","last_became_active":"2024-05-10T10:35:47.911328+0000","last_became_peered":"2024-05-10T10:35:47.911328+0000","last_unstale":"2024-05-10T10:37:35.719303+0000","last_undegraded":"2024-05-10T10:37:35.719303+0000","last_fullsized":"2024-05-10T10:37:35.719303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.361340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285082+0000","last_change":"2024-05-10T10:35:50.931541+0000","last_active":"2024-05-10T10:38:33.285082+0000","last_peered":"2024-05-10T10:38:33.285082+0000","last_clean":"2024-05-10T10:38:33.285082+0000","last_became_active":"2024-05-10T10:35:50.931244+0000","last_became_peered":"2024-05-10T10:35:50.931244+0000","last_unstale":"2024-05-10T10:38:33.285082+0000","last_undegraded":"2024-05-10T10:38:33.285082+0000","last_fullsized":"2024-05-10T10:38:33.285082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:38.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720689+0000","last_change":"2024-05-10T10:36:01.990390+0000","last_active":"2024-05-10T10:37:35.720689+0000","last_peered":"2024-05-10T10:37:35.720689+0000","last_clean":"2024-05-10T10:37:35.720689+0000","last_became_active":"2024-05-10T10:36:01.989434+0000","last_became_peered":"2024-05-10T10:36:01.989434+0000","last_unstale":"2024-05-10T10:37:35.720689+0000","last_undegraded":"2024-05-10T10:37:35.720689+0000","last_fullsized":"2024-05-10T10:37:35.720689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:53.129212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566392+0000","last_change":"2024-05-10T10:35:47.911590+0000","last_active":"2024-05-10T10:37:35.566392+0000","last_peered":"2024-05-10T10:37:35.566392+0000","last_clean":"2024-05-10T10:37:35.566392+0000","last_became_active":"2024-05-10T10:35:47.910950+0000","last_became_peered":"2024-05-10T10:35:47.910950+0000","last_unstale":"2024-05-10T10:37:35.566392+0000","last_undegraded":"2024-05-10T10:37:35.566392+0000","last_fullsized":"2024-05-10T10:37:35.566392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:18.844774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292200+0000","last_change":"2024-05-10T10:35:50.938174+0000","last_active":"2024-05-10T10:38:33.292200+0000","last_peered":"2024-05-10T10:38:33.292200+0000","last_clean":"2024-05-10T10:38:33.292200+0000","last_became_active":"2024-05-10T10:35:50.937618+0000","last_became_peered":"2024-05-10T10:35:50.937618+0000","last_unstale":"2024-05-10T10:38:33.292200+0000","last_undegraded":"2024-05-10T10:38:33.292200+0000","last_fullsized":"2024-05-10T10:38:33.292200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:26:48.865564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567141+0000","last_change":"2024-05-10T10:36:01.979226+0000","last_active":"2024-05-10T10:37:35.567141+0000","last_peered":"2024-05-10T10:37:35.567141+0000","last_clean":"2024-05-10T10:37:35.567141+0000","last_became_active":"2024-05-10T10:36:01.978847+0000","last_became_peered":"2024-05-10T10:36:01.978847+0000","last_unstale":"2024-05-10T10:37:35.567141+0000","last_undegraded":"2024-05-10T10:37:35.567141+0000","last_fullsized":"2024-05-10T10:37:35.567141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:10.629813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'42","reported_seq":490,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.661398+0000","last_change":"2024-05-10T10:36:35.150814+0000","last_active":"2024-05-10T10:40:08.661398+0000","last_peered":"2024-05-10T10:40:08.661398+0000","last_clean":"2024-05-10T10:40:08.661398+0000","last_became_active":"2024-05-10T10:36:35.142980+0000","last_became_peered":"2024-05-10T10:36:35.142980+0000","last_unstale":"2024-05-10T10:40:08.661398+0000","last_undegraded":"2024-05-10T10:40:08.661398+0000","last_fullsized":"2024-05-10T10:40:08.661398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:17.907275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67776,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":387,"num_read_kb":260,"num_write":44,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3701,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290064+0000","last_change":"2024-05-10T10:35:50.925584+0000","last_active":"2024-05-10T10:38:33.290064+0000","last_peered":"2024-05-10T10:38:33.290064+0000","last_clean":"2024-05-10T10:38:33.290064+0000","last_became_active":"2024-05-10T10:35:50.925215+0000","last_became_peered":"2024-05-10T10:35:50.925215+0000","last_unstale":"2024-05-10T10:38:33.290064+0000","last_undegraded":"2024-05-10T10:38:33.290064+0000","last_fullsized":"2024-05-10T10:38:33.290064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:03.377110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566636+0000","last_change":"2024-05-10T10:36:01.985090+0000","last_active":"2024-05-10T10:37:35.566636+0000","last_peered":"2024-05-10T10:37:35.566636+0000","last_clean":"2024-05-10T10:37:35.566636+0000","last_became_active":"2024-05-10T10:36:01.984059+0000","last_became_peered":"2024-05-10T10:36:01.984059+0000","last_unstale":"2024-05-10T10:37:35.566636+0000","last_undegraded":"2024-05-10T10:37:35.566636+0000","last_fullsized":"2024-05-10T10:37:35.566636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:41.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571763+0000","last_change":"2024-05-10T10:35:47.914377+0000","last_active":"2024-05-10T10:37:35.571763+0000","last_peered":"2024-05-10T10:37:35.571763+0000","last_clean":"2024-05-10T10:37:35.571763+0000","last_became_active":"2024-05-10T10:35:47.913830+0000","last_became_peered":"2024-05-10T10:35:47.913830+0000","last_unstale":"2024-05-10T10:37:35.571763+0000","last_undegraded":"2024-05-10T10:37:35.571763+0000","last_fullsized":"2024-05-10T10:37:35.571763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:54:18.540952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284127+0000","last_change":"2024-05-10T10:35:50.932721+0000","last_active":"2024-05-10T10:38:33.284127+0000","last_peered":"2024-05-10T10:38:33.284127+0000","last_clean":"2024-05-10T10:38:33.284127+0000","last_became_active":"2024-05-10T10:35:50.932410+0000","last_became_peered":"2024-05-10T10:35:50.932410+0000","last_unstale":"2024-05-10T10:38:33.284127+0000","last_undegraded":"2024-05-10T10:38:33.284127+0000","last_fullsized":"2024-05-10T10:38:33.284127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:42.255534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569383+0000","last_change":"2024-05-10T10:36:35.143581+0000","last_active":"2024-05-10T10:37:35.569383+0000","last_peered":"2024-05-10T10:37:35.569383+0000","last_clean":"2024-05-10T10:37:35.569383+0000","last_became_active":"2024-05-10T10:36:35.142957+0000","last_became_peered":"2024-05-10T10:36:35.142957+0000","last_unstale":"2024-05-10T10:37:35.569383+0000","last_undegraded":"2024-05-10T10:37:35.569383+0000","last_fullsized":"2024-05-10T10:37:35.569383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:14:39.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566100+0000","last_change":"2024-05-10T10:35:47.908874+0000","last_active":"2024-05-10T10:37:35.566100+0000","last_peered":"2024-05-10T10:37:35.566100+0000","last_clean":"2024-05-10T10:37:35.566100+0000","last_became_active":"2024-05-10T10:35:47.908478+0000","last_became_peered":"2024-05-10T10:35:47.908478+0000","last_unstale":"2024-05-10T10:37:35.566100+0000","last_undegraded":"2024-05-10T10:37:35.566100+0000","last_fullsized":"2024-05-10T10:37:35.566100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:13.147762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285463+0000","last_change":"2024-05-10T10:35:50.925168+0000","last_active":"2024-05-10T10:38:33.285463+0000","last_peered":"2024-05-10T10:38:33.285463+0000","last_clean":"2024-05-10T10:38:33.285463+0000","last_became_active":"2024-05-10T10:35:50.924509+0000","last_became_peered":"2024-05-10T10:35:50.924509+0000","last_unstale":"2024-05-10T10:38:33.285463+0000","last_undegraded":"2024-05-10T10:38:33.285463+0000","last_fullsized":"2024-05-10T10:38:33.285463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:08.458195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569737+0000","last_change":"2024-05-10T10:36:35.144575+0000","last_active":"2024-05-10T10:37:35.569737+0000","last_peered":"2024-05-10T10:37:35.569737+0000","last_clean":"2024-05-10T10:37:35.569737+0000","last_became_active":"2024-05-10T10:36:35.144165+0000","last_became_peered":"2024-05-10T10:36:35.144165+0000","last_unstale":"2024-05-10T10:37:35.569737+0000","last_undegraded":"2024-05-10T10:37:35.569737+0000","last_fullsized":"2024-05-10T10:37:35.569737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:45.179163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":247,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.680223+0000","last_change":"2024-05-10T10:35:47.911208+0000","last_active":"2024-05-10T10:40:08.680223+0000","last_peered":"2024-05-10T10:40:08.680223+0000","last_clean":"2024-05-10T10:40:08.680223+0000","last_became_active":"2024-05-10T10:35:47.910820+0000","last_became_peered":"2024-05-10T10:35:47.910820+0000","last_unstale":"2024-05-10T10:40:08.680223+0000","last_undegraded":"2024-05-10T10:40:08.680223+0000","last_fullsized":"2024-05-10T10:40:08.680223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:04.804203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289463+0000","last_change":"2024-05-10T10:35:50.928195+0000","last_active":"2024-05-10T10:38:33.289463+0000","last_peered":"2024-05-10T10:38:33.289463+0000","last_clean":"2024-05-10T10:38:33.289463+0000","last_became_active":"2024-05-10T10:35:50.927832+0000","last_became_peered":"2024-05-10T10:35:50.927832+0000","last_unstale":"2024-05-10T10:38:33.289463+0000","last_undegraded":"2024-05-10T10:38:33.289463+0000","last_fullsized":"2024-05-10T10:38:33.289463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:03.071525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565054+0000","last_change":"2024-05-10T10:36:01.982483+0000","last_active":"2024-05-10T10:37:35.565054+0000","last_peered":"2024-05-10T10:37:35.565054+0000","last_clean":"2024-05-10T10:37:35.565054+0000","last_became_active":"2024-05-10T10:36:01.982117+0000","last_became_peered":"2024-05-10T10:36:01.982117+0000","last_unstale":"2024-05-10T10:37:35.565054+0000","last_undegraded":"2024-05-10T10:37:35.565054+0000","last_fullsized":"2024-05-10T10:37:35.565054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:08.145035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569474+0000","last_change":"2024-05-10T10:35:47.912929+0000","last_active":"2024-05-10T10:37:35.569474+0000","last_peered":"2024-05-10T10:37:35.569474+0000","last_clean":"2024-05-10T10:37:35.569474+0000","last_became_active":"2024-05-10T10:35:47.912575+0000","last_became_peered":"2024-05-10T10:35:47.912575+0000","last_unstale":"2024-05-10T10:37:35.569474+0000","last_undegraded":"2024-05-10T10:37:35.569474+0000","last_fullsized":"2024-05-10T10:37:35.569474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:03.235579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296070+0000","last_change":"2024-05-10T10:35:50.935967+0000","last_active":"2024-05-10T10:38:33.296070+0000","last_peered":"2024-05-10T10:38:33.296070+0000","last_clean":"2024-05-10T10:38:33.296070+0000","last_became_active":"2024-05-10T10:35:50.935635+0000","last_became_peered":"2024-05-10T10:35:50.935635+0000","last_unstale":"2024-05-10T10:38:33.296070+0000","last_undegraded":"2024-05-10T10:38:33.296070+0000","last_fullsized":"2024-05-10T10:38:33.296070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:02:53.681522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565799+0000","last_change":"2024-05-10T10:36:01.979488+0000","last_active":"2024-05-10T10:37:35.565799+0000","last_peered":"2024-05-10T10:37:35.565799+0000","last_clean":"2024-05-10T10:37:35.565799+0000","last_became_active":"2024-05-10T10:36:01.978839+0000","last_became_peered":"2024-05-10T10:36:01.978839+0000","last_unstale":"2024-05-10T10:37:35.565799+0000","last_undegraded":"2024-05-10T10:37:35.565799+0000","last_fullsized":"2024-05-10T10:37:35.565799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:20.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:02.476105+0000","last_change":"2024-05-10T10:35:47.913235+0000","last_active":"2024-05-10T10:38:02.476105+0000","last_peered":"2024-05-10T10:38:02.476105+0000","last_clean":"2024-05-10T10:38:02.476105+0000","last_became_active":"2024-05-10T10:35:47.912788+0000","last_became_peered":"2024-05-10T10:35:47.912788+0000","last_unstale":"2024-05-10T10:38:02.476105+0000","last_undegraded":"2024-05-10T10:38:02.476105+0000","last_fullsized":"2024-05-10T10:38:02.476105+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:56:10.275488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290951+0000","last_change":"2024-05-10T10:35:50.930269+0000","last_active":"2024-05-10T10:38:33.290951+0000","last_peered":"2024-05-10T10:38:33.290951+0000","last_clean":"2024-05-10T10:38:33.290951+0000","last_became_active":"2024-05-10T10:35:50.929918+0000","last_became_peered":"2024-05-10T10:35:50.929918+0000","last_unstale":"2024-05-10T10:38:33.290951+0000","last_undegraded":"2024-05-10T10:38:33.290951+0000","last_fullsized":"2024-05-10T10:38:33.290951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:31.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.569677+0000","last_change":"2024-05-10T10:36:02.018506+0000","last_active":"2024-05-10T10:37:35.569677+0000","last_peered":"2024-05-10T10:37:35.569677+0000","last_clean":"2024-05-10T10:37:35.569677+0000","last_became_active":"2024-05-10T10:36:02.018004+0000","last_became_peered":"2024-05-10T10:36:02.018004+0000","last_unstale":"2024-05-10T10:37:35.569677+0000","last_undegraded":"2024-05-10T10:37:35.569677+0000","last_fullsized":"2024-05-10T10:37:35.569677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:37.404970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:52.729668+0000","last_change":"2024-05-10T10:35:47.910654+0000","last_active":"2024-05-10T10:37:52.729668+0000","last_peered":"2024-05-10T10:37:52.729668+0000","last_clean":"2024-05-10T10:37:52.729668+0000","last_became_active":"2024-05-10T10:35:47.910338+0000","last_became_peered":"2024-05-10T10:35:47.910338+0000","last_unstale":"2024-05-10T10:37:52.729668+0000","last_undegraded":"2024-05-10T10:37:52.729668+0000","last_fullsized":"2024-05-10T10:37:52.729668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:07.127574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290279+0000","last_change":"2024-05-10T10:35:50.940950+0000","last_active":"2024-05-10T10:38:33.290279+0000","last_peered":"2024-05-10T10:38:33.290279+0000","last_clean":"2024-05-10T10:38:33.290279+0000","last_became_active":"2024-05-10T10:35:50.940562+0000","last_became_peered":"2024-05-10T10:35:50.940562+0000","last_unstale":"2024-05-10T10:38:33.290279+0000","last_undegraded":"2024-05-10T10:38:33.290279+0000","last_fullsized":"2024-05-10T10:38:33.290279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:24.825900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.564778+0000","last_change":"2024-05-10T10:36:01.989277+0000","last_active":"2024-05-10T10:37:35.564778+0000","last_peered":"2024-05-10T10:37:35.564778+0000","last_clean":"2024-05-10T10:37:35.564778+0000","last_became_active":"2024-05-10T10:36:01.988866+0000","last_became_peered":"2024-05-10T10:36:01.988866+0000","last_unstale":"2024-05-10T10:37:35.564778+0000","last_undegraded":"2024-05-10T10:37:35.564778+0000","last_fullsized":"2024-05-10T10:37:35.564778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:08:29.740044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":244,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:09.662717+0000","last_change":"2024-05-10T10:35:47.908332+0000","last_active":"2024-05-10T10:40:09.662717+0000","last_peered":"2024-05-10T10:40:09.662717+0000","last_clean":"2024-05-10T10:40:09.662717+0000","last_became_active":"2024-05-10T10:35:47.907933+0000","last_became_peered":"2024-05-10T10:35:47.907933+0000","last_unstale":"2024-05-10T10:40:09.662717+0000","last_undegraded":"2024-05-10T10:40:09.662717+0000","last_fullsized":"2024-05-10T10:40:09.662717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:47:48.702338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290317+0000","last_change":"2024-05-10T10:35:50.934638+0000","last_active":"2024-05-10T10:38:33.290317+0000","last_peered":"2024-05-10T10:38:33.290317+0000","last_clean":"2024-05-10T10:38:33.290317+0000","last_became_active":"2024-05-10T10:35:50.934285+0000","last_became_peered":"2024-05-10T10:35:50.934285+0000","last_unstale":"2024-05-10T10:38:33.290317+0000","last_undegraded":"2024-05-10T10:38:33.290317+0000","last_fullsized":"2024-05-10T10:38:33.290317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:24:31.179005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565003+0000","last_change":"2024-05-10T10:36:02.018468+0000","last_active":"2024-05-10T10:37:35.565003+0000","last_peered":"2024-05-10T10:37:35.565003+0000","last_clean":"2024-05-10T10:37:35.565003+0000","last_became_active":"2024-05-10T10:36:02.018087+0000","last_became_peered":"2024-05-10T10:36:02.018087+0000","last_unstale":"2024-05-10T10:37:35.565003+0000","last_undegraded":"2024-05-10T10:37:35.565003+0000","last_fullsized":"2024-05-10T10:37:35.565003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:55.712012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295463+0000","last_change":"2024-05-10T10:35:50.933660+0000","last_active":"2024-05-10T10:38:33.295463+0000","last_peered":"2024-05-10T10:38:33.295463+0000","last_clean":"2024-05-10T10:38:33.295463+0000","last_became_active":"2024-05-10T10:35:50.933230+0000","last_became_peered":"2024-05-10T10:35:50.933230+0000","last_unstale":"2024-05-10T10:38:33.295463+0000","last_undegraded":"2024-05-10T10:38:33.295463+0000","last_fullsized":"2024-05-10T10:38:33.295463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:34:54.272278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.679698+0000","last_change":"2024-05-10T10:35:47.910161+0000","last_active":"2024-05-10T10:40:08.679698+0000","last_peered":"2024-05-10T10:40:08.679698+0000","last_clean":"2024-05-10T10:40:08.679698+0000","last_became_active":"2024-05-10T10:35:47.909875+0000","last_became_peered":"2024-05-10T10:35:47.909875+0000","last_unstale":"2024-05-10T10:40:08.679698+0000","last_undegraded":"2024-05-10T10:40:08.679698+0000","last_fullsized":"2024-05-10T10:40:08.679698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:21.620921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.205272+0000","last_change":"2024-05-10T10:36:01.986755+0000","last_active":"2024-05-10T10:38:06.205272+0000","last_peered":"2024-05-10T10:38:06.205272+0000","last_clean":"2024-05-10T10:38:06.205272+0000","last_became_active":"2024-05-10T10:36:01.986340+0000","last_became_peered":"2024-05-10T10:36:01.986340+0000","last_unstale":"2024-05-10T10:38:06.205272+0000","last_undegraded":"2024-05-10T10:38:06.205272+0000","last_fullsized":"2024-05-10T10:38:06.205272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:38.434323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293573+0000","last_change":"2024-05-10T10:36:35.114218+0000","last_active":"2024-05-10T10:38:33.293573+0000","last_peered":"2024-05-10T10:38:33.293573+0000","last_clean":"2024-05-10T10:38:33.293573+0000","last_became_active":"2024-05-10T10:36:35.113876+0000","last_became_peered":"2024-05-10T10:36:35.113876+0000","last_unstale":"2024-05-10T10:38:33.293573+0000","last_undegraded":"2024-05-10T10:38:33.293573+0000","last_fullsized":"2024-05-10T10:38:33.293573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:21.415678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:55.105058+0000","last_change":"2024-05-10T10:35:47.909161+0000","last_active":"2024-05-10T10:37:55.105058+0000","last_peered":"2024-05-10T10:37:55.105058+0000","last_clean":"2024-05-10T10:37:55.105058+0000","last_became_active":"2024-05-10T10:35:47.908546+0000","last_became_peered":"2024-05-10T10:35:47.908546+0000","last_unstale":"2024-05-10T10:37:55.105058+0000","last_undegraded":"2024-05-10T10:37:55.105058+0000","last_fullsized":"2024-05-10T10:37:55.105058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:52.364936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719232+0000","last_change":"2024-05-10T10:34:20.980948+0000","last_active":"2024-05-10T10:37:35.719232+0000","last_peered":"2024-05-10T10:37:35.719232+0000","last_clean":"2024-05-10T10:37:35.719232+0000","last_became_active":"2024-05-10T10:34:20.975662+0000","last_became_peered":"2024-05-10T10:34:20.975662+0000","last_unstale":"2024-05-10T10:37:35.719232+0000","last_undegraded":"2024-05-10T10:37:35.719232+0000","last_fullsized":"2024-05-10T10:37:35.719232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:30:55.419211+0000","last_clean_scrub_stamp":"2024-05-10T10:30:55.419211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:33.841788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566301+0000","last_change":"2024-05-10T10:36:02.018578+0000","last_active":"2024-05-10T10:37:35.566301+0000","last_peered":"2024-05-10T10:37:35.566301+0000","last_clean":"2024-05-10T10:37:35.566301+0000","last_became_active":"2024-05-10T10:36:02.018126+0000","last_became_peered":"2024-05-10T10:36:02.018126+0000","last_unstale":"2024-05-10T10:37:35.566301+0000","last_undegraded":"2024-05-10T10:37:35.566301+0000","last_fullsized":"2024-05-10T10:37:35.566301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:18:17.112432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285918+0000","last_change":"2024-05-10T10:35:50.927472+0000","last_active":"2024-05-10T10:38:33.285918+0000","last_peered":"2024-05-10T10:38:33.285918+0000","last_clean":"2024-05-10T10:38:33.285918+0000","last_became_active":"2024-05-10T10:35:50.927030+0000","last_became_peered":"2024-05-10T10:35:50.927030+0000","last_unstale":"2024-05-10T10:38:33.285918+0000","last_undegraded":"2024-05-10T10:38:33.285918+0000","last_fullsized":"2024-05-10T10:38:33.285918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:15:03.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:51.203426+0000","last_change":"2024-05-10T10:35:47.912675+0000","last_active":"2024-05-10T10:37:51.203426+0000","last_peered":"2024-05-10T10:37:51.203426+0000","last_clean":"2024-05-10T10:37:51.203426+0000","last_became_active":"2024-05-10T10:35:47.912297+0000","last_became_peered":"2024-05-10T10:35:47.912297+0000","last_unstale":"2024-05-10T10:37:51.203426+0000","last_undegraded":"2024-05-10T10:37:51.203426+0000","last_fullsized":"2024-05-10T10:37:51.203426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:46.178221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:06.047984+0000","last_change":"2024-05-10T10:36:01.981501+0000","last_active":"2024-05-10T10:38:06.047984+0000","last_peered":"2024-05-10T10:38:06.047984+0000","last_clean":"2024-05-10T10:38:06.047984+0000","last_became_active":"2024-05-10T10:36:01.981029+0000","last_became_peered":"2024-05-10T10:36:01.981029+0000","last_unstale":"2024-05-10T10:38:06.047984+0000","last_undegraded":"2024-05-10T10:38:06.047984+0000","last_fullsized":"2024-05-10T10:38:06.047984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:18.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:52.474989+0000","last_change":"2024-05-10T10:35:47.910701+0000","last_active":"2024-05-10T10:37:52.474989+0000","last_peered":"2024-05-10T10:37:52.474989+0000","last_clean":"2024-05-10T10:37:52.474989+0000","last_became_active":"2024-05-10T10:35:47.910286+0000","last_became_peered":"2024-05-10T10:35:47.910286+0000","last_unstale":"2024-05-10T10:37:52.474989+0000","last_undegraded":"2024-05-10T10:37:52.474989+0000","last_fullsized":"2024-05-10T10:37:52.474989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:07.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.286141+0000","last_change":"2024-05-10T10:35:50.926026+0000","last_active":"2024-05-10T10:38:33.286141+0000","last_peered":"2024-05-10T10:38:33.286141+0000","last_clean":"2024-05-10T10:38:33.286141+0000","last_became_active":"2024-05-10T10:35:50.925624+0000","last_became_peered":"2024-05-10T10:35:50.925624+0000","last_unstale":"2024-05-10T10:38:33.286141+0000","last_undegraded":"2024-05-10T10:38:33.286141+0000","last_fullsized":"2024-05-10T10:38:33.286141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:22:02.828681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:03.903648+0000","last_change":"2024-05-10T10:36:02.018403+0000","last_active":"2024-05-10T10:38:03.903648+0000","last_peered":"2024-05-10T10:38:03.903648+0000","last_clean":"2024-05-10T10:38:03.903648+0000","last_became_active":"2024-05-10T10:36:02.017980+0000","last_became_peered":"2024-05-10T10:36:02.017980+0000","last_unstale":"2024-05-10T10:38:03.903648+0000","last_undegraded":"2024-05-10T10:38:03.903648+0000","last_fullsized":"2024-05-10T10:38:03.903648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:07.264730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:05.971094+0000","last_change":"2024-05-10T10:37:35.577657+0000","last_active":"2024-05-10T10:40:05.971094+0000","last_peered":"2024-05-10T10:40:05.971094+0000","last_clean":"2024-05-10T10:40:05.971094+0000","last_became_active":"2024-05-10T10:37:35.570901+0000","last_became_peered":"2024-05-10T10:37:35.570901+0000","last_unstale":"2024-05-10T10:40:05.971094+0000","last_undegraded":"2024-05-10T10:40:05.971094+0000","last_fullsized":"2024-05-10T10:40:05.971094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:30.088160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293071+0000","last_change":"2024-05-10T10:35:50.935817+0000","last_active":"2024-05-10T10:38:33.293071+0000","last_peered":"2024-05-10T10:38:33.293071+0000","last_clean":"2024-05-10T10:38:33.293071+0000","last_became_active":"2024-05-10T10:35:50.935515+0000","last_became_peered":"2024-05-10T10:35:50.935515+0000","last_unstale":"2024-05-10T10:38:33.293071+0000","last_undegraded":"2024-05-10T10:38:33.293071+0000","last_fullsized":"2024-05-10T10:38:33.293071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:49.042193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.571516+0000","last_change":"2024-05-10T10:36:01.986002+0000","last_active":"2024-05-10T10:37:35.571516+0000","last_peered":"2024-05-10T10:37:35.571516+0000","last_clean":"2024-05-10T10:37:35.571516+0000","last_became_active":"2024-05-10T10:36:01.985558+0000","last_became_peered":"2024-05-10T10:36:01.985558+0000","last_unstale":"2024-05-10T10:37:35.571516+0000","last_undegraded":"2024-05-10T10:37:35.571516+0000","last_fullsized":"2024-05-10T10:37:35.571516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:27.732316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:00.105594+0000","last_change":"2024-05-10T10:35:47.909254+0000","last_active":"2024-05-10T10:38:00.105594+0000","last_peered":"2024-05-10T10:38:00.105594+0000","last_clean":"2024-05-10T10:38:00.105594+0000","last_became_active":"2024-05-10T10:35:47.908370+0000","last_became_peered":"2024-05-10T10:35:47.908370+0000","last_unstale":"2024-05-10T10:38:00.105594+0000","last_undegraded":"2024-05-10T10:38:00.105594+0000","last_fullsized":"2024-05-10T10:38:00.105594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:44.643592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287742+0000","last_change":"2024-05-10T10:35:50.933001+0000","last_active":"2024-05-10T10:38:33.287742+0000","last_peered":"2024-05-10T10:38:33.287742+0000","last_clean":"2024-05-10T10:38:33.287742+0000","last_became_active":"2024-05-10T10:35:50.932010+0000","last_became_peered":"2024-05-10T10:35:50.932010+0000","last_unstale":"2024-05-10T10:38:33.287742+0000","last_undegraded":"2024-05-10T10:38:33.287742+0000","last_fullsized":"2024-05-10T10:38:33.287742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:01.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720023+0000","last_change":"2024-05-10T10:36:02.018650+0000","last_active":"2024-05-10T10:37:35.720023+0000","last_peered":"2024-05-10T10:37:35.720023+0000","last_clean":"2024-05-10T10:37:35.720023+0000","last_became_active":"2024-05-10T10:36:02.018258+0000","last_became_peered":"2024-05-10T10:36:02.018258+0000","last_unstale":"2024-05-10T10:37:35.720023+0000","last_undegraded":"2024-05-10T10:37:35.720023+0000","last_fullsized":"2024-05-10T10:37:35.720023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:33.590752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:57.007213+0000","last_change":"2024-05-10T10:35:47.910419+0000","last_active":"2024-05-10T10:37:57.007213+0000","last_peered":"2024-05-10T10:37:57.007213+0000","last_clean":"2024-05-10T10:37:57.007213+0000","last_became_active":"2024-05-10T10:35:47.909930+0000","last_became_peered":"2024-05-10T10:35:47.909930+0000","last_unstale":"2024-05-10T10:37:57.007213+0000","last_undegraded":"2024-05-10T10:37:57.007213+0000","last_fullsized":"2024-05-10T10:37:57.007213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:32.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.250953+0000","last_change":"2024-05-10T10:35:50.923207+0000","last_active":"2024-05-10T10:38:33.250953+0000","last_peered":"2024-05-10T10:38:33.250953+0000","last_clean":"2024-05-10T10:38:33.250953+0000","last_became_active":"2024-05-10T10:35:50.922866+0000","last_became_peered":"2024-05-10T10:35:50.922866+0000","last_unstale":"2024-05-10T10:38:33.250953+0000","last_undegraded":"2024-05-10T10:38:33.250953+0000","last_fullsized":"2024-05-10T10:38:33.250953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:48.096498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.567305+0000","last_change":"2024-05-10T10:36:01.985399+0000","last_active":"2024-05-10T10:37:35.567305+0000","last_peered":"2024-05-10T10:37:35.567305+0000","last_clean":"2024-05-10T10:37:35.567305+0000","last_became_active":"2024-05-10T10:36:01.985016+0000","last_became_peered":"2024-05-10T10:36:01.985016+0000","last_unstale":"2024-05-10T10:37:35.567305+0000","last_undegraded":"2024-05-10T10:37:35.567305+0000","last_fullsized":"2024-05-10T10:37:35.567305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:12.151693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:58.902976+0000","last_change":"2024-05-10T10:35:47.908594+0000","last_active":"2024-05-10T10:37:58.902976+0000","last_peered":"2024-05-10T10:37:58.902976+0000","last_clean":"2024-05-10T10:37:58.902976+0000","last_became_active":"2024-05-10T10:35:47.908278+0000","last_became_peered":"2024-05-10T10:35:47.908278+0000","last_unstale":"2024-05-10T10:37:58.902976+0000","last_undegraded":"2024-05-10T10:37:58.902976+0000","last_fullsized":"2024-05-10T10:37:58.902976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:24.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295025+0000","last_change":"2024-05-10T10:35:50.933089+0000","last_active":"2024-05-10T10:38:33.295025+0000","last_peered":"2024-05-10T10:38:33.295025+0000","last_clean":"2024-05-10T10:38:33.295025+0000","last_became_active":"2024-05-10T10:35:50.932739+0000","last_became_peered":"2024-05-10T10:35:50.932739+0000","last_unstale":"2024-05-10T10:38:33.295025+0000","last_undegraded":"2024-05-10T10:38:33.295025+0000","last_fullsized":"2024-05-10T10:38:33.295025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:15.302832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.720006+0000","last_change":"2024-05-10T10:36:01.988561+0000","last_active":"2024-05-10T10:37:35.720006+0000","last_peered":"2024-05-10T10:37:35.720006+0000","last_clean":"2024-05-10T10:37:35.720006+0000","last_became_active":"2024-05-10T10:36:01.988120+0000","last_became_peered":"2024-05-10T10:36:01.988120+0000","last_unstale":"2024-05-10T10:37:35.720006+0000","last_undegraded":"2024-05-10T10:37:35.720006+0000","last_fullsized":"2024-05-10T10:37:35.720006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:35:37.120469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:39:46.068283+0000","last_change":"2024-05-10T10:37:35.580875+0000","last_active":"2024-05-10T10:39:46.068283+0000","last_peered":"2024-05-10T10:39:46.068283+0000","last_clean":"2024-05-10T10:39:46.068283+0000","last_became_active":"2024-05-10T10:37:35.572187+0000","last_became_peered":"2024-05-10T10:37:35.572187+0000","last_unstale":"2024-05-10T10:39:46.068283+0000","last_undegraded":"2024-05-10T10:39:46.068283+0000","last_fullsized":"2024-05-10T10:39:46.068283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:06.330404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.290565+0000","last_change":"2024-05-10T10:35:50.938466+0000","last_active":"2024-05-10T10:38:33.290565+0000","last_peered":"2024-05-10T10:38:33.290565+0000","last_clean":"2024-05-10T10:38:33.290565+0000","last_became_active":"2024-05-10T10:35:50.938167+0000","last_became_peered":"2024-05-10T10:35:50.938167+0000","last_unstale":"2024-05-10T10:38:33.290565+0000","last_undegraded":"2024-05-10T10:38:33.290565+0000","last_fullsized":"2024-05-10T10:38:33.290565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:23:18.014708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.719597+0000","last_change":"2024-05-10T10:36:02.018661+0000","last_active":"2024-05-10T10:37:35.719597+0000","last_peered":"2024-05-10T10:37:35.719597+0000","last_clean":"2024-05-10T10:37:35.719597+0000","last_became_active":"2024-05-10T10:36:02.018196+0000","last_became_peered":"2024-05-10T10:36:02.018196+0000","last_unstale":"2024-05-10T10:37:35.719597+0000","last_undegraded":"2024-05-10T10:37:35.719597+0000","last_fullsized":"2024-05-10T10:37:35.719597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:11.605012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.570017+0000","last_change":"2024-05-10T10:35:47.917009+0000","last_active":"2024-05-10T10:37:35.570017+0000","last_peered":"2024-05-10T10:37:35.570017+0000","last_clean":"2024-05-10T10:37:35.570017+0000","last_became_active":"2024-05-10T10:35:47.916536+0000","last_became_peered":"2024-05-10T10:35:47.916536+0000","last_unstale":"2024-05-10T10:37:35.570017+0000","last_undegraded":"2024-05-10T10:37:35.570017+0000","last_fullsized":"2024-05-10T10:37:35.570017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:17.783027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287116+0000","last_change":"2024-05-10T10:35:50.926173+0000","last_active":"2024-05-10T10:38:33.287116+0000","last_peered":"2024-05-10T10:38:33.287116+0000","last_clean":"2024-05-10T10:38:33.287116+0000","last_became_active":"2024-05-10T10:35:50.925846+0000","last_became_peered":"2024-05-10T10:35:50.925846+0000","last_unstale":"2024-05-10T10:38:33.287116+0000","last_undegraded":"2024-05-10T10:38:33.287116+0000","last_fullsized":"2024-05-10T10:38:33.287116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:29:02.587667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565679+0000","last_change":"2024-05-10T10:36:01.981312+0000","last_active":"2024-05-10T10:37:35.565679+0000","last_peered":"2024-05-10T10:37:35.565679+0000","last_clean":"2024-05-10T10:37:35.565679+0000","last_became_active":"2024-05-10T10:36:01.980861+0000","last_became_peered":"2024-05-10T10:36:01.980861+0000","last_unstale":"2024-05-10T10:37:35.565679+0000","last_undegraded":"2024-05-10T10:37:35.565679+0000","last_fullsized":"2024-05-10T10:37:35.565679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:29.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:40:08.680524+0000","last_change":"2024-05-10T10:35:47.911120+0000","last_active":"2024-05-10T10:40:08.680524+0000","last_peered":"2024-05-10T10:40:08.680524+0000","last_clean":"2024-05-10T10:40:08.680524+0000","last_became_active":"2024-05-10T10:35:47.909945+0000","last_became_peered":"2024-05-10T10:35:47.909945+0000","last_unstale":"2024-05-10T10:40:08.680524+0000","last_undegraded":"2024-05-10T10:40:08.680524+0000","last_fullsized":"2024-05-10T10:40:08.680524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:39:40.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287370+0000","last_change":"2024-05-10T10:35:50.933855+0000","last_active":"2024-05-10T10:38:33.287370+0000","last_peered":"2024-05-10T10:38:33.287370+0000","last_clean":"2024-05-10T10:38:33.287370+0000","last_became_active":"2024-05-10T10:35:50.933502+0000","last_became_peered":"2024-05-10T10:35:50.933502+0000","last_unstale":"2024-05-10T10:38:33.287370+0000","last_undegraded":"2024-05-10T10:38:33.287370+0000","last_fullsized":"2024-05-10T10:38:33.287370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:36.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.718378+0000","last_change":"2024-05-10T10:36:01.985133+0000","last_active":"2024-05-10T10:37:35.718378+0000","last_peered":"2024-05-10T10:37:35.718378+0000","last_clean":"2024-05-10T10:37:35.718378+0000","last_became_active":"2024-05-10T10:36:01.984758+0000","last_became_peered":"2024-05-10T10:36:01.984758+0000","last_unstale":"2024-05-10T10:37:35.718378+0000","last_undegraded":"2024-05-10T10:37:35.718378+0000","last_fullsized":"2024-05-10T10:37:35.718378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:52.733063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.566564+0000","last_change":"2024-05-10T10:35:47.908645+0000","last_active":"2024-05-10T10:37:35.566564+0000","last_peered":"2024-05-10T10:37:35.566564+0000","last_clean":"2024-05-10T10:37:35.566564+0000","last_became_active":"2024-05-10T10:35:47.908123+0000","last_became_peered":"2024-05-10T10:35:47.908123+0000","last_unstale":"2024-05-10T10:37:35.566564+0000","last_undegraded":"2024-05-10T10:37:35.566564+0000","last_fullsized":"2024-05-10T10:37:35.566564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:46.181724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.289675+0000","last_change":"2024-05-10T10:35:50.929215+0000","last_active":"2024-05-10T10:38:33.289675+0000","last_peered":"2024-05-10T10:38:33.289675+0000","last_clean":"2024-05-10T10:38:33.289675+0000","last_became_active":"2024-05-10T10:35:50.928907+0000","last_became_peered":"2024-05-10T10:35:50.928907+0000","last_unstale":"2024-05-10T10:38:33.289675+0000","last_undegraded":"2024-05-10T10:38:33.289675+0000","last_fullsized":"2024-05-10T10:38:33.289675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:57:17.340687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565667+0000","last_change":"2024-05-10T10:36:01.988548+0000","last_active":"2024-05-10T10:37:35.565667+0000","last_peered":"2024-05-10T10:37:35.565667+0000","last_clean":"2024-05-10T10:37:35.565667+0000","last_became_active":"2024-05-10T10:36:01.987221+0000","last_became_peered":"2024-05-10T10:36:01.987221+0000","last_unstale":"2024-05-10T10:37:35.565667+0000","last_undegraded":"2024-05-10T10:37:35.565667+0000","last_fullsized":"2024-05-10T10:37:35.565667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:36:00.951028+0000","last_clean_scrub_stamp":"2024-05-10T10:36:00.951028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:55:22.434536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:37:35.565063+0000","last_change":"2024-05-10T10:35:47.914022+0000","last_active":"2024-05-10T10:37:35.565063+0000","last_peered":"2024-05-10T10:37:35.565063+0000","last_clean":"2024-05-10T10:37:35.565063+0000","last_became_active":"2024-05-10T10:35:47.913711+0000","last_became_peered":"2024-05-10T10:35:47.913711+0000","last_unstale":"2024-05-10T10:37:35.565063+0000","last_undegraded":"2024-05-10T10:37:35.565063+0000","last_fullsized":"2024-05-10T10:37:35.565063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:46.883039+0000","last_clean_scrub_stamp":"2024-05-10T10:35:46.883039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:41:13.291307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.284901+0000","last_change":"2024-05-10T10:35:50.927452+0000","last_active":"2024-05-10T10:38:33.284901+0000","last_peered":"2024-05-10T10:38:33.284901+0000","last_clean":"2024-05-10T10:38:33.284901+0000","last_became_active":"2024-05-10T10:35:50.927102+0000","last_became_peered":"2024-05-10T10:35:50.927102+0000","last_unstale":"2024-05-10T10:38:33.284901+0000","last_undegraded":"2024-05-10T10:38:33.284901+0000","last_fullsized":"2024-05-10T10:38:33.284901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:07.704196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.293454+0000","last_change":"2024-05-10T10:35:50.933072+0000","last_active":"2024-05-10T10:38:33.293454+0000","last_peered":"2024-05-10T10:38:33.293454+0000","last_clean":"2024-05-10T10:38:33.293454+0000","last_became_active":"2024-05-10T10:35:50.932724+0000","last_became_peered":"2024-05-10T10:35:50.932724+0000","last_unstale":"2024-05-10T10:38:33.293454+0000","last_undegraded":"2024-05-10T10:38:33.293454+0000","last_fullsized":"2024-05-10T10:38:33.293454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:08.110381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.287960+0000","last_change":"2024-05-10T10:35:50.931635+0000","last_active":"2024-05-10T10:38:33.287960+0000","last_peered":"2024-05-10T10:38:33.287960+0000","last_clean":"2024-05-10T10:38:33.287960+0000","last_became_active":"2024-05-10T10:35:50.930158+0000","last_became_peered":"2024-05-10T10:35:50.930158+0000","last_unstale":"2024-05-10T10:38:33.287960+0000","last_undegraded":"2024-05-10T10:38:33.287960+0000","last_fullsized":"2024-05-10T10:38:33.287960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:22.963288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285388+0000","last_change":"2024-05-10T10:35:50.936947+0000","last_active":"2024-05-10T10:38:33.285388+0000","last_peered":"2024-05-10T10:38:33.285388+0000","last_clean":"2024-05-10T10:38:33.285388+0000","last_became_active":"2024-05-10T10:35:50.936156+0000","last_became_peered":"2024-05-10T10:35:50.936156+0000","last_unstale":"2024-05-10T10:38:33.285388+0000","last_undegraded":"2024-05-10T10:38:33.285388+0000","last_fullsized":"2024-05-10T10:38:33.285388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:21.167787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294726+0000","last_change":"2024-05-10T10:35:50.935384+0000","last_active":"2024-05-10T10:38:33.294726+0000","last_peered":"2024-05-10T10:38:33.294726+0000","last_clean":"2024-05-10T10:38:33.294726+0000","last_became_active":"2024-05-10T10:35:50.932555+0000","last_became_peered":"2024-05-10T10:35:50.932555+0000","last_unstale":"2024-05-10T10:38:33.294726+0000","last_undegraded":"2024-05-10T10:38:33.294726+0000","last_fullsized":"2024-05-10T10:38:33.294726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:09:11.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.295227+0000","last_change":"2024-05-10T10:35:50.929322+0000","last_active":"2024-05-10T10:38:33.295227+0000","last_peered":"2024-05-10T10:38:33.295227+0000","last_clean":"2024-05-10T10:38:33.295227+0000","last_became_active":"2024-05-10T10:35:50.928366+0000","last_became_peered":"2024-05-10T10:35:50.928366+0000","last_unstale":"2024-05-10T10:38:33.295227+0000","last_undegraded":"2024-05-10T10:38:33.295227+0000","last_fullsized":"2024-05-10T10:38:33.295227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:32:28.337962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292904+0000","last_change":"2024-05-10T10:36:35.110922+0000","last_active":"2024-05-10T10:38:33.292904+0000","last_peered":"2024-05-10T10:38:33.292904+0000","last_clean":"2024-05-10T10:38:33.292904+0000","last_became_active":"2024-05-10T10:36:35.110525+0000","last_became_peered":"2024-05-10T10:36:35.110525+0000","last_unstale":"2024-05-10T10:38:33.292904+0000","last_undegraded":"2024-05-10T10:38:33.292904+0000","last_fullsized":"2024-05-10T10:38:33.292904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:47.825168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.288252+0000","last_change":"2024-05-10T10:35:50.927869+0000","last_active":"2024-05-10T10:38:33.288252+0000","last_peered":"2024-05-10T10:38:33.288252+0000","last_clean":"2024-05-10T10:38:33.288252+0000","last_became_active":"2024-05-10T10:35:50.927552+0000","last_became_peered":"2024-05-10T10:35:50.927552+0000","last_unstale":"2024-05-10T10:38:33.288252+0000","last_undegraded":"2024-05-10T10:38:33.288252+0000","last_fullsized":"2024-05-10T10:38:33.288252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:36.962316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285786+0000","last_change":"2024-05-10T10:35:50.933099+0000","last_active":"2024-05-10T10:38:33.285786+0000","last_peered":"2024-05-10T10:38:33.285786+0000","last_clean":"2024-05-10T10:38:33.285786+0000","last_became_active":"2024-05-10T10:35:50.932765+0000","last_became_peered":"2024-05-10T10:35:50.932765+0000","last_unstale":"2024-05-10T10:38:33.285786+0000","last_undegraded":"2024-05-10T10:38:33.285786+0000","last_fullsized":"2024-05-10T10:38:33.285786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:35:19.095217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.294758+0000","last_change":"2024-05-10T10:35:50.937704+0000","last_active":"2024-05-10T10:38:33.294758+0000","last_peered":"2024-05-10T10:38:33.294758+0000","last_clean":"2024-05-10T10:38:33.294758+0000","last_became_active":"2024-05-10T10:35:50.937236+0000","last_became_peered":"2024-05-10T10:35:50.937236+0000","last_unstale":"2024-05-10T10:38:33.294758+0000","last_undegraded":"2024-05-10T10:38:33.294758+0000","last_fullsized":"2024-05-10T10:38:33.294758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:46.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292751+0000","last_change":"2024-05-10T10:35:50.935181+0000","last_active":"2024-05-10T10:38:33.292751+0000","last_peered":"2024-05-10T10:38:33.292751+0000","last_clean":"2024-05-10T10:38:33.292751+0000","last_became_active":"2024-05-10T10:35:50.933446+0000","last_became_peered":"2024-05-10T10:35:50.933446+0000","last_unstale":"2024-05-10T10:38:33.292751+0000","last_undegraded":"2024-05-10T10:38:33.292751+0000","last_fullsized":"2024-05-10T10:38:33.292751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:03.972410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.285940+0000","last_change":"2024-05-10T10:35:50.931797+0000","last_active":"2024-05-10T10:38:33.285940+0000","last_peered":"2024-05-10T10:38:33.285940+0000","last_clean":"2024-05-10T10:38:33.285940+0000","last_became_active":"2024-05-10T10:35:50.931494+0000","last_became_peered":"2024-05-10T10:35:50.931494+0000","last_unstale":"2024-05-10T10:38:33.285940+0000","last_undegraded":"2024-05-10T10:38:33.285940+0000","last_fullsized":"2024-05-10T10:38:33.285940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:30.797462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.296149+0000","last_change":"2024-05-10T10:35:50.931688+0000","last_active":"2024-05-10T10:38:33.296149+0000","last_peered":"2024-05-10T10:38:33.296149+0000","last_clean":"2024-05-10T10:38:33.296149+0000","last_became_active":"2024-05-10T10:35:50.931362+0000","last_became_peered":"2024-05-10T10:35:50.931362+0000","last_unstale":"2024-05-10T10:38:33.296149+0000","last_undegraded":"2024-05-10T10:38:33.296149+0000","last_fullsized":"2024-05-10T10:38:33.296149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:24.263752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T10:38:33.292719+0000","last_change":"2024-05-10T10:35:50.930905+0000","last_active":"2024-05-10T10:38:33.292719+0000","last_peered":"2024-05-10T10:38:33.292719+0000","last_clean":"2024-05-10T10:38:33.292719+0000","last_became_active":"2024-05-10T10:35:50.930448+0000","last_became_peered":"2024-05-10T10:35:50.930448+0000","last_unstale":"2024-05-10T10:38:33.292719+0000","last_undegraded":"2024-05-10T10:38:33.292719+0000","last_fullsized":"2024-05-10T10:38:33.292719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:49.902139+0000","last_clean_scrub_stamp":"2024-05-10T10:35:49.902139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:52:24.468600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76702,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1932,"num_read_kb":1048,"num_write":271,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3701,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":234420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678085,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":521483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874313,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1176607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037838,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":525016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234068,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430298,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1119415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626527,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":516917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":585723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":518373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":518660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":516132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":593613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1113479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:40:12.399 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:40:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:12.794 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:12 smithi112 ceph-mon[30100]: pgmap v429: 129 pgs: 129 active+clean; 652 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-10T10:40:12.795 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:12 smithi112 ceph-mon[30100]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:12.795 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:12 smithi112 ceph-mon[30100]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:13.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:12 smithi148 ceph-mon[33568]: pgmap v429: 129 pgs: 129 active+clean; 652 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-10T10:40:13.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:12 smithi148 ceph-mon[33568]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:13.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:12 smithi148 ceph-mon[33568]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:12 smithi195 ceph-mon[33773]: pgmap v429: 129 pgs: 129 active+clean; 652 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-10T10:40:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:12 smithi195 ceph-mon[33773]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:12 smithi195 ceph-mon[33773]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:13.967 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:13 smithi195 ceph-mon[33773]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:13.970 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:13 smithi112 ceph-mon[30100]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:14.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:13 smithi148 ceph-mon[33568]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T10:40:14.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.524+0000 7f1f76686700 1 -- 172.21.15.112:0/2588110964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70104100 msgr2=0x7f1f70108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:14.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.524+0000 7f1f76686700 1 --2- 172.21.15.112:0/2588110964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70104100 0x7f1f70108120 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f640099c0 tx=0x7f1f64009cd0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.524+0000 7f1f76686700 1 -- 172.21.15.112:0/2588110964 shutdown_connections 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.524+0000 7f1f76686700 1 --2- 172.21.15.112:0/2588110964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f70108850 0x7f1f701103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.525+0000 7f1f76686700 1 --2- 172.21.15.112:0/2588110964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f70103750 0x7f1f70103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.525+0000 7f1f76686700 1 --2- 172.21.15.112:0/2588110964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70104100 0x7f1f70108120 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.525+0000 7f1f76686700 1 -- 172.21.15.112:0/2588110964 >> 172.21.15.112:0/2588110964 conn(0x7f1f7007a600 msgr2=0x7f1f7007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.525+0000 7f1f76686700 1 -- 172.21.15.112:0/2588110964 shutdown_connections 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.525+0000 7f1f76686700 1 -- 172.21.15.112:0/2588110964 wait complete. 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.525+0000 7f1f76686700 1 Processor -- start 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.525+0000 7f1f76686700 1 -- start start 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f76686700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70103750 0x7f1f7019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f76686700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f70104100 0x7f1f7019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f76686700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f70108850 0x7f1f70199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f76686700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f701134c0 con 0x7f1f70103750 2024-05-10T10:40:14.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f76686700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70113340 con 0x7f1f70108850 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f76686700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70113640 con 0x7f1f70104100 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70103750 0x7f1f7019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70103750 0x7f1f7019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46508/0 (socket says 172.21.15.112:46508) 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 -- 172.21.15.112:0/3260007949 learned_addr learned my addr 172.21.15.112:0/3260007949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6f7fe700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f70104100 0x7f1f7019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 -- 172.21.15.112:0/3260007949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f70104100 msgr2=0x7f1f7019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f74c23700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f70108850 0x7f1f70199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f70104100 0x7f1f7019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 -- 172.21.15.112:0/3260007949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f70108850 msgr2=0x7f1f70199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f70108850 0x7f1f70199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 -- 172.21.15.112:0/3260007949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70199fc0 con 0x7f1f70103750 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6ffff700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70103750 0x7f1f7019f6a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c920 tx=0x7f1f6000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6d7fa700 1 -- 172.21.15.112:0/3260007949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60007b00 con 0x7f1f70103750 2024-05-10T10:40:14.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f6d7fa700 1 -- 172.21.15.112:0/3260007949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f60004cc0 con 0x7f1f70103750 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.526+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7019a2b0 con 0x7f1f70103750 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.527+0000 7f1f6d7fa700 1 -- 172.21.15.112:0/3260007949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600054f0 con 0x7f1f70103750 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.527+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f701a6570 con 0x7f1f70103750 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.527+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7006a190 con 0x7f1f70103750 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.529+0000 7f1f6d7fa700 1 -- 172.21.15.112:0/3260007949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1f60007ca0 con 0x7f1f70103750 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.530+0000 7f1f6d7fa700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1f580b2880 0x7f1f580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.530+0000 7f1f6f7fe700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1f580b2880 0x7f1f580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.530+0000 7f1f6d7fa700 1 -- 172.21.15.112:0/3260007949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1f600e2650 con 0x7f1f70103750 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.530+0000 7f1f6f7fe700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1f580b2880 0x7f1f580b4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f640058e0 tx=0x7f1f6401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:14.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.533+0000 7f1f6d7fa700 1 -- 172.21.15.112:0/3260007949 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1f60014030 con 0x7f1f70103750 2024-05-10T10:40:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.788+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f7002e510 con 0x7f1f70103750 2024-05-10T10:40:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.788+0000 7f1f6d7fa700 1 -- 172.21.15.112:0/3260007949 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f1f600a9f80 con 0x7f1f70103750 2024-05-10T10:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:14.790 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:40:06.380474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:40:14.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1f580b2880 msgr2=0x7f1f580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:14.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1f580b2880 0x7f1f580b4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f640058e0 tx=0x7f1f6401a040 comp rx=0 tx=0).stop 2024-05-10T10:40:14.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70103750 msgr2=0x7f1f7019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:14.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70103750 0x7f1f7019f6a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c920 tx=0x7f1f6000cce0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 shutdown_connections 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f70108850 0x7f1f70199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f70103750 0x7f1f7019f6a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1f580b2880 0x7f1f580b4d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 --2- 172.21.15.112:0/3260007949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f70104100 0x7f1f7019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 >> 172.21.15.112:0/3260007949 conn(0x7f1f7007a600 msgr2=0x7f1f70101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 shutdown_connections 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:14.791+0000 7f1f76686700 1 -- 172.21.15.112:0/3260007949 wait complete. 2024-05-10T10:40:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-05-10T10:40:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:14 smithi148 ceph-mon[33568]: pgmap v430: 129 pgs: 129 active+clean; 652 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T10:40:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:14 smithi195 ceph-mon[33773]: pgmap v430: 129 pgs: 129 active+clean; 652 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T10:40:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:14 smithi112 ceph-mon[30100]: pgmap v430: 129 pgs: 129 active+clean; 652 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T10:40:15.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:40:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:15.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:15 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3260007949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:40:16.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:15 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3260007949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:40:16.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:15 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3260007949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:40:17.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:16 smithi148 ceph-mon[33568]: pgmap v431: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-10T10:40:17.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:16 smithi112 ceph-mon[30100]: pgmap v431: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-10T10:40:17.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:16 smithi195 ceph-mon[33773]: pgmap v431: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-10T10:40:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 -- 172.21.15.112:0/934829474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb41037b0 msgr2=0x7f3fb4107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/934829474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb41037b0 0x7f3fb4107030 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80099c0 tx=0x7f3fa8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 -- 172.21.15.112:0/934829474 shutdown_connections 2024-05-10T10:40:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/934829474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4107760 0x7f3fb410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/934829474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb4102e00 0x7f3fb41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/934829474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb41037b0 0x7f3fb4107030 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 -- 172.21.15.112:0/934829474 >> 172.21.15.112:0/934829474 conn(0x7f3fb4078a10 msgr2=0x7f3fb40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 -- 172.21.15.112:0/934829474 shutdown_connections 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.628+0000 7f3fbbe26700 1 -- 172.21.15.112:0/934829474 wait complete. 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 Processor -- start 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 -- start start 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4102e00 0x7f3fb419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb41037b0 0x7f3fb419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb4107760 0x7f3fb4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4112450 con 0x7f3fb4107760 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41122d0 con 0x7f3fb4102e00 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fbbe26700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41125d0 con 0x7f3fb41037b0 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fba3c3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb4107760 0x7f3fb4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fba3c3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb4107760 0x7f3fb4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34138/0 (socket says 172.21.15.112:34138) 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fba3c3700 1 -- 172.21.15.112:0/260182316 learned_addr learned my addr 172.21.15.112:0/260182316 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fb93c1700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb41037b0 0x7f3fb419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.629+0000 7f3fb9bc2700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4102e00 0x7f3fb419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fba3c3700 1 -- 172.21.15.112:0/260182316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb41037b0 msgr2=0x7f3fb419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fba3c3700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb41037b0 0x7f3fb419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fba3c3700 1 -- 172.21.15.112:0/260182316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4102e00 msgr2=0x7f3fb419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fba3c3700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4102e00 0x7f3fb419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fba3c3700 1 -- 172.21.15.112:0/260182316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb4199010 con 0x7f3fb4107760 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fb9bc2700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4102e00 0x7f3fb419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fba3c3700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb4107760 0x7f3fb4198730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4007d70 tx=0x7f3fa400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fb2ffd700 1 -- 172.21.15.112:0/260182316 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4016020 con 0x7f3fb4107760 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb4199300 con 0x7f3fb4107760 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb41a55e0 con 0x7f3fb4107760 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fb2ffd700 1 -- 172.21.15.112:0/260182316 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fa400ad40 con 0x7f3fb4107760 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.630+0000 7f3fb2ffd700 1 -- 172.21.15.112:0/260182316 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40115e0 con 0x7f3fb4107760 2024-05-10T10:40:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.631+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb406a190 con 0x7f3fb4107760 2024-05-10T10:40:17.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.633+0000 7f3fb2ffd700 1 -- 172.21.15.112:0/260182316 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3fa4019020 con 0x7f3fb4107760 2024-05-10T10:40:17.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.634+0000 7f3fb2ffd700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f9c0b4b10 0x7f3f9c0b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:17.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.634+0000 7f3fb9bc2700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f9c0b4b10 0x7f3f9c0b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:17.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.634+0000 7f3fb2ffd700 1 -- 172.21.15.112:0/260182316 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3fa40e2990 con 0x7f3fb4107760 2024-05-10T10:40:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.635+0000 7f3fb9bc2700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f9c0b4b10 0x7f3f9c0b6fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0099d0 tx=0x7f3fac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:17.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.638+0000 7f3fb2ffd700 1 -- 172.21.15.112:0/260182316 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3fa4020020 con 0x7f3fb4107760 2024-05-10T10:40:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.843+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fb410a380 con 0x7f3fb4107760 2024-05-10T10:40:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.844+0000 7f3fb2ffd700 1 -- 172.21.15.112:0/260182316 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f3fa4055e20 con 0x7f3fb4107760 2024-05-10T10:40:17.846 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:17.846 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.846+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f9c0b4b10 msgr2=0x7f3f9c0b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f9c0b4b10 0x7f3f9c0b6fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0099d0 tx=0x7f3fac008040 comp rx=0 tx=0).stop 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb4107760 msgr2=0x7f3fb4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb4107760 0x7f3fb4198730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4007d70 tx=0x7f3fa400a340 comp rx=0 tx=0).stop 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 shutdown_connections 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb4107760 0x7f3fb4198730 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fb4102e00 0x7f3fb419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3f9c0b4b10 0x7f3f9c0b6fd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 --2- 172.21.15.112:0/260182316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fb41037b0 0x7f3fb419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 >> 172.21.15.112:0/260182316 conn(0x7f3fb4078a10 msgr2=0x7f3fb40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 shutdown_connections 2024-05-10T10:40:17.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:17.847+0000 7f3fbbe26700 1 -- 172.21.15.112:0/260182316 wait complete. 2024-05-10T10:40:18.703 DEBUG:tasks.fwd_scrub:Yielding 2024-05-10T10:40:18.703 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-10T10:40:18.714 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:40:18.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-10T10:40:18.715 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-10T10:40:18.715 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:40:18.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:40:18.718 INFO:teuthology.run_tasks:Running task workunit... 2024-05-10T10:40:18.736 INFO:tasks.workunit:Pulling workunits from ref d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:40:18.737 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-10T10:40:18.737 INFO:tasks.workunit:timeout=3h 2024-05-10T10:40:18.737 INFO:tasks.workunit:cleanup=True 2024-05-10T10:40:18.738 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-10 10:37:09.495327014 +0000 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-10 10:37:09.495327014 +0000 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-10 10:40:06.004163670 +0000 2024-05-10T10:40:18.757 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2024-05-10T10:40:18.758 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:18.758 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-10T10:40:18.830 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:40:18.886 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-10T10:40:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:18 smithi148 ceph-mon[33568]: pgmap v432: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-10T10:40:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:18 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/260182316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:40:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:18 smithi195 ceph-mon[33773]: pgmap v432: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-10T10:40:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:18 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/260182316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:40:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:18 smithi112 ceph-mon[30100]: pgmap v432: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-10T10:40:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:18 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/260182316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 -- 172.21.15.112:0/2447808688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318109540 msgr2=0x7fa318109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 --2- 172.21.15.112:0/2447808688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318109540 0x7fa318109920 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa30800b0d0 tx=0x7fa30800b3e0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 -- 172.21.15.112:0/2447808688 shutdown_connections 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 --2- 172.21.15.112:0/2447808688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa318103f20 0x7fa318110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 --2- 172.21.15.112:0/2447808688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318109540 0x7fa318109920 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 --2- 172.21.15.112:0/2447808688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa318103560 0x7fa3181039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 -- 172.21.15.112:0/2447808688 >> 172.21.15.112:0/2447808688 conn(0x7fa31807a660 msgr2=0x7fa31807aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 -- 172.21.15.112:0/2447808688 shutdown_connections 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 -- 172.21.15.112:0/2447808688 wait complete. 2024-05-10T10:40:20.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.803+0000 7fa31daf0700 1 Processor -- start 2024-05-10T10:40:20.806 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:20 smithi112 ceph-mon[30100]: pgmap v433: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa31daf0700 1 -- start start 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa31daf0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318103560 0x7fa3180757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa31daf0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa318103f20 0x7fa318075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa31daf0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa318109540 0x7fa318076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa31daf0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa318113550 con 0x7fa318103560 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa31daf0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181133d0 con 0x7fa318103f20 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa31daf0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181136d0 con 0x7fa318109540 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318103560 0x7fa3180757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318103560 0x7fa3180757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34152/0 (socket says 172.21.15.112:34152) 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 -- 172.21.15.112:0/3891215336 learned_addr learned my addr 172.21.15.112:0/3891215336 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa316ffd700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa318103f20 0x7fa318075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa317fff700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa318109540 0x7fa318076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 -- 172.21.15.112:0/3891215336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa318109540 msgr2=0x7fa318076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa318109540 0x7fa318076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 -- 172.21.15.112:0/3891215336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa318103f20 msgr2=0x7fa318075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa318103f20 0x7fa318075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 -- 172.21.15.112:0/3891215336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3181a7fb0 con 0x7fa318103560 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa316ffd700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa318103f20 0x7fa318075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa317fff700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa318109540 0x7fa318076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.804+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318103560 0x7fa3180757a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa308007d90 tx=0x7fa308007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.805+0000 7fa314ff9700 1 -- 172.21.15.112:0/3891215336 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30801f030 con 0x7fa318103560 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.805+0000 7fa314ff9700 1 -- 172.21.15.112:0/3891215336 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa30801bad0 con 0x7fa318103560 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.805+0000 7fa314ff9700 1 -- 172.21.15.112:0/3891215336 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308012b50 con 0x7fa318103560 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.805+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3181a8240 con 0x7fa318103560 2024-05-10T10:40:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.805+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3181a86f0 con 0x7fa318103560 2024-05-10T10:40:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.807+0000 7fa314ff9700 1 -- 172.21.15.112:0/3891215336 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa308022030 con 0x7fa318103560 2024-05-10T10:40:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.808+0000 7fa314ff9700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3000b2880 0x7fa3000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.808+0000 7fa316ffd700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3000b2880 0x7fa3000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.808+0000 7fa314ff9700 1 -- 172.21.15.112:0/3891215336 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa30800f020 con 0x7fa318103560 2024-05-10T10:40:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa316ffd700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3000b2880 0x7fa3000b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005950 tx=0x7fa30c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 Processor -- start 2024-05-10T10:40:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 -- start start 2024-05-10T10:40:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf2f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf29890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf2a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d5bde7b9c0 con 0x55d5bdf2f230 2024-05-10T10:40:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d5bdde0e20 con 0x55d5bdf29dd0 2024-05-10T10:40:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa328383b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5bde77980 con 0x55d5bdf2fb50 2024-05-10T10:40:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa3177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf2f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa3177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf2f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34154/0 (socket says 172.21.15.112:34154) 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.809+0000 7fa3177fe700 1 -- 172.21.15.112:0/2870795889 learned_addr learned my addr 172.21.15.112:0/2870795889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa317fff700 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf2a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa316ffd700 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf29890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa3177fe700 1 -- 172.21.15.112:0/2870795889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2fb50 msgr2=0x55d5bdf29890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa3177fe700 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf29890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa3177fe700 1 -- 172.21.15.112:0/2870795889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 msgr2=0x55d5bdf2a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa3177fe700 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf2a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa3177fe700 1 -- 172.21.15.112:0/2870795889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bddc8950 con 0x55d5bdf2f230 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa316ffd700 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf29890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa3177fe700 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf2f610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa3080a7de0 tx=0x7fa3080099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd8141c12c28122 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa2fdffb700 1 -- 172.21.15.112:0/2870795889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3080aef20 con 0x55d5bdf2f230 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa2fdffb700 1 -- 172.21.15.112:0/2870795889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3080b3940 con 0x55d5bdf2f230 2024-05-10T10:40:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.810+0000 7fa2fdffb700 1 -- 172.21.15.112:0/2870795889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3080aadd0 con 0x55d5bdf2f230 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 -- 172.21.15.112:0/2870795889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 msgr2=0x55d5bdf2f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf2f610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa3080a7de0 tx=0x7fa3080099e0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 -- 172.21.15.112:0/2870795889 shutdown_connections 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf2a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf2f610 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 --2- 172.21.15.112:0/2870795889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf29890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 -- 172.21.15.112:0/2870795889 >> 172.21.15.112:0/2870795889 conn(0x55d5bde55bf0 msgr2=0x55d5bdf24320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 -- 172.21.15.112:0/2870795889 shutdown_connections 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 -- 172.21.15.112:0/2870795889 wait complete. 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 -- start start 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.811+0000 7fa328383b80 1 Processor -- start 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa328383b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf35820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa328383b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa328383b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf33100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa328383b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d5bde7b9c0 con 0x55d5bdf2f230 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa328383b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d5bde34f50 con 0x55d5bdf2fb50 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa328383b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5bdf36290 con 0x55d5bdf29dd0 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa328383b80 10 client.0 _ll_register_callbacks cb 0x55d5bde0f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf32bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34162/0 (socket says 172.21.15.112:34162) 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 -- 172.21.15.112:0/756694435 learned_addr learned my addr 172.21.15.112:0/756694435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa3177fe700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf35820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa317fff700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf33100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 -- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf29dd0 msgr2=0x55d5bdf35820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf35820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 -- 172.21.15.112:0/756694435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf2fb50 msgr2=0x55d5bdf33100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf33100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 -- 172.21.15.112:0/756694435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bde327e0 con 0x55d5bdf2f230 2024-05-10T10:40:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa317fff700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf33100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa316ffd700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf32bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa31807ad40 tx=0x7fa30c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d043b172d768b1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa2f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.812+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/756694435 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c023020 con 0x55d5bdf2f230 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/756694435 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa30c003880 con 0x55d5bdf2f230 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3080b3940 con 0x55d5bdf2f230 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/756694435 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa30c02e050 con 0x55d5bdf2f230 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 10 client.15378 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d5bdf409f0 con 0x55d5bdf2f230 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/756694435 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3080b3940 con 0x55d5bdf2f230 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/756694435 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa30c011a60 con 0x55d5bdf2f230 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 20 client.15378 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdf454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 4 client.15378 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.813+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5bde893a0 con 0x55d5bdf430d0 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.814+0000 7fa3177fe700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdf454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.814+0000 7fa2f6ffd700 10 client.15378 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.814+0000 7fa3177fe700 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdf454e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bbe6894f48ed82e server_cookie=7c8aff7720ecdae in_seq=0 out_seq=0 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/756694435 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d5bde893a0 con 0x55d5bdf430d0 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa2f6ffd700 10 client.15378 handle_command_reply: tid=0 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 client.15378 shutdown 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 msgr2=0x55d5bdf32bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf32bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa31807ad40 tx=0x7fa30c004750 comp rx=0 tx=0).stop 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 shutdown_connections 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdf454e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa2f6ffd700 0 client.15378 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf33100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf35820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 --2- 172.21.15.112:0/756694435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2f230 0x55d5bdf32bc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 >> 172.21.15.112:0/756694435 conn(0x55d5bde55bf0 msgr2=0x55d5bdf1ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 shutdown_connections 2024-05-10T10:40:20.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 1 -- 172.21.15.112:0/756694435 wait complete. 2024-05-10T10:40:20.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.815+0000 7fa328383b80 20 client.15378 trim_cache size 0 max 16384 2024-05-10T10:40:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 Processor -- start 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 -- start start 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf29890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf31b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf3f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181b1380 con 0x55d5bdf2fb50 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181b2ec0 con 0x55d5bdf29dd0 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.835+0000 7fa328383b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181b1a50 con 0x55d5bdf2f230 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf3f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf3f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34174/0 (socket says 172.21.15.112:34174) 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 -- 172.21.15.112:0/3352041007 learned_addr learned my addr 172.21.15.112:0/3352041007 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf29890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa316ffd700 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf31b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 -- 172.21.15.112:0/3352041007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 msgr2=0x55d5bdf31b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf31b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 -- 172.21.15.112:0/3352041007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 msgr2=0x55d5bdf29890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf29890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 -- 172.21.15.112:0/3352041007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bdf2f860 con 0x55d5bdf2fb50 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf29890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa316ffd700 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf31b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa317fff700 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf3f1a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa31000cb30 tx=0x7fa31000ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3261d3379f5d2b85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa2fd7fa700 1 -- 172.21.15.112:0/3352041007 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310018070 con 0x55d5bdf2fb50 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa2fd7fa700 1 -- 172.21.15.112:0/3352041007 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa310014430 con 0x55d5bdf2fb50 2024-05-10T10:40:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.836+0000 7fa2fd7fa700 1 -- 172.21.15.112:0/3352041007 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310010ea0 con 0x55d5bdf2fb50 2024-05-10T10:40:20.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 -- 172.21.15.112:0/3352041007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 msgr2=0x55d5bdf3f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf3f1a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa31000cb30 tx=0x7fa31000ce40 comp rx=0 tx=0).stop 2024-05-10T10:40:20.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 -- 172.21.15.112:0/3352041007 shutdown_connections 2024-05-10T10:40:20.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf3f1a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf29890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 --2- 172.21.15.112:0/3352041007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf31b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 -- 172.21.15.112:0/3352041007 >> 172.21.15.112:0/3352041007 conn(0x55d5bde55bf0 msgr2=0x55d5bdf2ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 -- 172.21.15.112:0/3352041007 shutdown_connections 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 -- 172.21.15.112:0/3352041007 wait complete. 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.837+0000 7fa328383b80 1 -- start start 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 1 Processor -- start 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf4a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf4aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf4b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181b27f0 con 0x55d5bdf2fb50 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181b2120 con 0x55d5bdf29dd0 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181af320 con 0x55d5bdf2f230 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa328383b80 10 client.0 _ll_register_callbacks cb 0x55d5bde0f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf4b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf4b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34184/0 (socket says 172.21.15.112:34184) 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 -- 172.21.15.112:0/3899747364 learned_addr learned my addr 172.21.15.112:0/3899747364 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf4a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa316ffd700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf4aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 -- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 msgr2=0x55d5bdf4aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf4aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 -- 172.21.15.112:0/3899747364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 msgr2=0x55d5bdf4a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf4a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa317fff700 1 -- 172.21.15.112:0/3899747364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bdfacb20 con 0x55d5bdf2fb50 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.838+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf4a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa317fff700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf4b1e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa310006e70 tx=0x7fa310010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3182eb63ce4cecfe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa2f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/3899747364 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31002a060 con 0x55d5bdf2fb50 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/3899747364 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa31001d860 con 0x55d5bdf2fb50 2024-05-10T10:40:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa310014430 con 0x55d5bdf2fb50 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/3899747364 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa310031050 con 0x55d5bdf2fb50 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 10 client.15384 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d5bdfaddb0 con 0x55d5bdf2fb50 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/3899747364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa310014430 con 0x55d5bdf2fb50 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/3899747364 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa3100135d0 con 0x55d5bdf2fb50 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 20 client.15384 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdfb0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 4 client.15384 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.839+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5bde83440 con 0x55d5bdf430d0 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.840+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdfb0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.840+0000 7fa3177fe700 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdfb0410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522a18e84aeb8796 server_cookie=bf81789a9c26ff6f in_seq=0 out_seq=0 2024-05-10T10:40:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.840+0000 7fa2f6ffd700 10 client.15384 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.841+0000 7fa2f6ffd700 1 -- 172.21.15.112:0/3899747364 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5bde83440 con 0x55d5bdf430d0 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.841+0000 7fa2f6ffd700 10 client.15384 handle_command_reply: tid=0 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.841+0000 7fa328383b80 1 client.15384 shutdown 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.841+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 msgr2=0x55d5bdf4b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.841+0000 7fa328383b80 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf4b1e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa310006e70 tx=0x7fa310010cc0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 shutdown_connections 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d5bdf430d0 0x55d5bdfb0410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5bdf2fb50 0x55d5bdf4b1e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5bdf29dd0 0x55d5bdf4a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa2f6ffd700 0 client.15384 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 --2- 172.21.15.112:0/3899747364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5bdf2f230 0x55d5bdf4aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 >> 172.21.15.112:0/3899747364 conn(0x55d5bde55bf0 msgr2=0x55d5bdf2dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 shutdown_connections 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 1 -- 172.21.15.112:0/3899747364 wait complete. 2024-05-10T10:40:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa328383b80 20 client.15384 trim_cache size 0 max 16384 2024-05-10T10:40:20.848 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:40:20.848 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:40:20.848 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "8cb92a69-715c-453b-819c-76c0ec1639b4", 2024-05-10T10:40:20.848 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:40:20.848 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3000b2880 msgr2=0x7fa3000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3000b2880 0x7fa3000b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005950 tx=0x7fa30c0058e0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318103560 msgr2=0x7fa3180757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318103560 0x7fa3180757a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa308007d90 tx=0x7fa308007e70 comp rx=0 tx=0).stop 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 shutdown_connections 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa3000b2880 0x7fa3000b4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa318109540 0x7fa318076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa318103560 0x7fa3180757a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 --2- 172.21.15.112:0/3891215336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa318103f20 0x7fa318075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.842+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 >> 172.21.15.112:0/3891215336 conn(0x7fa31807a660 msgr2=0x7fa3181023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.843+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 shutdown_connections 2024-05-10T10:40:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:20.843+0000 7fa31daf0700 1 -- 172.21.15.112:0/3891215336 wait complete. 2024-05-10T10:40:21.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:20 smithi148 ceph-mon[33568]: pgmap v433: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-10T10:40:21.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:20 smithi195 ceph-mon[33773]: pgmap v433: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-10T10:40:21.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:40:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:21 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:40:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:21 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:40:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:21 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:40:21.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:21 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:40:21.869 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:22.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:21 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:40:22.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:21 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:40:22.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:21 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:40:22.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:21 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:40:22.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:21 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:40:22.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:21 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:40:22.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:21 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:40:22.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:21 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:40:23.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:22 smithi148 ceph-mon[33568]: pgmap v434: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T10:40:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:22 smithi195 ceph-mon[33773]: pgmap v434: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T10:40:23.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:22 smithi112 ceph-mon[30100]: pgmap v434: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T10:40:23.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 -- 172.21.15.112:0/3094220880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4901019f0 msgr2=0x7fe490101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 --2- 172.21.15.112:0/3094220880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4901019f0 0x7fe490101e70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe4800099c0 tx=0x7fe480009cd0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 -- 172.21.15.112:0/3094220880 shutdown_connections 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 --2- 172.21.15.112:0/3094220880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4901023b0 0x7fe490110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 --2- 172.21.15.112:0/3094220880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe490109570 0x7fe490109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 --2- 172.21.15.112:0/3094220880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4901019f0 0x7fe490101e70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 -- 172.21.15.112:0/3094220880 >> 172.21.15.112:0/3094220880 conn(0x7fe4900789a0 msgr2=0x7fe4900ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 -- 172.21.15.112:0/3094220880 shutdown_connections 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 -- 172.21.15.112:0/3094220880 wait complete. 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 Processor -- start 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.644+0000 7fe4987ef700 1 -- start start 2024-05-10T10:40:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe4987ef700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4901019f0 0x7fe49019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe4987ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4901023b0 0x7fe49019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe4987ef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe490109570 0x7fe4901997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe4987ef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe490113510 con 0x7fe490109570 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe4987ef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe490113390 con 0x7fe4901023b0 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe4987ef700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe490113690 con 0x7fe4901019f0 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe490109570 0x7fe4901997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe490109570 0x7fe4901997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34208/0 (socket says 172.21.15.112:34208) 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 -- 172.21.15.112:0/851530381 learned_addr learned my addr 172.21.15.112:0/851530381 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe49658b700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4901019f0 0x7fe49019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe495d8a700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4901023b0 0x7fe49019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 -- 172.21.15.112:0/851530381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4901019f0 msgr2=0x7fe49019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4901019f0 0x7fe49019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 -- 172.21.15.112:0/851530381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4901023b0 msgr2=0x7fe49019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4901023b0 0x7fe49019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 -- 172.21.15.112:0/851530381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe490199fb0 con 0x7fe490109570 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe495d8a700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4901023b0 0x7fe49019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe496d8c700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe490109570 0x7fe4901997d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe48800ada0 tx=0x7fe48800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.645+0000 7fe4877fe700 1 -- 172.21.15.112:0/851530381 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488017020 con 0x7fe490109570 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.646+0000 7fe4877fe700 1 -- 172.21.15.112:0/851530381 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe48800ce10 con 0x7fe490109570 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.646+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49019a2a0 con 0x7fe490109570 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.646+0000 7fe4877fe700 1 -- 172.21.15.112:0/851530381 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488005280 con 0x7fe490109570 2024-05-10T10:40:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.646+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4901a64b0 con 0x7fe490109570 2024-05-10T10:40:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.648+0000 7fe4877fe700 1 -- 172.21.15.112:0/851530381 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe48800c980 con 0x7fe490109570 2024-05-10T10:40:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.649+0000 7fe4877fe700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe47c0b2880 0x7fe47c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.649+0000 7fe4877fe700 1 -- 172.21.15.112:0/851530381 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe4880e1d40 con 0x7fe490109570 2024-05-10T10:40:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.649+0000 7fe49658b700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe47c0b2880 0x7fe47c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.649+0000 7fe49658b700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe47c0b2880 0x7fe47c0b4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00ac50 tx=0x7fe48c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.649+0000 7fe4a3082b80 1 Processor -- start 2024-05-10T10:40:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe4a3082b80 1 -- start start 2024-05-10T10:40:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dc87970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc81c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dc82610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56213dbcf940 con 0x56213dc821b0 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56213db34da0 con 0x56213dc87eb0 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56213dbcb900 con 0x56213dc87590 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dc82610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dc82610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34222/0 (socket says 172.21.15.112:34222) 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 -- 172.21.15.112:0/1167177793 learned_addr learned my addr 172.21.15.112:0/1167177793 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe49658b700 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dc87970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe495d8a700 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc81c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 -- 172.21.15.112:0/1167177793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 msgr2=0x56213dc87970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dc87970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 -- 172.21.15.112:0/1167177793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 msgr2=0x56213dc81c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc81c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 -- 172.21.15.112:0/1167177793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213db1c8d0 con 0x56213dc821b0 2024-05-10T10:40:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.650+0000 7fe496d8c700 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dc82610 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe4880a8ff0 tx=0x7fe48800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fcab77e6c8b35d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe484ff9700 1 -- 172.21.15.112:0/1167177793 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4880a8b70 con 0x56213dc821b0 2024-05-10T10:40:23.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe484ff9700 1 -- 172.21.15.112:0/1167177793 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4880a9760 con 0x56213dc821b0 2024-05-10T10:40:23.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe484ff9700 1 -- 172.21.15.112:0/1167177793 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4880a9520 con 0x56213dc821b0 2024-05-10T10:40:23.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1167177793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 msgr2=0x56213dc82610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dc82610 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe4880a8ff0 tx=0x7fe48800ccf0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1167177793 shutdown_connections 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dc82610 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dc87970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1167177793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc81c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.651+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1167177793 >> 172.21.15.112:0/1167177793 conn(0x56213dba9b70 msgr2=0x56213dc782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1167177793 shutdown_connections 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1167177793 wait complete. 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 -- start start 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 Processor -- start 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc8db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87590 0x56213dc8aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87eb0 0x56213dc8b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56213dc8e8e0 con 0x56213dc87eb0 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56213db88ed0 con 0x56213dc87590 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56213dc8ea60 con 0x56213dc821b0 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe496d8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87eb0 0x56213dc8b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe4a3082b80 10 client.0 _ll_register_callbacks cb 0x56213db635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe496d8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87eb0 0x56213dc8b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34238/0 (socket says 172.21.15.112:34238) 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe496d8c700 1 -- 172.21.15.112:0/1680619805 learned_addr learned my addr 172.21.15.112:0/1680619805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe495d8a700 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87590 0x56213dc8aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.652+0000 7fe49658b700 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc8db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe496d8c700 1 -- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 msgr2=0x56213dc8db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe496d8c700 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc8db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe496d8c700 1 -- 172.21.15.112:0/1680619805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87590 msgr2=0x56213dc8aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe496d8c700 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87590 0x56213dc8aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe496d8c700 1 -- 172.21.15.112:0/1680619805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213db86760 con 0x56213dc87eb0 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe496d8c700 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87eb0 0x56213dc8b410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe4880a62d0 tx=0x7fe4880ad2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09cc08023d3d6f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe479ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe479ffb700 1 -- 172.21.15.112:0/1680619805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48806c070 con 0x56213dc87eb0 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe479ffb700 1 -- 172.21.15.112:0/1680619805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe4880b1bb0 con 0x56213dc87eb0 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4880a9760 con 0x56213dc87eb0 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe479ffb700 1 -- 172.21.15.112:0/1680619805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4880a9060 con 0x56213dc87eb0 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe4a3082b80 10 client.15396 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:23.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56213dc99150 con 0x56213dc87eb0 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.653+0000 7fe479ffb700 1 -- 172.21.15.112:0/1680619805 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4880a9760 con 0x56213dc87eb0 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe479ffb700 1 -- 172.21.15.112:0/1680619805 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe488012070 con 0x56213dc87eb0 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe4a3082b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe4a3082b80 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe4a3082b80 20 client.15396 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dc9b620 0x56213dc9da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe4a3082b80 4 client.15396 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56213dbdd320 con 0x56213dc9b620 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe49658b700 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dc9b620 0x56213dc9da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe49658b700 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dc9b620 0x56213dc9da10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e70000ebca4e95 server_cookie=856f13ece9501c46 in_seq=0 out_seq=0 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.654+0000 7fe479ffb700 10 client.15396 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe479ffb700 1 -- 172.21.15.112:0/1680619805 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56213dbdd320 con 0x56213dc9b620 2024-05-10T10:40:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe479ffb700 10 client.15396 handle_command_reply: tid=0 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 client.15396 shutdown 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87eb0 msgr2=0x56213dc8b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87eb0 0x56213dc8b410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe4880a62d0 tx=0x7fe4880ad2f0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 shutdown_connections 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dc9b620 0x56213dc9da10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87eb0 0x56213dc8b410 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc8db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1680619805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87590 0x56213dc8aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe479ffb700 0 client.15396 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 >> 172.21.15.112:0/1680619805 conn(0x56213dba9b70 msgr2=0x56213dc72b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 shutdown_connections 2024-05-10T10:40:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.655+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1680619805 wait complete. 2024-05-10T10:40:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.656+0000 7fe4a3082b80 20 client.15396 trim_cache size 0 max 16384 2024-05-10T10:40:23.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 Processor -- start 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 -- start start 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc72c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87590 0x56213dc96cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc97230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901acf70 con 0x56213dc87590 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901aeab0 con 0x56213dc87eb0 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901ad640 con 0x56213dc821b0 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe495d8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87590 0x56213dc96cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.675+0000 7fe495d8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87590 0x56213dc96cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34254/0 (socket says 172.21.15.112:34254) 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe495d8a700 1 -- 172.21.15.112:0/1430018828 learned_addr learned my addr 172.21.15.112:0/1430018828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe49658b700 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc72c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe495d8a700 1 -- 172.21.15.112:0/1430018828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 msgr2=0x56213dc72c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe495d8a700 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc72c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe496d8c700 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc97230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe495d8a700 1 -- 172.21.15.112:0/1430018828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 msgr2=0x56213dc97230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe495d8a700 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc97230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe495d8a700 1 -- 172.21.15.112:0/1430018828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213dc74c80 con 0x56213dc87590 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe495d8a700 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87590 0x56213dc96cf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe48000a010 tx=0x7fe48001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a3014c22c8e9c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe47bfff700 1 -- 172.21.15.112:0/1430018828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480005590 con 0x56213dc87590 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe47bfff700 1 -- 172.21.15.112:0/1430018828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe480029070 con 0x56213dc87590 2024-05-10T10:40:23.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.676+0000 7fe47bfff700 1 -- 172.21.15.112:0/1430018828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480003be0 con 0x56213dc87590 2024-05-10T10:40:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1430018828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87590 msgr2=0x56213dc96cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87590 0x56213dc96cf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe48000a010 tx=0x7fe48001a040 comp rx=0 tx=0).stop 2024-05-10T10:40:23.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1430018828 shutdown_connections 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dc97230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc821b0 0x56213dc72c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/1430018828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc87590 0x56213dc96cf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1430018828 >> 172.21.15.112:0/1430018828 conn(0x56213dba9b70 msgr2=0x56213dc7d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1430018828 shutdown_connections 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 -- 172.21.15.112:0/1430018828 wait complete. 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 -- start start 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.677+0000 7fe4a3082b80 1 Processor -- start 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dcf9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dcfa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe4a3082b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dd04b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901ae3e0 con 0x56213dc821b0 2024-05-10T10:40:23.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901add10 con 0x56213dc87eb0 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe4a3082b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901aaf30 con 0x56213dc87590 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe4a3082b80 10 client.0 _ll_register_callbacks cb 0x56213db635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dcf9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dcf9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34268/0 (socket says 172.21.15.112:34268) 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 -- 172.21.15.112:0/4218184575 learned_addr learned my addr 172.21.15.112:0/4218184575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe495d8a700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dcfa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe496d8c700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dd04b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 -- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 msgr2=0x56213dcfa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dcfa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 -- 172.21.15.112:0/4218184575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 msgr2=0x56213dd04b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dd04b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 -- 172.21.15.112:0/4218184575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213dd05110 con 0x56213dc821b0 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe495d8a700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dcfa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:40:23.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe49658b700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dcf9cb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe48c004490 tx=0x7fe48c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e8993343a26216 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.678+0000 7fe479ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe479ffb700 1 -- 172.21.15.112:0/4218184575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c016070 con 0x56213dc821b0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe480029070 con 0x56213dc821b0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe479ffb700 1 -- 172.21.15.112:0/4218184575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe48c0293b0 con 0x56213dc821b0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe479ffb700 1 -- 172.21.15.112:0/4218184575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe48c038050 con 0x56213dc821b0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 10 client.15402 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56213dd06480 con 0x56213dc821b0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe479ffb700 1 -- 172.21.15.112:0/4218184575 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe480029070 con 0x56213dc821b0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe479ffb700 1 -- 172.21.15.112:0/4218184575 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe48c010910 con 0x56213dc821b0 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 10 client.15402 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 10 client.15402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 20 client.15402 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:23.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dd06bc0 0x56213dd08fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 4 client.15402 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.679+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56213dbd73c0 con 0x56213dd06bc0 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.680+0000 7fe495d8a700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dd06bc0 0x56213dd08fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.680+0000 7fe479ffb700 10 client.15402 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.680+0000 7fe495d8a700 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dd06bc0 0x56213dd08fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a00b8e7e2fa39c server_cookie=21efee61965772cb in_seq=0 out_seq=0 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.680+0000 7fe479ffb700 1 -- 172.21.15.112:0/4218184575 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56213dbd73c0 con 0x56213dd06bc0 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.680+0000 7fe479ffb700 10 client.15402 handle_command_reply: tid=0 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.680+0000 7fe4a3082b80 1 client.15402 shutdown 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 msgr2=0x56213dcf9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dcf9cb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe48c004490 tx=0x7fe48c0047c0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 shutdown_connections 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56213dd06bc0 0x56213dd08fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56213dc87eb0 0x56213dd04b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56213dc821b0 0x56213dcf9cb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe479ffb700 0 client.15402 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 --2- 172.21.15.112:0/4218184575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56213dc87590 0x56213dcfa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 >> 172.21.15.112:0/4218184575 conn(0x56213dba9b70 msgr2=0x56213dc8c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 shutdown_connections 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 1 -- 172.21.15.112:0/4218184575 wait complete. 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4a3082b80 20 client.15402 trim_cache size 0 max 16384 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-10T10:40:23.688 INFO:teuthology.orchestra.run.smithi112.stdout: "8cb92a69-715c-453b-819c-76c0ec1639b4": { 2024-05-10T10:40:23.689 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-10T10:40:23.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "8cb92a69-715c-453b-819c-76c0ec1639b4", 2024-05-10T10:40:23.689 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-10T10:40:23.689 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:40:23.689 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:40:23.689 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:40:23.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe47c0b2880 msgr2=0x7fe47c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4987ef700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe47c0b2880 0x7fe47c0b4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00ac50 tx=0x7fe48c00a380 comp rx=0 tx=0).stop 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe490109570 msgr2=0x7fe4901997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.681+0000 7fe4987ef700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe490109570 0x7fe4901997d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe48800ada0 tx=0x7fe48800c330 comp rx=0 tx=0).stop 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 shutdown_connections 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe490109570 0x7fe4901997d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe47c0b2880 0x7fe47c0b4d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4901019f0 0x7fe49019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 --2- 172.21.15.112:0/851530381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4901023b0 0x7fe49019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 >> 172.21.15.112:0/851530381 conn(0x7fe4900789a0 msgr2=0x7fe4901001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 shutdown_connections 2024-05-10T10:40:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:23.682+0000 7fe4987ef700 1 -- 172.21.15.112:0/851530381 wait complete. 2024-05-10T10:40:24.443 INFO:tasks.cephfs.filesystem:scrub status for tag:8cb92a69-715c-453b-819c-76c0ec1639b4 - {'path': '/', 'tag': '8cb92a69-715c-453b-819c-76c0ec1639b4', 'options': 'recursive,force'} 2024-05-10T10:40:25.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:24 smithi148 ceph-mon[33568]: pgmap v435: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-10T10:40:25.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:24 smithi195 ceph-mon[33773]: pgmap v435: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-10T10:40:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:24 smithi112 ceph-mon[30100]: pgmap v435: 129 pgs: 129 active+clean; 656 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-10T10:40:27.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:26 smithi148 ceph-mon[33568]: pgmap v436: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-10T10:40:27.069 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:26 smithi195 ceph-mon[33773]: pgmap v436: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-10T10:40:27.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:26 smithi112 ceph-mon[30100]: pgmap v436: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-10T10:40:29.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:28 smithi148 ceph-mon[33568]: pgmap v437: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-10T10:40:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:28 smithi195 ceph-mon[33773]: pgmap v437: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-10T10:40:29.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:28 smithi112 ceph-mon[30100]: pgmap v437: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-10T10:40:31.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:30 smithi148 ceph-mon[33568]: pgmap v438: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T10:40:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:30 smithi195 ceph-mon[33773]: pgmap v438: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T10:40:31.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:30 smithi112 ceph-mon[30100]: pgmap v438: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T10:40:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/1562785974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 msgr2=0x7fa5a4109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/1562785974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a4109e00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ca0 tx=0x7fa59c02a8c0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/1562785974 shutdown_connections 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/1562785974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a4109e00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/1562785974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a4102bb0 0x7fa5a4103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/1562785974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5a41019c0 0x7fa5a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/1562785974 >> 172.21.15.112:0/1562785974 conn(0x7fa5a40fcf30 msgr2=0x7fa5a40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/1562785974 shutdown_connections 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/1562785974 wait complete. 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.866+0000 7fa5aa7f2640 1 Processor -- start 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5aa7f2640 1 -- start start 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5aa7f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5a41019c0 0x7fa5a41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5aa7f2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a4102bb0 0x7fa5a4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5aa7f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a419ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5aa7f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a410e490 con 0x7fa5a4103570 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5aa7f2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a410e310 con 0x7fa5a4102bb0 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5aa7f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a410e610 con 0x7fa5a41019c0 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a419ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a419ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48754/0 (socket says 172.21.15.112:48754) 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/4018965043 learned_addr learned my addr 172.21.15.112:0/4018965043 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5a41019c0 0x7fa5a41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa59bfff640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a4102bb0 0x7fa5a4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/4018965043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5a41019c0 msgr2=0x7fa5a41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5a41019c0 0x7fa5a41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/4018965043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a4102bb0 msgr2=0x7fa5a4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a4102bb0 0x7fa5a4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/4018965043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a41a0600 con 0x7fa5a4103570 2024-05-10T10:40:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.867+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a419ff90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02aeb0 tx=0x7fa59c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.868+0000 7fa5a1ffb640 1 -- 172.21.15.112:0/4018965043 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c032ed0 con 0x7fa5a4103570 2024-05-10T10:40:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.868+0000 7fa5a1ffb640 1 -- 172.21.15.112:0/4018965043 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa59c003e10 con 0x7fa5a4103570 2024-05-10T10:40:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.868+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a41a0890 con 0x7fa5a4103570 2024-05-10T10:40:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.868+0000 7fa5a1ffb640 1 -- 172.21.15.112:0/4018965043 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c030c10 con 0x7fa5a4103570 2024-05-10T10:40:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.868+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a41a0e30 con 0x7fa5a4103570 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.870+0000 7fa5a1ffb640 1 -- 172.21.15.112:0/4018965043 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa59c004240 con 0x7fa5a4103570 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.870+0000 7fa5ad5e1740 1 Processor -- start 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.870+0000 7fa5ad5e1740 1 -- start start 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.870+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030aba61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.870+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abadda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56030ab0a5d0 con 0x56030abab910 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56030aac3890 con 0x56030aba5db0 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56030ab06590 con 0x56030aba67a0 2024-05-10T10:40:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abadda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abadda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48758/0 (socket says 172.21.15.112:48758) 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 -- 172.21.15.112:0/1913183220 learned_addr learned my addr 172.21.15.112:0/1913183220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030aba61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 -- 172.21.15.112:0/1913183220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 msgr2=0x56030abab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 -- 172.21.15.112:0/1913183220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 msgr2=0x56030aba61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030aba61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 -- 172.21.15.112:0/1913183220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030a81c210 con 0x56030abab910 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5a1ffb640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5900b2920 0x7fa5900b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5900b2920 0x7fa5900b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59bfff640 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abadda0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00a5c0 tx=0x7fa58c00a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e904111f6ff3d2c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa5a1ffb640 1 -- 172.21.15.112:0/4018965043 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa59c103b50 con 0x7fa5a4103570 2024-05-10T10:40:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59a7fc640 1 -- 172.21.15.112:0/1913183220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00f910 con 0x56030abab910 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59a7fc640 1 -- 172.21.15.112:0/1913183220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa58c01b390 con 0x56030abab910 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.871+0000 7fa59a7fc640 1 -- 172.21.15.112:0/1913183220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00d540 con 0x56030abab910 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/1913183220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 msgr2=0x56030abadda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abadda0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00a5c0 tx=0x7fa58c00a810 comp rx=0 tx=0).stop 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5900b2920 0x7fa5900b4de0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56030aba6ba0 tx=0x7fa59400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/1913183220 shutdown_connections 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abadda0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/1913183220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030aba61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/1913183220 >> 172.21.15.112:0/1913183220 conn(0x56030aba1420 msgr2=0x56030aba1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/1913183220 shutdown_connections 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/1913183220 wait complete. 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.872+0000 7fa5ad5e1740 1 -- start start 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 1 Processor -- start 2024-05-10T10:40:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030abb7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56030abae8c0 con 0x56030aba5db0 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56030aac7370 con 0x56030abab910 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56030abaea40 con 0x56030aba67a0 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5ad5e1740 10 client.0 _ll_register_callbacks cb 0x56030a81ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48766/0 (socket says 172.21.15.112:48766) 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 -- 172.21.15.112:0/2312826760 learned_addr learned my addr 172.21.15.112:0/2312826760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa59bfff640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 -- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 msgr2=0x56030abb05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030abb7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 -- 172.21.15.112:0/2312826760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 msgr2=0x56030abb7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030abb7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a3fff640 1 -- 172.21.15.112:0/2312826760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56030a8e8620 con 0x56030aba5db0 2024-05-10T10:40:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa59bfff640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.873+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030abb7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa594010860 tx=0x7fa5940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4045d0b64dfcbf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5837fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5837fe640 1 -- 172.21.15.112:0/2312826760 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400fac0 con 0x56030aba5db0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5837fe640 1 -- 172.21.15.112:0/2312826760 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa59400fc60 con 0x56030aba5db0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c01b390 con 0x56030aba5db0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5837fe640 1 -- 172.21.15.112:0/2312826760 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa59400fe60 con 0x56030aba5db0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 10 client.15414 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56030abba6b0 con 0x56030aba5db0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5837fe640 1 -- 172.21.15.112:0/2312826760 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa58c01b390 con 0x56030aba5db0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5837fe640 1 -- 172.21.15.112:0/2312826760 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa59401f430 con 0x56030aba5db0 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 10 client.15414 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 20 client.15414 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030abbbda0 0x56030abbe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 4 client.15414 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.874+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56030ab181e0 con 0x56030abbbda0 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.875+0000 7fa59bfff640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030abbbda0 0x56030abbe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.875+0000 7fa5837fe640 10 client.15414 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.875+0000 7fa59bfff640 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030abbbda0 0x56030abbe240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=117bcfb26912b4af server_cookie=2a9965224f1850d2 in_seq=0 out_seq=0 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5837fe640 1 -- 172.21.15.112:0/2312826760 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56030ab181e0 con 0x56030abbbda0 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5837fe640 10 client.15414 handle_command_reply: tid=0 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 client.15414 shutdown 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 msgr2=0x56030abb0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa594010860 tx=0x7fa5940046c0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 shutdown_connections 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030abbbda0 0x56030abbe240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030abb7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5837fe640 0 client.15414 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/2312826760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.876+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 >> 172.21.15.112:0/2312826760 conn(0x56030aba1420 msgr2=0x56030aba0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.877+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 shutdown_connections 2024-05-10T10:40:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.877+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/2312826760 wait complete. 2024-05-10T10:40:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.877+0000 7fa5ad5e1740 20 client.15414 trim_cache size 0 max 16384 2024-05-10T10:40:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 Processor -- start 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 -- start start 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030aba4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030aba4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41ab290 con 0x56030aba5db0 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41acdd0 con 0x56030abab910 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41ab960 con 0x56030aba67a0 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48782/0 (socket says 172.21.15.112:48782) 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 -- 172.21.15.112:0/3468275353 learned_addr learned my addr 172.21.15.112:0/3468275353 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa59bfff640 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030aba4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 -- 172.21.15.112:0/3468275353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 msgr2=0x56030aba4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030aba4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030aba4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 -- 172.21.15.112:0/3468275353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 msgr2=0x56030aba4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030aba4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 -- 172.21.15.112:0/3468275353 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030aba1250 con 0x56030aba5db0 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.892+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa594005940 tx=0x7fa594010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaa7681c3ee9cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa599ffb640 1 -- 172.21.15.112:0/3468275353 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59401f3f0 con 0x56030aba5db0 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa599ffb640 1 -- 172.21.15.112:0/3468275353 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa59400fc20 con 0x56030aba5db0 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa599ffb640 1 -- 172.21.15.112:0/3468275353 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594034a30 con 0x56030aba5db0 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/3468275353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 msgr2=0x56030abb0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa594005940 tx=0x7fa594010440 comp rx=0 tx=0).stop 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/3468275353 shutdown_connections 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030abab910 0x56030aba4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030aba4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/3468275353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030aba5db0 0x56030abb0520 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/3468275353 >> 172.21.15.112:0/3468275353 conn(0x56030aba1420 msgr2=0x56030abaf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/3468275353 shutdown_connections 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/3468275353 wait complete. 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.893+0000 7fa5ad5e1740 1 -- start start 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 1 Processor -- start 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030abb4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abb61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41ac700 con 0x56030abab910 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41ac030 con 0x56030aba5db0 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41a9270 con 0x56030aba67a0 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5ad5e1740 10 client.0 _ll_register_callbacks cb 0x7fa59400b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abb61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abb61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48786/0 (socket says 172.21.15.112:48786) 2024-05-10T10:40:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/639610048 learned_addr learned my addr 172.21.15.112:0/639610048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa59bfff640 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 msgr2=0x56030abb51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030abb4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/639610048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 msgr2=0x56030abb4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030abb4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.894+0000 7fa5a8d8f640 1 -- 172.21.15.112:0/639610048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56030abb67a0 con 0x56030abab910 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5a8d8f640 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abb61b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0c6e60 tx=0x7fa59c030700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa6235c2a1ef05b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5837fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5837fe640 1 -- 172.21.15.112:0/639610048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0cade0 con 0x56030abab910 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5837fe640 1 -- 172.21.15.112:0/639610048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa59c0cb040 con 0x56030abab910 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59400fc20 con 0x56030abab910 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5837fe640 1 -- 172.21.15.112:0/639610048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa59c0cf190 con 0x56030abab910 2024-05-10T10:40:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 10 client.15420 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56030ac27060 con 0x56030abab910 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5837fe640 1 -- 172.21.15.112:0/639610048 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa59400fc20 con 0x56030abab910 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5837fe640 1 -- 172.21.15.112:0/639610048 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa59c004b40 con 0x56030abab910 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 10 client.15420 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 20 client.15420 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030ac27a60 0x56030ac29e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 4 client.15420 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56030ab12050 con 0x56030ac27a60 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.895+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030ac27a60 0x56030ac29e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.896+0000 7fa5a3fff640 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030ac27a60 0x56030ac29e80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9471ba139b1a27f2 server_cookie=b82f6b5501a44fb0 in_seq=0 out_seq=0 2024-05-10T10:40:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.896+0000 7fa5837fe640 10 client.15420 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.909+0000 7fa5837fe640 1 -- 172.21.15.112:0/639610048 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56030ab12050 con 0x56030ac27a60 2024-05-10T10:40:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.909+0000 7fa5837fe640 10 client.15420 handle_command_reply: tid=0 2024-05-10T10:40:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.909+0000 7fa5ad5e1740 1 client.15420 shutdown 2024-05-10T10:40:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.909+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 msgr2=0x56030abb61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abb61b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0c6e60 tx=0x7fa59c030700 comp rx=0 tx=0).stop 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 shutdown_connections 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030abab910 0x56030abb61b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56030aba67a0 0x56030abb51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56030ac27a60 0x56030ac29e80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 --2- 172.21.15.112:0/639610048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56030aba5db0 0x56030abb4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5837fe640 0 client.15420 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 >> 172.21.15.112:0/639610048 conn(0x56030aba1420 msgr2=0x56030aba4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 shutdown_connections 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 1 -- 172.21.15.112:0/639610048 wait complete. 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5ad5e1740 20 client.15420 trim_cache size 0 max 16384 2024-05-10T10:40:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:40:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:40:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:40:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:40:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5900b2920 msgr2=0x7fa5900b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5900b2920 0x7fa5900b4de0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x56030aba6ba0 tx=0x7fa59400b390 comp rx=0 tx=0).stop 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 msgr2=0x7fa5a419ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a419ff90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02aeb0 tx=0x7fa59c003c20 comp rx=0 tx=0).stop 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 shutdown_connections 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5900b2920 0x7fa5900b4de0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a4103570 0x7fa5a419ff90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a4102bb0 0x7fa5a4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 --2- 172.21.15.112:0/4018965043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5a41019c0 0x7fa5a41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.910+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 >> 172.21.15.112:0/4018965043 conn(0x7fa5a40fcf30 msgr2=0x7fa5a4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.911+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 shutdown_connections 2024-05-10T10:40:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:31.911+0000 7fa5aa7f2640 1 -- 172.21.15.112:0/4018965043 wait complete. 2024-05-10T10:40:33.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:32 smithi148 ceph-mon[33568]: pgmap v439: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T10:40:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:32 smithi195 ceph-mon[33773]: pgmap v439: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T10:40:33.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:32 smithi112 ceph-mon[30100]: pgmap v439: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T10:40:35.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:34 smithi148 ceph-mon[33568]: pgmap v440: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:40:35.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:34 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:40:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:34 smithi195 ceph-mon[33773]: pgmap v440: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:40:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:34 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:40:35.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:34 smithi112 ceph-mon[30100]: pgmap v440: 129 pgs: 129 active+clean; 661 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:40:35.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:34 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:40:37.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:36 smithi148 ceph-mon[33568]: pgmap v441: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-05-10T10:40:37.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:36 smithi195 ceph-mon[33773]: pgmap v441: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-05-10T10:40:37.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:36 smithi112 ceph-mon[30100]: pgmap v441: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-05-10T10:40:38.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:37 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:40:38.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:37 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:40:38.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:37 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:40:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:38 smithi148 ceph-mon[33568]: pgmap v442: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-05-10T10:40:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:40:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:40:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:40:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:38 smithi195 ceph-mon[33773]: pgmap v442: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-05-10T10:40:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:40:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:40:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:40:39.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:38 smithi112 ceph-mon[30100]: pgmap v442: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-05-10T10:40:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:40:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:40:39.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:40:41.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:40 smithi148 ceph-mon[33568]: pgmap v443: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-10T10:40:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:40 smithi195 ceph-mon[33773]: pgmap v443: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-10T10:40:41.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:40 smithi112 ceph-mon[30100]: pgmap v443: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-10T10:40:43.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:42 smithi148 ceph-mon[33568]: pgmap v444: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-10T10:40:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:42 smithi195 ceph-mon[33773]: pgmap v444: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-10T10:40:43.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:42 smithi112 ceph-mon[30100]: pgmap v444: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-05-10T10:40:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 -- 172.21.15.112:0/2610796802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 msgr2=0x7f3244109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 --2- 172.21.15.112:0/2610796802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f3244109d50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f323c009ca0 tx=0x7f323c02a8c0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 -- 172.21.15.112:0/2610796802 shutdown_connections 2024-05-10T10:40:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 --2- 172.21.15.112:0/2610796802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f3244109d50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 --2- 172.21.15.112:0/2610796802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244102b00 0x7f3244102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 --2- 172.21.15.112:0/2610796802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3244101910 0x7f3244101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 -- 172.21.15.112:0/2610796802 >> 172.21.15.112:0/2610796802 conn(0x7f32440fcea0 msgr2=0x7f32440ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 -- 172.21.15.112:0/2610796802 shutdown_connections 2024-05-10T10:40:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.007+0000 7f3249578640 1 -- 172.21.15.112:0/2610796802 wait complete. 2024-05-10T10:40:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 Processor -- start 2024-05-10T10:40:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 -- start start 2024-05-10T10:40:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244101910 0x7f324406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3244102b00 0x7f324419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f32441a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f324410d340 con 0x7f32441034c0 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f324410d1c0 con 0x7f3244101910 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3249578640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f324410d4c0 con 0x7f3244102b00 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f32437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f32441a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f32437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f32441a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34950/0 (socket says 172.21.15.112:34950) 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f32437fe640 1 -- 172.21.15.112:0/916744748 learned_addr learned my addr 172.21.15.112:0/916744748 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f32427fc640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3244102b00 0x7f324419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.008+0000 7f3242ffd640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244101910 0x7f324406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f32437fe640 1 -- 172.21.15.112:0/916744748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3244102b00 msgr2=0x7f324419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f32437fe640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3244102b00 0x7f324419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f32437fe640 1 -- 172.21.15.112:0/916744748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244101910 msgr2=0x7f324406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f32437fe640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244101910 0x7f324406bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f32437fe640 1 -- 172.21.15.112:0/916744748 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32441a28e0 con 0x7f32441034c0 2024-05-10T10:40:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f32437fe640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f32441a20f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f323c02aeb0 tx=0x7f323c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f323bfff640 1 -- 172.21.15.112:0/916744748 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c003e10 con 0x7f32441034c0 2024-05-10T10:40:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f323bfff640 1 -- 172.21.15.112:0/916744748 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f323c0044d0 con 0x7f32441034c0 2024-05-10T10:40:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f323bfff640 1 -- 172.21.15.112:0/916744748 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c030c90 con 0x7f32441034c0 2024-05-10T10:40:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32441a2b10 con 0x7f32441034c0 2024-05-10T10:40:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.009+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32441a3060 con 0x7f32441034c0 2024-05-10T10:40:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.012+0000 7f324c36d740 1 Processor -- start 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.012+0000 7f323bfff640 1 -- 172.21.15.112:0/916744748 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f323c004680 con 0x7f32441034c0 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.012+0000 7f324c36d740 1 -- start start 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f324c36d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059e31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f324c36d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f324c36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e8910 0x55d9059eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f324c36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d9059475d0 con 0x55d9059e8910 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f324c36d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d905900890 con 0x55d9059e37a0 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f324c36d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d905943590 con 0x55d9059e2db0 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e8910 0x55d9059eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e8910 0x55d9059eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34954/0 (socket says 172.21.15.112:34954) 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 -- 172.21.15.112:0/3527502163 learned_addr learned my addr 172.21.15.112:0/3527502163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059e31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f3242ffd640 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f323bfff640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32300b28c0 0x7f32300b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 -- 172.21.15.112:0/3527502163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 msgr2=0x55d9059e31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059e31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 -- 172.21.15.112:0/3527502163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 msgr2=0x55d9059e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.013+0000 7f32427fc640 1 -- 172.21.15.112:0/3527502163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d905659210 con 0x55d9059e8910 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f32427fc640 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e8910 0x55d9059eada0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3234009fd0 tx=0x7f3234005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed92778a9f8c5af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f323bfff640 1 -- 172.21.15.112:0/916744748 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f323c103a80 con 0x7f32441034c0 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f32437fe640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32300b28c0 0x7f32300b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f32397fa640 1 -- 172.21.15.112:0/3527502163 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234004040 con 0x55d9059e8910 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f32397fa640 1 -- 172.21.15.112:0/3527502163 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3234005020 con 0x55d9059e8910 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f32397fa640 1 -- 172.21.15.112:0/3527502163 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234004a90 con 0x55d9059e8910 2024-05-10T10:40:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f324c36d740 1 -- 172.21.15.112:0/3527502163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e8910 msgr2=0x55d9059eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f32437fe640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32300b28c0 0x7f32300b4d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f323c0caca0 tx=0x7f323c004970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.014+0000 7f324c36d740 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e8910 0x55d9059eada0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3234009fd0 tx=0x7f3234005850 comp rx=0 tx=0).stop 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- 172.21.15.112:0/3527502163 shutdown_connections 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e8910 0x55d9059eada0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 --2- 172.21.15.112:0/3527502163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059e31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- 172.21.15.112:0/3527502163 >> 172.21.15.112:0/3527502163 conn(0x55d9059de420 msgr2=0x55d9059de820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- 172.21.15.112:0/3527502163 shutdown_connections 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- 172.21.15.112:0/3527502163 wait complete. 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- start start 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 Processor -- start 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e2db0 0x55d9059ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e8910 0x55d9059f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d9059eb8c0 con 0x55d9059e2db0 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d905904370 con 0x55d9059e37a0 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d9059eba40 con 0x55d9059e8910 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f324c36d740 10 client.0 _ll_register_callbacks cb 0x55d905658a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e2db0 0x55d9059ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e2db0 0x55d9059ed0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34964/0 (socket says 172.21.15.112:34964) 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f32427fc640 1 -- 172.21.15.112:0/799828230 learned_addr learned my addr 172.21.15.112:0/799828230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f32427fc640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e8910 0x55d9059f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.015+0000 7f3242ffd640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f32427fc640 1 -- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e8910 msgr2=0x55d9059f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f32427fc640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e8910 0x55d9059f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f32427fc640 1 -- 172.21.15.112:0/799828230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 msgr2=0x55d9059ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f32427fc640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059ed5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f32427fc640 1 -- 172.21.15.112:0/799828230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d905725620 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f3242ffd640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f32427fc640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e2db0 0x55d9059ed0b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3234011660 tx=0x7f32340115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65f50c91de79112 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f322a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f322a7fc640 1 -- 172.21.15.112:0/799828230 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234004d20 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f322a7fc640 1 -- 172.21.15.112:0/799828230 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f323400f790 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3234005020 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f322a7fc640 1 -- 172.21.15.112:0/799828230 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3234034050 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 10 client.15432 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9059f7610 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f322a7fc640 1 -- 172.21.15.112:0/799828230 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3234005020 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f322a7fc640 1 -- 172.21.15.112:0/799828230 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f323401f890 con 0x55d9059e2db0 2024-05-10T10:40:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 10 client.15432 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 20 client.15432 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059f8a80 0x55d9059fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.016+0000 7f324c36d740 4 client.15432 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.017+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9059551e0 con 0x55d9059f8a80 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.017+0000 7f3242ffd640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059f8a80 0x55d9059fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.017+0000 7f322a7fc640 10 client.15432 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.017+0000 7f3242ffd640 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059f8a80 0x55d9059fae70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e453c4c2ddef451 server_cookie=241ca77719faf5bd in_seq=0 out_seq=0 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f322a7fc640 1 -- 172.21.15.112:0/799828230 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d9059551e0 con 0x55d9059f8a80 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f322a7fc640 10 client.15432 handle_command_reply: tid=0 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 client.15432 shutdown 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e2db0 msgr2=0x55d9059ed0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e2db0 0x55d9059ed0b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3234011660 tx=0x7f32340115f0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 shutdown_connections 2024-05-10T10:40:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e8910 0x55d9059f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059f8a80 0x55d9059fae70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e37a0 0x55d9059ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 --2- 172.21.15.112:0/799828230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e2db0 0x55d9059ed0b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f322a7fc640 0 client.15432 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 >> 172.21.15.112:0/799828230 conn(0x55d9059de420 msgr2=0x55d9059dd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 shutdown_connections 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 1 -- 172.21.15.112:0/799828230 wait complete. 2024-05-10T10:40:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.018+0000 7f324c36d740 20 client.15432 trim_cache size 0 max 16384 2024-05-10T10:40:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.033+0000 7f324c36d740 1 Processor -- start 2024-05-10T10:40:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.033+0000 7f324c36d740 1 -- start start 2024-05-10T10:40:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.033+0000 7f324c36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059de420 0x55d9059e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.033+0000 7f324c36d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e2db0 0x55d9059eccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.033+0000 7f324c36d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e37a0 0x55d9059ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f324c36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ad5d0 con 0x55d9059de420 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f324c36d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32441af110 con 0x55d9059e2db0 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f324c36d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f32441adca0 con 0x55d9059e37a0 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059de420 0x55d9059e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059de420 0x55d9059e1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34970/0 (socket says 172.21.15.112:34970) 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 -- 172.21.15.112:0/3499078168 learned_addr learned my addr 172.21.15.112:0/3499078168 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f3242ffd640 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e2db0 0x55d9059eccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 -- 172.21.15.112:0/3499078168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e37a0 msgr2=0x55d9059ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e37a0 0x55d9059ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 -- 172.21.15.112:0/3499078168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e2db0 msgr2=0x55d9059eccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e2db0 0x55d9059eccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 -- 172.21.15.112:0/3499078168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9059fb1e0 con 0x55d9059de420 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f3242ffd640 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e2db0 0x55d9059eccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f32427fc640 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059de420 0x55d9059e1a40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3234009800 tx=0x7f32340056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5a6aaaf4bc1e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f3238ff9640 1 -- 172.21.15.112:0/3499078168 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234009c90 con 0x55d9059de420 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f3238ff9640 1 -- 172.21.15.112:0/3499078168 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3234004750 con 0x55d9059de420 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.034+0000 7f3238ff9640 1 -- 172.21.15.112:0/3499078168 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323401f6a0 con 0x55d9059de420 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 -- 172.21.15.112:0/3499078168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059de420 msgr2=0x55d9059e1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059de420 0x55d9059e1a40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3234009800 tx=0x7f32340056a0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 -- 172.21.15.112:0/3499078168 shutdown_connections 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e37a0 0x55d9059ed200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059e2db0 0x55d9059eccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 --2- 172.21.15.112:0/3499078168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059de420 0x55d9059e1a40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 -- 172.21.15.112:0/3499078168 >> 172.21.15.112:0/3499078168 conn(0x55d9059e83e0 msgr2=0x55d9059e1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 -- 172.21.15.112:0/3499078168 shutdown_connections 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 -- 172.21.15.112:0/3499078168 wait complete. 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 -- start start 2024-05-10T10:40:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.035+0000 7f324c36d740 1 Processor -- start 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059de420 0x55d9059f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e37a0 0x55d9059f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32441aea40 con 0x55d9059e37a0 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ae370 con 0x55d9059de420 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ab5b0 con 0x55d9059e2db0 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 10 client.0 _ll_register_callbacks cb 0x7f322c00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e37a0 0x55d9059f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e37a0 0x55d9059f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34974/0 (socket says 172.21.15.112:34974) 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 -- 172.21.15.112:0/1014479930 learned_addr learned my addr 172.21.15.112:0/1014479930 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f3242ffd640 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059de420 0x55d9059f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 -- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 msgr2=0x55d9059f1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059f1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 -- 172.21.15.112:0/1014479930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059de420 msgr2=0x55d9059f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059de420 0x55d9059f12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 -- 172.21.15.112:0/1014479930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9059f2da0 con 0x55d9059e37a0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f32427fc640 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e37a0 0x55d9059f27b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3234030040 tx=0x7f3234009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d599bcc76c3bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f322a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f322a7fc640 1 -- 172.21.15.112:0/1014479930 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234041040 con 0x55d9059e37a0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f322a7fc640 1 -- 172.21.15.112:0/1014479930 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3234009c90 con 0x55d9059e37a0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.036+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d90594f050 con 0x55d9059e37a0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f322a7fc640 1 -- 172.21.15.112:0/1014479930 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f323401f6d0 con 0x55d9059e37a0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 10 client.15438 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9059f4020 con 0x55d9059e37a0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f322a7fc640 1 -- 172.21.15.112:0/1014479930 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d90594f050 con 0x55d9059e37a0 2024-05-10T10:40:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f322a7fc640 1 -- 172.21.15.112:0/1014479930 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f323402e790 con 0x55d9059e37a0 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 10 client.15438 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 20 client.15438 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059e8910 0x55d905a65a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 4 client.15438 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9058ca5f0 con 0x55d9059e8910 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.037+0000 7f3242ffd640 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059e8910 0x55d905a65a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.038+0000 7f322a7fc640 10 client.15438 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.038+0000 7f3242ffd640 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059e8910 0x55d905a65a00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7df724d7e18834b server_cookie=d379129f158a285e in_seq=0 out_seq=0 2024-05-10T10:40:45.048 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:44 smithi148 ceph-mon[33568]: pgmap v445: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f322a7fc640 1 -- 172.21.15.112:0/1014479930 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9058ca5f0 con 0x55d9059e8910 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f322a7fc640 10 client.15438 handle_command_reply: tid=0 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 client.15438 shutdown 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e37a0 msgr2=0x55d9059f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e37a0 0x55d9059f27b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3234030040 tx=0x7f3234009870 comp rx=0 tx=0).stop 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 shutdown_connections 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9059e37a0 0x55d9059f27b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9059e2db0 0x55d9059f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d9059e8910 0x55d905a65a00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 --2- 172.21.15.112:0/1014479930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9059de420 0x55d9059f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f322a7fc640 0 client.15438 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 >> 172.21.15.112:0/1014479930 conn(0x55d9059e83e0 msgr2=0x55d9059e1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 shutdown_connections 2024-05-10T10:40:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 1 -- 172.21.15.112:0/1014479930 wait complete. 2024-05-10T10:40:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.044+0000 7f324c36d740 20 client.15438 trim_cache size 0 max 16384 2024-05-10T10:40:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:40:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:40:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:40:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:40:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32300b28c0 msgr2=0x7f32300b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32300b28c0 0x7f32300b4d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f323c0caca0 tx=0x7f323c004970 comp rx=0 tx=0).stop 2024-05-10T10:40:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 msgr2=0x7f32441a20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f32441a20f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f323c02aeb0 tx=0x7f323c003c20 comp rx=0 tx=0).stop 2024-05-10T10:40:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 shutdown_connections 2024-05-10T10:40:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f32300b28c0 0x7f32300b4d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32441034c0 0x7f32441a20f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3244102b00 0x7f324419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 --2- 172.21.15.112:0/916744748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244101910 0x7f324406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 >> 172.21.15.112:0/916744748 conn(0x7f32440fcea0 msgr2=0x7f3244105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 shutdown_connections 2024-05-10T10:40:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:45.045+0000 7f3249578640 1 -- 172.21.15.112:0/916744748 wait complete. 2024-05-10T10:40:45.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:44 smithi195 ceph-mon[33773]: pgmap v445: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-10T10:40:45.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:44 smithi112 ceph-mon[30100]: pgmap v445: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-10T10:40:47.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:46 smithi148 ceph-mon[33568]: pgmap v446: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-10T10:40:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:46 smithi195 ceph-mon[33773]: pgmap v446: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-10T10:40:47.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:46 smithi112 ceph-mon[30100]: pgmap v446: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-10T10:40:49.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:48 smithi148 ceph-mon[33568]: pgmap v447: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T10:40:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:48 smithi195 ceph-mon[33773]: pgmap v447: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T10:40:49.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:48 smithi112 ceph-mon[30100]: pgmap v447: 129 pgs: 129 active+clean; 669 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T10:40:51.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:50 smithi148 ceph-mon[33568]: pgmap v448: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:40:51.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:50 smithi195 ceph-mon[33773]: pgmap v448: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:40:51.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:50 smithi112 ceph-mon[30100]: pgmap v448: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:40:53.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:52 smithi148 ceph-mon[33568]: pgmap v449: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:40:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:52 smithi195 ceph-mon[33773]: pgmap v449: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:40:53.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:52 smithi112 ceph-mon[30100]: pgmap v449: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:40:54.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:40:54.701 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:54.823 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:54 smithi112 ceph-mon[30100]: pgmap v450: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:40:55.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:54 smithi148 ceph-mon[33568]: pgmap v450: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:40:55.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:54 smithi195 ceph-mon[33773]: pgmap v450: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 -- 172.21.15.112:0/2639579472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928104100 msgr2=0x7f1928108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 --2- 172.21.15.112:0/2639579472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928104100 0x7f1928108120 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f19180099c0 tx=0x7f1918009cd0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 -- 172.21.15.112:0/2639579472 shutdown_connections 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 --2- 172.21.15.112:0/2639579472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1928108850 0x7f19281103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 --2- 172.21.15.112:0/2639579472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1928103750 0x7f1928103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 --2- 172.21.15.112:0/2639579472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928104100 0x7f1928108120 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 -- 172.21.15.112:0/2639579472 >> 172.21.15.112:0/2639579472 conn(0x7f192807a600 msgr2=0x7f192807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 -- 172.21.15.112:0/2639579472 shutdown_connections 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.607+0000 7f193053c700 1 -- 172.21.15.112:0/2639579472 wait complete. 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 Processor -- start 2024-05-10T10:40:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 -- start start 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1928103750 0x7f192819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1928104100 0x7f192819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928108850 0x7f1928199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19281135c0 con 0x7f1928108850 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1928113440 con 0x7f1928104100 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f193053c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1928113740 con 0x7f1928103750 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928108850 0x7f1928199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928108850 0x7f1928199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41122/0 (socket says 172.21.15.112:41122) 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 -- 172.21.15.112:0/189616675 learned_addr learned my addr 172.21.15.112:0/189616675 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192e2d8700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1928103750 0x7f192819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192dad7700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1928104100 0x7f192819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 -- 172.21.15.112:0/189616675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1928103750 msgr2=0x7f192819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1928103750 0x7f192819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 -- 172.21.15.112:0/189616675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1928104100 msgr2=0x7f192819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1928104100 0x7f192819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.608+0000 7f192ead9700 1 -- 172.21.15.112:0/189616675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192819a060 con 0x7f1928108850 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.609+0000 7f192ead9700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928108850 0x7f1928199900 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f192000b970 tx=0x7f192000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.609+0000 7f191f7fe700 1 -- 172.21.15.112:0/189616675 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920019070 con 0x7f1928108850 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.609+0000 7f191f7fe700 1 -- 172.21.15.112:0/189616675 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f192000cdc0 con 0x7f1928108850 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.609+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192819a350 con 0x7f1928108850 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.609+0000 7f191f7fe700 1 -- 172.21.15.112:0/189616675 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920014e30 con 0x7f1928108850 2024-05-10T10:40:56.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.609+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19281a6610 con 0x7f1928108850 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.611+0000 7f191f7fe700 1 -- 172.21.15.112:0/189616675 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f192001c070 con 0x7f1928108850 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.612+0000 7f191f7fe700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f19140b2900 0x7f19140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.612+0000 7f192e2d8700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f19140b2900 0x7f19140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.612+0000 7f191f7fe700 1 -- 172.21.15.112:0/189616675 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f19200e18e0 con 0x7f1928108850 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.612+0000 7f192e2d8700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f19140b2900 0x7f19140b4dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f192819aa40 tx=0x7f1924009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.612+0000 7f193adcfb80 1 Processor -- start 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f193adcfb80 1 -- start start 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2e1b0 0x559be2a2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a28810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a28d50 0x559be2a29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f193adcfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559be297a940 con 0x559be2a28d50 2024-05-10T10:40:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f193adcfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559be28dfda0 con 0x559be2a2ead0 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f193adcfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559be2976900 con 0x559be2a2e1b0 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a28d50 0x559be2a29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a28d50 0x559be2a29190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41136/0 (socket says 172.21.15.112:41136) 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 -- 172.21.15.112:0/1503922434 learned_addr learned my addr 172.21.15.112:0/1503922434 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192dad7700 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a28810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192e2d8700 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2e1b0 0x559be2a2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 -- 172.21.15.112:0/1503922434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2e1b0 msgr2=0x559be2a2e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2e1b0 0x559be2a2e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 -- 172.21.15.112:0/1503922434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 msgr2=0x559be2a28810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a28810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.613+0000 7f192ead9700 1 -- 172.21.15.112:0/1503922434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be28c78d0 con 0x559be2a28d50 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.614+0000 7f192e2d8700 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2e1b0 0x559be2a2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.614+0000 7f192ead9700 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a28d50 0x559be2a29190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1920016000 tx=0x7f19200a8360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96c16b608e111b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:56.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.614+0000 7f191cff9700 1 -- 172.21.15.112:0/1503922434 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19200a9270 con 0x559be2a28d50 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.614+0000 7f191cff9700 1 -- 172.21.15.112:0/1503922434 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19200a8b70 con 0x559be2a28d50 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.614+0000 7f191cff9700 1 -- 172.21.15.112:0/1503922434 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19200b1540 con 0x559be2a28d50 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- 172.21.15.112:0/1503922434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a28d50 msgr2=0x559be2a29190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a28d50 0x559be2a29190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1920016000 tx=0x7f19200a8360 comp rx=0 tx=0).stop 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- 172.21.15.112:0/1503922434 shutdown_connections 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a28d50 0x559be2a29190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2e1b0 0x559be2a2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1503922434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a28810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- 172.21.15.112:0/1503922434 >> 172.21.15.112:0/1503922434 conn(0x559be2954b70 msgr2=0x559be2a232a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- 172.21.15.112:0/1503922434 shutdown_connections 2024-05-10T10:40:56.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- 172.21.15.112:0/1503922434 wait complete. 2024-05-10T10:40:56.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- start start 2024-05-10T10:40:56.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 Processor -- start 2024-05-10T10:40:56.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559be2a35450 con 0x559be2a2e1b0 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559be2933ed0 con 0x559be2a2ead0 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559be2a355d0 con 0x559be2a28d50 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.615+0000 7f193adcfb80 10 client.0 _ll_register_callbacks cb 0x559be290e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a31ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41140/0 (socket says 172.21.15.112:41140) 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 -- 172.21.15.112:0/3694604063 learned_addr learned my addr 172.21.15.112:0/3694604063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192e2d8700 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192ead9700 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 -- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 msgr2=0x559be2a34800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a34800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 -- 172.21.15.112:0/3694604063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 msgr2=0x559be2a320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 -- 172.21.15.112:0/3694604063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be2931760 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f192dad7700 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a31ba0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1918009f50 tx=0x7f1918005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b01479cf6056e54 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f1911ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f1911ffb700 1 -- 172.21.15.112:0/3694604063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19180054e0 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f1911ffb700 1 -- 172.21.15.112:0/3694604063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1918005680 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19200a8b70 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f1911ffb700 1 -- 172.21.15.112:0/3694604063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1918030050 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f193adcfb80 10 client.15450 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.616+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559be2a3fde0 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f1911ffb700 1 -- 172.21.15.112:0/3694604063 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19200a8b70 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f1911ffb700 1 -- 172.21.15.112:0/3694604063 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f191802b030 con 0x559be2a2e1b0 2024-05-10T10:40:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f193adcfb80 10 client.15450 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f193adcfb80 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f193adcfb80 20 client.15450 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f193adcfb80 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2a42470 0x559be2a44860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f193adcfb80 4 client.15450 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559be2988320 con 0x559be2a42470 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f192e2d8700 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2a42470 0x559be2a44860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f192e2d8700 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2a42470 0x559be2a44860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7065cb281463ed19 server_cookie=41dd28b4b5c36bd5 in_seq=0 out_seq=0 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.617+0000 7f1911ffb700 10 client.15450 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f1911ffb700 1 -- 172.21.15.112:0/3694604063 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559be2988320 con 0x559be2a42470 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f1911ffb700 10 client.15450 handle_command_reply: tid=0 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 client.15450 shutdown 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 msgr2=0x559be2a31ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a31ba0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1918009f50 tx=0x7f1918005ab0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 shutdown_connections 2024-05-10T10:40:56.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2a42470 0x559be2a44860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2ead0 0x559be2a320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f1911ffb700 0 client.15450 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 --2- 172.21.15.112:0/3694604063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a31ba0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 >> 172.21.15.112:0/3694604063 conn(0x559be2954b70 msgr2=0x559be2a1db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.618+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 shutdown_connections 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.619+0000 7f193adcfb80 1 -- 172.21.15.112:0/3694604063 wait complete. 2024-05-10T10:40:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.619+0000 7f193adcfb80 20 client.15450 trim_cache size 0 max 16384 2024-05-10T10:40:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 Processor -- start 2024-05-10T10:40:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 -- start start 2024-05-10T10:40:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a28810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2e1b0 0x559be2a3e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2ead0 0x559be2a1dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19281af180 con 0x559be2a2ead0 2024-05-10T10:40:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f19281b0cc0 con 0x559be2a2e1b0 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f193adcfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f19281af850 con 0x559be2a28d50 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2ead0 0x559be2a1dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2ead0 0x559be2a1dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41152/0 (socket says 172.21.15.112:41152) 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 -- 172.21.15.112:0/2971628938 learned_addr learned my addr 172.21.15.112:0/2971628938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192e2d8700 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a28810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192dad7700 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2e1b0 0x559be2a3e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 -- 172.21.15.112:0/2971628938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 msgr2=0x559be2a28810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a28810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 -- 172.21.15.112:0/2971628938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2e1b0 msgr2=0x559be2a3e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2e1b0 0x559be2a3e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 -- 172.21.15.112:0/2971628938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be2a2c640 con 0x559be2a2ead0 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192dad7700 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2e1b0 0x559be2a3e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192e2d8700 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a28810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.639+0000 7f192ead9700 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2ead0 0x559be2a1dc40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f19200df3c0 tx=0x7f192000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58048ccd455f3cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f1913fff700 1 -- 172.21.15.112:0/2971628938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19200a8b70 con 0x559be2a2ead0 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f1913fff700 1 -- 172.21.15.112:0/2971628938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19200acf20 con 0x559be2a2ead0 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f1913fff700 1 -- 172.21.15.112:0/2971628938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19200a8dd0 con 0x559be2a2ead0 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 -- 172.21.15.112:0/2971628938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2ead0 msgr2=0x559be2a1dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2ead0 0x559be2a1dc40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f19200df3c0 tx=0x7f192000e040 comp rx=0 tx=0).stop 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 -- 172.21.15.112:0/2971628938 shutdown_connections 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2ead0 0x559be2a1dc40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a28d50 0x559be2a28810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 --2- 172.21.15.112:0/2971628938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a2e1b0 0x559be2a3e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 -- 172.21.15.112:0/2971628938 >> 172.21.15.112:0/2971628938 conn(0x559be2954b70 msgr2=0x559be2a31ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 -- 172.21.15.112:0/2971628938 shutdown_connections 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 -- 172.21.15.112:0/2971628938 wait complete. 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.640+0000 7f193adcfb80 1 -- start start 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 1 Processor -- start 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a28d50 0x559be2a3a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2ead0 0x559be2a37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19281b05f0 con 0x559be2a2e1b0 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f19281aff20 con 0x559be2a28d50 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f19281ad120 con 0x559be2a2ead0 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f193adcfb80 10 client.0 _ll_register_callbacks cb 0x559be290e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a3ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41160/0 (socket says 172.21.15.112:41160) 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 -- 172.21.15.112:0/1209247851 learned_addr learned my addr 172.21.15.112:0/1209247851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:56.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192ead9700 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2ead0 0x559be2a37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192e2d8700 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a28d50 0x559be2a3a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 -- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2ead0 msgr2=0x559be2a37830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2ead0 0x559be2a37830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 -- 172.21.15.112:0/1209247851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a28d50 msgr2=0x559be2a3a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a28d50 0x559be2a3a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 -- 172.21.15.112:0/1209247851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be2a37d70 con 0x559be2a2e1b0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f192dad7700 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a3ada0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f191800bf50 tx=0x7f1918009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a4817defd14046 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.641+0000 7f1911ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f1911ffb700 1 -- 172.21.15.112:0/1209247851 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918038060 con 0x559be2a2e1b0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f1911ffb700 1 -- 172.21.15.112:0/1209247851 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f19180052e0 con 0x559be2a2e1b0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559be29823c0 con 0x559be2a2e1b0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f1911ffb700 1 -- 172.21.15.112:0/1209247851 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f191803f050 con 0x559be2a2e1b0 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 10 client.15456 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559be2ab1770 con 0x559be2a2e1b0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f1911ffb700 1 -- 172.21.15.112:0/1209247851 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559be29823c0 con 0x559be2a2e1b0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f1911ffb700 1 -- 172.21.15.112:0/1209247851 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f191802b040 con 0x559be2a2e1b0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 10 client.15456 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 20 client.15456 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2ab1df0 0x559be2ab41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 4 client.15456 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559be27fa250 con 0x559be2ab1df0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.642+0000 7f192e2d8700 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2ab1df0 0x559be2ab41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f1911ffb700 10 client.15456 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f192e2d8700 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2ab1df0 0x559be2ab41e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb18e6afb3424084 server_cookie=680c0259b8ae2d57 in_seq=0 out_seq=0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f1911ffb700 1 -- 172.21.15.112:0/1209247851 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559be27fa250 con 0x559be2ab1df0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f1911ffb700 10 client.15456 handle_command_reply: tid=0 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 client.15456 shutdown 2024-05-10T10:40:56.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 msgr2=0x559be2a3ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a3ada0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f191800bf50 tx=0x7f1918009300 comp rx=0 tx=0).stop 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 shutdown_connections 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559be2ab1df0 0x559be2ab41e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559be2a2ead0 0x559be2a37830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559be2a28d50 0x559be2a3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 --2- 172.21.15.112:0/1209247851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559be2a2e1b0 0x559be2a3ada0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 >> 172.21.15.112:0/1209247851 conn(0x559be2954b70 msgr2=0x559be2a2cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f1911ffb700 0 client.15456 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 shutdown_connections 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 1 -- 172.21.15.112:0/1209247851 wait complete. 2024-05-10T10:40:56.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.643+0000 7f193adcfb80 20 client.15456 trim_cache size 0 max 16384 2024-05-10T10:40:56.654 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:40:56.654 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:40:56.654 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:40:56.654 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:40:56.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f19140b2900 msgr2=0x7f19140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f19140b2900 0x7f19140b4dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f192819aa40 tx=0x7f1924009190 comp rx=0 tx=0).stop 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928108850 msgr2=0x7f1928199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928108850 0x7f1928199900 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f192000b970 tx=0x7f192000bc80 comp rx=0 tx=0).stop 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 shutdown_connections 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1928108850 0x7f1928199900 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f19140b2900 0x7f19140b4dc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1928103750 0x7f192819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 --2- 172.21.15.112:0/189616675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1928104100 0x7f192819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 >> 172.21.15.112:0/189616675 conn(0x7f192807a600 msgr2=0x7f1928101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 shutdown_connections 2024-05-10T10:40:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:56.644+0000 7f193053c700 1 -- 172.21.15.112:0/189616675 wait complete. 2024-05-10T10:40:56.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:56 smithi112 ceph-mon[30100]: pgmap v451: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-10T10:40:57.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:56 smithi148 ceph-mon[33568]: pgmap v451: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-10T10:40:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:56 smithi195 ceph-mon[33773]: pgmap v451: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-10T10:40:57.410 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:40:57.410 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:40:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:40:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 -- 172.21.15.112:0/65800274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820103570 msgr2=0x7f8820109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 --2- 172.21.15.112:0/65800274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820103570 0x7f8820109e00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8818009900 tx=0x7f881802a410 comp rx=0 tx=0).stop 2024-05-10T10:40:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 -- 172.21.15.112:0/65800274 shutdown_connections 2024-05-10T10:40:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 --2- 172.21.15.112:0/65800274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820103570 0x7f8820109e00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 --2- 172.21.15.112:0/65800274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8820102bb0 0x7f8820103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 --2- 172.21.15.112:0/65800274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88201019c0 0x7f8820101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 -- 172.21.15.112:0/65800274 >> 172.21.15.112:0/65800274 conn(0x7f88200fcf30 msgr2=0x7f88200ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 -- 172.21.15.112:0/65800274 shutdown_connections 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.134+0000 7f8828787640 1 -- 172.21.15.112:0/65800274 wait complete. 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 Processor -- start 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 -- start start 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88201019c0 0x7f882006bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820102bb0 0x7f882019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8820103570 0x7f88201a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f882010d330 con 0x7f8820102bb0 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f882010d1b0 con 0x7f88201019c0 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8828787640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f882010d4b0 con 0x7f8820103570 2024-05-10T10:40:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820102bb0 0x7f882019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820102bb0 0x7f882019fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41192/0 (socket says 172.21.15.112:41192) 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 -- 172.21.15.112:0/3739203828 learned_addr learned my addr 172.21.15.112:0/3739203828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8826523640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88201019c0 0x7f882006bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8826d24640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8820103570 0x7f88201a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 -- 172.21.15.112:0/3739203828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8820103570 msgr2=0x7f88201a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8820103570 0x7f88201a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 -- 172.21.15.112:0/3739203828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88201019c0 msgr2=0x7f882006bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88201019c0 0x7f882006bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.135+0000 7f8825d22640 1 -- 172.21.15.112:0/3739203828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88201a2870 con 0x7f8820102bb0 2024-05-10T10:40:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.136+0000 7f8825d22640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820102bb0 0x7f882019fb30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f881400c970 tx=0x7f881400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.136+0000 7f881f7fe640 1 -- 172.21.15.112:0/3739203828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814007bb0 con 0x7f8820102bb0 2024-05-10T10:40:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.136+0000 7f881f7fe640 1 -- 172.21.15.112:0/3739203828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8814007d50 con 0x7f8820102bb0 2024-05-10T10:40:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.136+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88201a2b00 con 0x7f8820102bb0 2024-05-10T10:40:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.136+0000 7f881f7fe640 1 -- 172.21.15.112:0/3739203828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814004e40 con 0x7f8820102bb0 2024-05-10T10:40:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.136+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88201a3050 con 0x7f8820102bb0 2024-05-10T10:40:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 Processor -- start 2024-05-10T10:40:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 -- start start 2024-05-10T10:40:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f881f7fe640 1 -- 172.21.15.112:0/3739203828 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8814005020 con 0x7f8820102bb0 2024-05-10T10:40:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd1ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0cdc775d0 con 0x55a0cdd137a0 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a0cdc30890 con 0x55a0cdd18910 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f882b577740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a0cdc73590 con 0x55a0cdd12db0 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f8826523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f8826523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd182a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41202/0 (socket says 172.21.15.112:41202) 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f8826523640 1 -- 172.21.15.112:0/3855814412 learned_addr learned my addr 172.21.15.112:0/3855814412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f8826d24640 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.139+0000 7f8825d22640 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd1ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f8826523640 1 -- 172.21.15.112:0/3855814412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 msgr2=0x55a0cdd131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f8826523640 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f8826523640 1 -- 172.21.15.112:0/3855814412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 msgr2=0x55a0cdd1ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f8826523640 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd1ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f8826523640 1 -- 172.21.15.112:0/3855814412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cd989210 con 0x55a0cdd137a0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f8826523640 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd182a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8810007f50 tx=0x7f881000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f548443ee3af77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f881cff9640 1 -- 172.21.15.112:0/3855814412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810003b70 con 0x55a0cdd137a0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f881cff9640 1 -- 172.21.15.112:0/3855814412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8810003d10 con 0x55a0cdd137a0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f881cff9640 1 -- 172.21.15.112:0/3855814412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810010ac0 con 0x55a0cdd137a0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f881f7fe640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2940 0x7f88080b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f8826d24640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2940 0x7f88080b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.140+0000 7f881f7fe640 1 -- 172.21.15.112:0/3739203828 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f88140e2250 con 0x7f8820102bb0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- 172.21.15.112:0/3855814412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 msgr2=0x55a0cdd182a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f8826d24640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2940 0x7f88080b4e00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a0cdd14000 tx=0x7f8818031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd182a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8810007f50 tx=0x7f881000f850 comp rx=0 tx=0).stop 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- 172.21.15.112:0/3855814412 shutdown_connections 2024-05-10T10:40:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd182a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 --2- 172.21.15.112:0/3855814412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- 172.21.15.112:0/3855814412 >> 172.21.15.112:0/3855814412 conn(0x55a0cdd0e420 msgr2=0x55a0cdd0e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- 172.21.15.112:0/3855814412 shutdown_connections 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- 172.21.15.112:0/3855814412 wait complete. 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- start start 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 Processor -- start 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd12db0 0x55a0cdd1cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd137a0 0x55a0cdd1d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0cdd1b8c0 con 0x55a0cdd12db0 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a0cdc34370 con 0x55a0cdd18910 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a0cdd1ba40 con 0x55a0cdd137a0 2024-05-10T10:40:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f882b577740 10 client.0 _ll_register_callbacks cb 0x55a0cd988a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f8826523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd12db0 0x55a0cdd1cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f8826523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd12db0 0x55a0cdd1cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41208/0 (socket says 172.21.15.112:41208) 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.141+0000 7f8826523640 1 -- 172.21.15.112:0/2321625906 learned_addr learned my addr 172.21.15.112:0/2321625906 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826d24640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8825d22640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd137a0 0x55a0cdd1d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826523640 1 -- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd137a0 msgr2=0x55a0cdd1d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826523640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd137a0 0x55a0cdd1d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826523640 1 -- 172.21.15.112:0/2321625906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 msgr2=0x55a0cdd24510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826523640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd24510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826523640 1 -- 172.21.15.112:0/2321625906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cda55620 con 0x55a0cdd12db0 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826d24640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8826523640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd12db0 0x55a0cdd1cf90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8810000f80 tx=0x7f8810007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be75b5c94a0f794d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8805ffb640 1 -- 172.21.15.112:0/2321625906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810026020 con 0x55a0cdd12db0 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8805ffb640 1 -- 172.21.15.112:0/2321625906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8810003ec0 con 0x55a0cdd12db0 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8810003d10 con 0x55a0cdd12db0 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8805ffb640 1 -- 172.21.15.112:0/2321625906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8810031050 con 0x55a0cdd12db0 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f882b577740 10 client.15468 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0cdd274f0 con 0x55a0cdd12db0 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.142+0000 7f8805ffb640 1 -- 172.21.15.112:0/2321625906 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8810003d10 con 0x55a0cdd12db0 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f8805ffb640 1 -- 172.21.15.112:0/2321625906 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f88100226d0 con 0x55a0cdd12db0 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f882b577740 10 client.15468 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f882b577740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f882b577740 20 client.15468 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f882b577740 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd288f0 0x55a0cdd2ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f882b577740 4 client.15468 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0cdc851e0 con 0x55a0cdd288f0 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f8825d22640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd288f0 0x55a0cdd2ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f8825d22640 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd288f0 0x55a0cdd2ace0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744e33c02317f50b server_cookie=c2261f67211a2069 in_seq=0 out_seq=0 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.143+0000 7f8805ffb640 10 client.15468 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f8805ffb640 1 -- 172.21.15.112:0/2321625906 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0cdc851e0 con 0x55a0cdd288f0 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f8805ffb640 10 client.15468 handle_command_reply: tid=0 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 client.15468 shutdown 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd12db0 msgr2=0x55a0cdd1cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd12db0 0x55a0cdd1cf90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8810000f80 tx=0x7f8810007cd0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 shutdown_connections 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd288f0 0x55a0cdd2ace0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f8805ffb640 0 client.15468 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd137a0 0x55a0cdd1d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 --2- 172.21.15.112:0/2321625906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd12db0 0x55a0cdd1cf90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 >> 172.21.15.112:0/2321625906 conn(0x55a0cdd0e420 msgr2=0x55a0cdd0d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 shutdown_connections 2024-05-10T10:40:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 1 -- 172.21.15.112:0/2321625906 wait complete. 2024-05-10T10:40:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.144+0000 7f882b577740 20 client.15468 trim_cache size 0 max 16384 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 Processor -- start 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 -- start start 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd0d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd0d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88201ad540 con 0x55a0cdd137a0 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f88201af080 con 0x55a0cdd18910 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f882b577740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88201adc10 con 0x55a0cdd12db0 2024-05-10T10:40:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f8825d22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd0d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f8825d22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd0d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41218/0 (socket says 172.21.15.112:41218) 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.159+0000 7f8825d22640 1 -- 172.21.15.112:0/875759687 learned_addr learned my addr 172.21.15.112:0/875759687 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8826523640 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd0d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8825d22640 1 -- 172.21.15.112:0/875759687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 msgr2=0x55a0cdd0d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8825d22640 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd0d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8825d22640 1 -- 172.21.15.112:0/875759687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 msgr2=0x55a0cdd244d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8825d22640 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8825d22640 1 -- 172.21.15.112:0/875759687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cdd2b010 con 0x55a0cdd137a0 2024-05-10T10:40:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8825d22640 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd0d940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f88140df9c0 tx=0x7f88140e5a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3266bb8723f1d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8807fff640 1 -- 172.21.15.112:0/875759687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881401d040 con 0x55a0cdd137a0 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8807fff640 1 -- 172.21.15.112:0/875759687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88140a9be0 con 0x55a0cdd137a0 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f8807fff640 1 -- 172.21.15.112:0/875759687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814005310 con 0x55a0cdd137a0 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f882b577740 1 -- 172.21.15.112:0/875759687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 msgr2=0x55a0cdd0d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f882b577740 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd0d940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f88140df9c0 tx=0x7f88140e5a60 comp rx=0 tx=0).stop 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f882b577740 1 -- 172.21.15.112:0/875759687 shutdown_connections 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f882b577740 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd18910 0x55a0cdd244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f882b577740 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd0d940 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f882b577740 1 --2- 172.21.15.112:0/875759687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd12db0 0x55a0cdd0d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.160+0000 7f882b577740 1 -- 172.21.15.112:0/875759687 >> 172.21.15.112:0/875759687 conn(0x55a0cdd0e420 msgr2=0x55a0cdd0ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 -- 172.21.15.112:0/875759687 shutdown_connections 2024-05-10T10:40:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 -- 172.21.15.112:0/875759687 wait complete. 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 -- start start 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 Processor -- start 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd12db0 0x55a0cdd21dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd22310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd18910 0x55a0cdd23320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88201ae9b0 con 0x55a0cdd137a0 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f88201ae2e0 con 0x55a0cdd12db0 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88201ab520 con 0x55a0cdd18910 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f882b577740 10 client.0 _ll_register_callbacks cb 0x7f8810007dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd22310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd22310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41224/0 (socket says 172.21.15.112:41224) 2024-05-10T10:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 -- 172.21.15.112:0/4139112646 learned_addr learned my addr 172.21.15.112:0/4139112646 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8826d24640 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd18910 0x55a0cdd23320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 -- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd18910 msgr2=0x55a0cdd23320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd18910 0x55a0cdd23320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 -- 172.21.15.112:0/4139112646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd12db0 msgr2=0x55a0cdd21dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd12db0 0x55a0cdd21dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.161+0000 7f8825d22640 1 -- 172.21.15.112:0/4139112646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cdd23910 con 0x55a0cdd137a0 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8825d22640 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd22310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f88140dfa10 tx=0x7f88140ad890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b7fad60c8bb471 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:40:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8805ffb640 1 -- 172.21.15.112:0/4139112646 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881401d070 con 0x55a0cdd137a0 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8805ffb640 1 -- 172.21.15.112:0/4139112646 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f88140046c0 con 0x55a0cdd137a0 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88140a9be0 con 0x55a0cdd137a0 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8805ffb640 1 -- 172.21.15.112:0/4139112646 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8814005310 con 0x55a0cdd137a0 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 10 client.15474 fetch_fsmap learned FSMap version 30 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0cdd93330 con 0x55a0cdd137a0 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8805ffb640 1 -- 172.21.15.112:0/4139112646 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88140a9be0 con 0x55a0cdd137a0 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8805ffb640 1 -- 172.21.15.112:0/4139112646 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f88140a4e50 con 0x55a0cdd137a0 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 10 client.15474 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 20 client.15474 populate_metadata read hostname 'smithi112' 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd93ca0 0x55a0cdd960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 4 client.15474 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:40:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0cdc7f050 con 0x55a0cdd93ca0 2024-05-10T10:40:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.162+0000 7f8826523640 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd93ca0 0x55a0cdd960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.163+0000 7f8826523640 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd93ca0 0x55a0cdd960c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dbd4302f2025536 server_cookie=385462c9f6f2a978 in_seq=0 out_seq=0 2024-05-10T10:40:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.163+0000 7f8805ffb640 10 client.15474 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f8805ffb640 1 -- 172.21.15.112:0/4139112646 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0cdc7f050 con 0x55a0cdd93ca0 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f8805ffb640 10 client.15474 handle_command_reply: tid=0 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 client.15474 shutdown 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 msgr2=0x55a0cdd22310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd22310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f88140dfa10 tx=0x7f88140ad890 comp rx=0 tx=0).stop 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 shutdown_connections 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a0cdd18910 0x55a0cdd23320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0cdd137a0 0x55a0cdd22310 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a0cdd93ca0 0x55a0cdd960c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 --2- 172.21.15.112:0/4139112646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0cdd12db0 0x55a0cdd21dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f8805ffb640 0 client.15474 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 >> 172.21.15.112:0/4139112646 conn(0x55a0cdd0e420 msgr2=0x55a0cdd0d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 shutdown_connections 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 1 -- 172.21.15.112:0/4139112646 wait complete. 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f882b577740 20 client.15474 trim_cache size 0 max 16384 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:40:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:40:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:40:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2940 msgr2=0x7f88080b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f8828787640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2940 0x7f88080b4e00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a0cdd14000 tx=0x7f8818031400 comp rx=0 tx=0).stop 2024-05-10T10:40:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820102bb0 msgr2=0x7f882019fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.167+0000 7f8828787640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820102bb0 0x7f882019fb30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f881400c970 tx=0x7f881400ce30 comp rx=0 tx=0).stop 2024-05-10T10:40:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 shutdown_connections 2024-05-10T10:40:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2940 0x7f88080b4e00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8820103570 0x7f88201a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820102bb0 0x7f882019fb30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 --2- 172.21.15.112:0/3739203828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88201019c0 0x7f882006bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 >> 172.21.15.112:0/3739203828 conn(0x7f88200fcf30 msgr2=0x7f8820105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 shutdown_connections 2024-05-10T10:40:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:40:58.168+0000 7f8828787640 1 -- 172.21.15.112:0/3739203828 wait complete. 2024-05-10T10:40:58.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:40:58 smithi112 ceph-mon[30100]: pgmap v452: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-10T10:40:59.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:40:58 smithi148 ceph-mon[33568]: pgmap v452: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-10T10:40:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:40:58 smithi195 ceph-mon[33773]: pgmap v452: 129 pgs: 129 active+clean; 670 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.505+0000 7f4067a91700 1 -- 172.21.15.112:0/568076316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4060102d70 msgr2=0x7f406010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.505+0000 7f4067a91700 1 --2- 172.21.15.112:0/568076316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4060102d70 0x7f406010f250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f405800b0a0 tx=0x7f405800b3b0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.505+0000 7f4067a91700 1 -- 172.21.15.112:0/568076316 shutdown_connections 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.505+0000 7f4067a91700 1 --2- 172.21.15.112:0/568076316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4060102d70 0x7f406010f250 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.505+0000 7f4067a91700 1 --2- 172.21.15.112:0/568076316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4060108410 0x7f40601087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.505+0000 7f4067a91700 1 --2- 172.21.15.112:0/568076316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40601023d0 0x7f4060102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.505+0000 7f4067a91700 1 -- 172.21.15.112:0/568076316 >> 172.21.15.112:0/568076316 conn(0x7f4060079340 msgr2=0x7f4060079750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 -- 172.21.15.112:0/568076316 shutdown_connections 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 -- 172.21.15.112:0/568076316 wait complete. 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 Processor -- start 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 -- start start 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40601023d0 0x7f406019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4060102d70 0x7f406019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4060108410 0x7f40601997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40601123c0 con 0x7f40601023d0 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4060112240 con 0x7f4060102d70 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.506+0000 7f4067a91700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4060112540 con 0x7f4060108410 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40601023d0 0x7f406019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406502c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4060102d70 0x7f406019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40601023d0 0x7f406019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41242/0 (socket says 172.21.15.112:41242) 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 -- 172.21.15.112:0/2205341567 learned_addr learned my addr 172.21.15.112:0/2205341567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:40:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406602e700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4060108410 0x7f40601997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 -- 172.21.15.112:0/2205341567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4060108410 msgr2=0x7f40601997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4060108410 0x7f40601997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 -- 172.21.15.112:0/2205341567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4060102d70 msgr2=0x7f406019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4060102d70 0x7f406019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 -- 172.21.15.112:0/2205341567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4060199fb0 con 0x7f40601023d0 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f406582d700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40601023d0 0x7f406019f650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d950 tx=0x7f405c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f4056ffd700 1 -- 172.21.15.112:0/2205341567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c016070 con 0x7f40601023d0 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406019a240 con 0x7f40601023d0 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f4056ffd700 1 -- 172.21.15.112:0/2205341567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f405c00fd20 con 0x7f40601023d0 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f4056ffd700 1 -- 172.21.15.112:0/2205341567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c00e3b0 con 0x7f40601023d0 2024-05-10T10:40:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.507+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40601a64b0 con 0x7f40601023d0 2024-05-10T10:40:59.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.508+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4060103ac0 con 0x7f40601023d0 2024-05-10T10:40:59.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.510+0000 7f4056ffd700 1 -- 172.21.15.112:0/2205341567 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f405c019070 con 0x7f40601023d0 2024-05-10T10:40:59.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.511+0000 7f4056ffd700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f404c0b2880 0x7f404c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.511+0000 7f406502c700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f404c0b2880 0x7f404c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.511+0000 7f4056ffd700 1 -- 172.21.15.112:0/2205341567 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f405c0e1fb0 con 0x7f40601023d0 2024-05-10T10:40:59.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.511+0000 7f406502c700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f404c0b2880 0x7f404c0b4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40600798d0 tx=0x7f405000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.516+0000 7f4056ffd700 1 -- 172.21.15.112:0/2205341567 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f405c0a9940 con 0x7f40601023d0 2024-05-10T10:40:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.770+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40601a66e0 con 0x7f40601023d0 2024-05-10T10:40:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.770+0000 7f4056ffd700 1 -- 172.21.15.112:0/2205341567 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f405c0a9240 con 0x7f40601023d0 2024-05-10T10:40:59.772 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:40:59.772 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:40:06.380474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f404c0b2880 msgr2=0x7f404c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f404c0b2880 0x7f404c0b4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40600798d0 tx=0x7f405000a380 comp rx=0 tx=0).stop 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40601023d0 msgr2=0x7f406019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40601023d0 0x7f406019f650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d950 tx=0x7f405c00dc60 comp rx=0 tx=0).stop 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 shutdown_connections 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f404c0b2880 0x7f404c0b4d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4060108410 0x7f40601997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40601023d0 0x7f406019f650 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 --2- 172.21.15.112:0/2205341567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4060102d70 0x7f406019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 >> 172.21.15.112:0/2205341567 conn(0x7f4060079340 msgr2=0x7f4060101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 shutdown_connections 2024-05-10T10:40:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:40:59.773+0000 7f4067a91700 1 -- 172.21.15.112:0/2205341567 wait complete. 2024-05-10T10:40:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-05-10T10:41:00.635 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:41:00.882 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:01.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:00 smithi112 ceph-mon[30100]: pgmap v453: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T10:41:01.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:00 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2205341567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:00 smithi148 ceph-mon[33568]: pgmap v453: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T10:41:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:00 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2205341567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:00 smithi195 ceph-mon[33773]: pgmap v453: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T10:41:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:00 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2205341567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 -- 172.21.15.112:0/323005930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c101da0 msgr2=0x7f7d9c102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 --2- 172.21.15.112:0/323005930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c101da0 0x7f7d9c102180 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0099c0 tx=0x7f7d8c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 -- 172.21.15.112:0/323005930 shutdown_connections 2024-05-10T10:41:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 --2- 172.21.15.112:0/323005930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c106e80 0x7f7d9c10ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 --2- 172.21.15.112:0/323005930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c101da0 0x7f7d9c102180 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 --2- 172.21.15.112:0/323005930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d9c102750 0x7f7d9c1067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 -- 172.21.15.112:0/323005930 >> 172.21.15.112:0/323005930 conn(0x7f7d9c0789a0 msgr2=0x7f7d9c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 -- 172.21.15.112:0/323005930 shutdown_connections 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.560+0000 7f7da2277700 1 -- 172.21.15.112:0/323005930 wait complete. 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 Processor -- start 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 -- start start 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c101da0 0x7f7d9c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d9c102750 0x7f7d9c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c106e80 0x7f7d9c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c111c40 con 0x7f7d9c106e80 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c111ac0 con 0x7f7d9c101da0 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da2277700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c111dc0 con 0x7f7d9c102750 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c106e80 0x7f7d9c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c106e80 0x7f7d9c1a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41252/0 (socket says 172.21.15.112:41252) 2024-05-10T10:41:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 -- 172.21.15.112:0/2689729991 learned_addr learned my addr 172.21.15.112:0/2689729991 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c101da0 0x7f7d9c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7d93fff700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d9c102750 0x7f7d9c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 -- 172.21.15.112:0/2689729991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d9c102750 msgr2=0x7f7d9c19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d9c102750 0x7f7d9c19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 -- 172.21.15.112:0/2689729991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c101da0 msgr2=0x7f7d9c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c101da0 0x7f7d9c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 -- 172.21.15.112:0/2689729991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c1a5f60 con 0x7f7d9c106e80 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c101da0 0x7f7d9c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.561+0000 7f7da0814700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c106e80 0x7f7d9c1a5a20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400ab50 tx=0x7f7d9400af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.562+0000 7f7d99ffb700 1 -- 172.21.15.112:0/2689729991 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94017020 con 0x7f7d9c106e80 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.562+0000 7f7d99ffb700 1 -- 172.21.15.112:0/2689729991 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d94004bb0 con 0x7f7d9c106e80 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.562+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c1a6190 con 0x7f7d9c106e80 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.562+0000 7f7d99ffb700 1 -- 172.21.15.112:0/2689729991 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94005280 con 0x7f7d9c106e80 2024-05-10T10:41:02.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.562+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c1a6650 con 0x7f7d9c106e80 2024-05-10T10:41:02.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.564+0000 7f7d99ffb700 1 -- 172.21.15.112:0/2689729991 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7d9400c6d0 con 0x7f7d9c106e80 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.565+0000 7f7d99ffb700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2880 0x7f7d880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.565+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2880 0x7f7d880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.565+0000 7f7d99ffb700 1 -- 172.21.15.112:0/2689729991 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7d940e1c80 con 0x7f7d9c106e80 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.565+0000 7f7dacb0ab80 1 Processor -- start 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.565+0000 7f7dacb0ab80 1 -- start start 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2880 0x7f7d880b4d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c000c00 tx=0x7f7d8c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7eb0 0x564f8b1c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f8b10f940 con 0x564f8b1c21b0 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f8b074da0 con 0x564f8b1c7590 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f8b10b900 con 0x564f8b1c7eb0 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41264/0 (socket says 172.21.15.112:41264) 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7d93fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7eb0 0x564f8b1c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 -- 172.21.15.112:0/1000109845 learned_addr learned my addr 172.21.15.112:0/1000109845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 -- 172.21.15.112:0/1000109845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7eb0 msgr2=0x564f8b1c1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7eb0 0x564f8b1c1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 -- 172.21.15.112:0/1000109845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 msgr2=0x564f8b1c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 -- 172.21.15.112:0/1000109845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8b05c8d0 con 0x564f8b1c21b0 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7da0814700 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c2610 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7d940a5020 tx=0x7f7d940a8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e1092beebf7b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7d927fc700 1 -- 172.21.15.112:0/1000109845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940ad350 con 0x564f8b1c21b0 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7d927fc700 1 -- 172.21.15.112:0/1000109845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d940b1d70 con 0x564f8b1c21b0 2024-05-10T10:41:02.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.566+0000 7f7d927fc700 1 -- 172.21.15.112:0/1000109845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940b18e0 con 0x564f8b1c21b0 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/1000109845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 msgr2=0x564f8b1c2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c2610 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7d940a5020 tx=0x7f7d940a8b80 comp rx=0 tx=0).stop 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/1000109845 shutdown_connections 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c2610 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/1000109845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7eb0 0x564f8b1c1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/1000109845 >> 172.21.15.112:0/1000109845 conn(0x564f8b0e9b70 msgr2=0x564f8b1b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/1000109845 shutdown_connections 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.567+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/1000109845 wait complete. 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 -- start start 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 Processor -- start 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7eb0 0x564f8b1cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f8b1ce8e0 con 0x564f8b1c7eb0 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f8b0c8ed0 con 0x564f8b1c7590 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f8b1cea60 con 0x564f8b1c21b0 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7dacb0ab80 10 client.0 _ll_register_callbacks cb 0x564f8b0a35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7eb0 0x564f8b1cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7eb0 0x564f8b1cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41266/0 (socket says 172.21.15.112:41266) 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 -- 172.21.15.112:0/2422399802 learned_addr learned my addr 172.21.15.112:0/2422399802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7d93fff700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 -- 172.21.15.112:0/2422399802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 msgr2=0x564f8b1cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 -- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 msgr2=0x564f8b1caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7da0814700 1 -- 172.21.15.112:0/2422399802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8b0c6760 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.568+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7da0814700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7eb0 0x564f8b1cb4a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7d940a4ed0 tx=0x7f7d940a5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24b75fce733fa55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7d7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2422399802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9406c030 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2422399802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7d940b18e0 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d940b1d70 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2422399802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d940b1ae0 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 10 client.15492 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f8b1d91f0 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2422399802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d940b1d70 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2422399802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7d94012070 con 0x564f8b1c7eb0 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 10 client.15492 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 10 client.15492 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 20 client.15492 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b1db6c0 0x564f8b1ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 4 client.15492 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.569+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f8b11d320 con 0x564f8b1db6c0 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.570+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b1db6c0 0x564f8b1ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.570+0000 7f7d7b7fe700 10 client.15492 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.570+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b1db6c0 0x564f8b1ddab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf7b7fe3ac27c793 server_cookie=836833374e56453b in_seq=0 out_seq=0 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2422399802 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f8b11d320 con 0x564f8b1db6c0 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7d7b7fe700 10 client.15492 handle_command_reply: tid=0 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 client.15492 shutdown 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7eb0 msgr2=0x564f8b1cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7eb0 0x564f8b1cb4a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7d940a4ed0 tx=0x7f7d940a5640 comp rx=0 tx=0).stop 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 shutdown_connections 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7eb0 0x564f8b1cb4a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b1db6c0 0x564f8b1ddab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2422399802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7590 0x564f8b1caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 >> 172.21.15.112:0/2422399802 conn(0x564f8b0e9b70 msgr2=0x564f8b1b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7d7b7fe700 0 client.15492 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 shutdown_connections 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2422399802 wait complete. 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.571+0000 7f7dacb0ab80 20 client.15492 trim_cache size 0 max 16384 2024-05-10T10:41:02.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 Processor -- start 2024-05-10T10:41:02.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 -- start start 2024-05-10T10:41:02.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7590 0x564f8b1ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c1ad100 con 0x564f8b1c21b0 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c1aec40 con 0x564f8b1c7eb0 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c1ad7d0 con 0x564f8b1c7590 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7d9bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7d9bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41282/0 (socket says 172.21.15.112:41282) 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7d9bfff700 1 -- 172.21.15.112:0/3406247178 learned_addr learned my addr 172.21.15.112:0/3406247178 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.586+0000 7f7da0814700 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d9bfff700 1 -- 172.21.15.112:0/3406247178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7590 msgr2=0x564f8b1ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7590 0x564f8b1ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d9bfff700 1 -- 172.21.15.112:0/3406247178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 msgr2=0x564f8b1b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d9bfff700 1 -- 172.21.15.112:0/3406247178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8b1b9190 con 0x564f8b1c21b0 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7da0814700 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c8b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0059d0 tx=0x7f7d8c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e490b08a1c432ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d91ffb700 1 -- 172.21.15.112:0/3406247178 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c038070 con 0x564f8b1c21b0 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d91ffb700 1 -- 172.21.15.112:0/3406247178 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d8c003d10 con 0x564f8b1c21b0 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7d91ffb700 1 -- 172.21.15.112:0/3406247178 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c031bc0 con 0x564f8b1c21b0 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/3406247178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 msgr2=0x564f8b1c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.587+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c8b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0059d0 tx=0x7f7d8c021000 comp rx=0 tx=0).stop 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/3406247178 shutdown_connections 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c21b0 0x564f8b1c8b50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/3406247178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c7590 0x564f8b1ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/3406247178 >> 172.21.15.112:0/3406247178 conn(0x564f8b0e9b70 msgr2=0x564f8b1bc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/3406247178 shutdown_connections 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/3406247178 wait complete. 2024-05-10T10:41:02.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- start start 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 Processor -- start 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7590 0x564f8b1de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1dec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c1ae570 con 0x564f8b1c7590 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c1adea0 con 0x564f8b1c7eb0 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c1ab0a0 con 0x564f8b1c21b0 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7dacb0ab80 10 client.0 _ll_register_callbacks cb 0x564f8b0a35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.588+0000 7f7d93fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7590 0x564f8b1de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7590 0x564f8b1de550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41294/0 (socket says 172.21.15.112:41294) 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 -- 172.21.15.112:0/2307541765 learned_addr learned my addr 172.21.15.112:0/2307541765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:02.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7da0814700 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 -- 172.21.15.112:0/2307541765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 msgr2=0x564f8b1e11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1e11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 -- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 msgr2=0x564f8b1dec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1dec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 -- 172.21.15.112:0/2307541765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8b244be0 con 0x564f8b1c7590 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d93fff700 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7590 0x564f8b1de550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c106250 tx=0x7f7d84012830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102686eb622dd4cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2307541765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84019030 con 0x564f8b1c7590 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2307541765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7d8400f040 con 0x564f8b1c7590 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8c003d10 con 0x564f8b1c7590 2024-05-10T10:41:02.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2307541765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d84026050 con 0x564f8b1c7590 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7dacb0ab80 10 client.15498 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.589+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f8b245da0 con 0x564f8b1c7590 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2307541765 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d8c003d10 con 0x564f8b1c7590 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2307541765 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7d8401e430 con 0x564f8b1c7590 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7dacb0ab80 10 client.15498 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7dacb0ab80 10 client.15498 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7dacb0ab80 20 client.15498 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b246560 0x564f8b248950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7dacb0ab80 4 client.15498 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f8b1173c0 con 0x564f8b246560 2024-05-10T10:41:02.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.590+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b246560 0x564f8b248950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7d7b7fe700 10 client.15498 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7d9bfff700 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b246560 0x564f8b248950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b70d45b0b4840b1 server_cookie=97daf16a6ed5af98 in_seq=0 out_seq=0 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7d7b7fe700 1 -- 172.21.15.112:0/2307541765 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f8b1173c0 con 0x564f8b246560 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7d7b7fe700 10 client.15498 handle_command_reply: tid=0 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 client.15498 shutdown 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7590 msgr2=0x564f8b1de550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7590 0x564f8b1de550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c106250 tx=0x7f7d84012830 comp rx=0 tx=0).stop 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 shutdown_connections 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8b1c7eb0 0x564f8b1dec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x564f8b246560 0x564f8b248950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7d7b7fe700 0 client.15498 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f8b1c21b0 0x564f8b1e11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 --2- 172.21.15.112:0/2307541765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f8b1c7590 0x564f8b1de550 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 >> 172.21.15.112:0/2307541765 conn(0x564f8b0e9b70 msgr2=0x564f8b1bd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 shutdown_connections 2024-05-10T10:41:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 1 -- 172.21.15.112:0/2307541765 wait complete. 2024-05-10T10:41:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.591+0000 7f7dacb0ab80 20 client.15498 trim_cache size 0 max 16384 2024-05-10T10:41:02.599 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2880 msgr2=0x7f7d880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2880 0x7f7d880b4d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c000c00 tx=0x7f7d8c01a040 comp rx=0 tx=0).stop 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c106e80 msgr2=0x7f7d9c1a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c106e80 0x7f7d9c1a5a20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400ab50 tx=0x7f7d9400af10 comp rx=0 tx=0).stop 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 shutdown_connections 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d9c106e80 0x7f7d9c1a5a20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c101da0 0x7f7d9c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7d880b2880 0x7f7d880b4d40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 --2- 172.21.15.112:0/2689729991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d9c102750 0x7f7d9c19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 >> 172.21.15.112:0/2689729991 conn(0x7f7d9c0789a0 msgr2=0x7f7d9c107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 shutdown_connections 2024-05-10T10:41:02.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:02.592+0000 7f7da2277700 1 -- 172.21.15.112:0/2689729991 wait complete. 2024-05-10T10:41:02.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:02 smithi112 ceph-mon[30100]: pgmap v454: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:02 smithi148 ceph-mon[33568]: pgmap v454: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:02 smithi195 ceph-mon[33773]: pgmap v454: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:03.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:41:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:05.013 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:04 smithi112 ceph-mon[30100]: pgmap v455: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:04 smithi148 ceph-mon[33568]: pgmap v455: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:05.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:04 smithi195 ceph-mon[33773]: pgmap v455: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2084591108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac4102d60 msgr2=0x7f7ac410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2084591108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac4102d60 0x7f7ac410f250 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00b0a0 tx=0x7f7abc00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2084591108 shutdown_connections 2024-05-10T10:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2084591108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac4102d60 0x7f7ac410f250 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2084591108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac4108410 0x7f7ac41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2084591108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ac41023a0 0x7f7ac4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2084591108 >> 172.21.15.112:0/2084591108 conn(0x7f7ac4079350 msgr2=0x7f7ac4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2084591108 shutdown_connections 2024-05-10T10:41:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.254+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2084591108 wait complete. 2024-05-10T10:41:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 Processor -- start 2024-05-10T10:41:05.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 -- start start 2024-05-10T10:41:05.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac41023a0 0x7f7ac419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ac4102d60 0x7f7ac419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac4108410 0x7f7ac41a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac4112460 con 0x7f7ac41023a0 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41122e0 con 0x7f7ac4108410 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.255+0000 7f7aca3a4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41125e0 con 0x7f7ac4102d60 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac41023a0 0x7f7ac419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac41023a0 0x7f7ac419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41316/0 (socket says 172.21.15.112:41316) 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2682610996 learned_addr learned my addr 172.21.15.112:0/2682610996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac8941700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac4108410 0x7f7ac41a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ac4102d60 0x7f7ac419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2682610996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ac4102d60 msgr2=0x7f7ac419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ac4102d60 0x7f7ac419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2682610996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac4108410 msgr2=0x7f7ac41a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac4108410 0x7f7ac41a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2682610996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac41a5f20 con 0x7f7ac41023a0 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ac4102d60 0x7f7ac419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac41023a0 0x7f7ac419a960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00d980 tx=0x7f7aac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac17fa700 1 -- 172.21.15.112:0/2682610996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac016070 con 0x7f7ac41023a0 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac41a60f0 con 0x7f7ac41023a0 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac17fa700 1 -- 172.21.15.112:0/2682610996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7aac004830 con 0x7f7ac41023a0 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7ac17fa700 1 -- 172.21.15.112:0/2682610996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac005440 con 0x7f7ac41023a0 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.256+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac41a65f0 con 0x7f7ac41023a0 2024-05-10T10:41:05.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.259+0000 7f7ac17fa700 1 -- 172.21.15.112:0/2682610996 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7aac0055e0 con 0x7f7ac41023a0 2024-05-10T10:41:05.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.260+0000 7f7ac17fa700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7ab00b2880 0x7f7ab00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.260+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7ab00b2880 0x7f7ab00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.260+0000 7f7ac17fa700 1 -- 172.21.15.112:0/2682610996 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7aac0e22e0 con 0x7f7ac41023a0 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.260+0000 7f7ad4c37b80 1 Processor -- start 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.260+0000 7f7ad4c37b80 1 -- start start 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7ab00b2880 0x7f7ab00b4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40799c0 tx=0x7f7ab4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 0x5647f5e96810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e97190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647f5de8940 con 0x5647f5e9c1b0 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5647f5d4dda0 con 0x5647f5e96d50 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647f5de4900 con 0x5647f5e9cad0 2024-05-10T10:41:05.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41326/0 (socket says 172.21.15.112:41326) 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2280687535 learned_addr learned my addr 172.21.15.112:0/2280687535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 0x5647f5e96810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac8941700 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e97190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2280687535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 msgr2=0x5647f5e96810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 0x5647f5e96810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2280687535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 msgr2=0x5647f5e97190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e97190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 -- 172.21.15.112:0/2280687535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f5d358d0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9c590 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0a9270 tx=0x7f7aac0a9c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be34f12012cc81d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.261+0000 7f7aba7fc700 1 -- 172.21.15.112:0/2280687535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0ad920 con 0x5647f5e9c1b0 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7aba7fc700 1 -- 172.21.15.112:0/2280687535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7aac0b23d0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7aba7fc700 1 -- 172.21.15.112:0/2280687535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0b1f40 con 0x5647f5e9c1b0 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2280687535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 msgr2=0x5647f5e9c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9c590 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0a9270 tx=0x7f7aac0a9c70 comp rx=0 tx=0).stop 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2280687535 shutdown_connections 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e97190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9c590 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2280687535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 0x5647f5e96810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2280687535 >> 172.21.15.112:0/2280687535 conn(0x5647f5dc2b70 msgr2=0x5647f5e912a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.262+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2280687535 shutdown_connections 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2280687535 wait complete. 2024-05-10T10:41:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 -- start start 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 Processor -- start 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e96d50 0x5647f5ea2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5ea0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647f5ea33f0 con 0x5647f5e96d50 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5647f5da1ed0 con 0x5647f5e9cad0 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647f5ea3570 con 0x5647f5e9c1b0 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ad4c37b80 10 client.0 _ll_register_callbacks cb 0x5647f5d7c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ac3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e96d50 0x5647f5ea2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.263+0000 7f7ac3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e96d50 0x5647f5ea2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41334/0 (socket says 172.21.15.112:41334) 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac3fff700 1 -- 172.21.15.112:0/203254235 learned_addr learned my addr 172.21.15.112:0/203254235 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac8941700 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5ea0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac3fff700 1 -- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9c1b0 msgr2=0x5647f5e9fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac3fff700 1 -- 172.21.15.112:0/203254235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 msgr2=0x5647f5ea0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5ea0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac3fff700 1 -- 172.21.15.112:0/203254235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f5d9f760 con 0x5647f5e96d50 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e96d50 0x5647f5ea2770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0a9730 tx=0x7f7aac004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b07bd5da624e916 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7aa37fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7aa37fe700 1 -- 172.21.15.112:0/203254235 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0c3e60 con 0x5647f5e96d50 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7aa37fe700 1 -- 172.21.15.112:0/203254235 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7aac0a99d0 con 0x5647f5e96d50 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aac0b23d0 con 0x5647f5e96d50 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7aa37fe700 1 -- 172.21.15.112:0/203254235 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7aac0b1f40 con 0x5647f5e96d50 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ad4c37b80 10 client.15510 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.264+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647f5eb20e0 con 0x5647f5e96d50 2024-05-10T10:41:05.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7aa37fe700 1 -- 172.21.15.112:0/203254235 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7aac0b23d0 con 0x5647f5e96d50 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7aa37fe700 1 -- 172.21.15.112:0/203254235 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7aac08fae0 con 0x5647f5e96d50 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ad4c37b80 10 client.15510 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ad4c37b80 10 client.15510 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ad4c37b80 20 client.15510 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5eb45b0 0x5647f5eb69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ad4c37b80 4 client.15510 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647f5df6320 con 0x5647f5eb45b0 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5eb45b0 0x5647f5eb69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7aa37fe700 10 client.15510 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.265+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5eb45b0 0x5647f5eb69a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81d2a66fd2117b70 server_cookie=ee450c0ab522a113 in_seq=0 out_seq=0 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7aa37fe700 1 -- 172.21.15.112:0/203254235 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647f5df6320 con 0x5647f5eb45b0 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7aa37fe700 10 client.15510 handle_command_reply: tid=0 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 client.15510 shutdown 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e96d50 msgr2=0x5647f5ea2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e96d50 0x5647f5ea2770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0a9730 tx=0x7f7aac004ef0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 shutdown_connections 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5eb45b0 0x5647f5eb69a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5ea0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e96d50 0x5647f5ea2770 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/203254235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9c1b0 0x5647f5e9fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 >> 172.21.15.112:0/203254235 conn(0x5647f5dc2b70 msgr2=0x5647f5e8bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.266+0000 7f7aa37fe700 0 client.15510 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.267+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 shutdown_connections 2024-05-10T10:41:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.267+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/203254235 wait complete. 2024-05-10T10:41:05.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.267+0000 7f7ad4c37b80 20 client.15510 trim_cache size 0 max 16384 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.282+0000 7f7ad4c37b80 1 Processor -- start 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.282+0000 7f7ad4c37b80 1 -- start start 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e96810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e8bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 0x5647f5eabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41ad0c0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41aec00 con 0x5647f5e96d50 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41ad790 con 0x5647f5e9cad0 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e8bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e8bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41348/0 (socket says 172.21.15.112:41348) 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 -- 172.21.15.112:0/1984773330 learned_addr learned my addr 172.21.15.112:0/1984773330 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:05.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e96810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 -- 172.21.15.112:0/1984773330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 msgr2=0x5647f5eabf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 0x5647f5eabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 -- 172.21.15.112:0/1984773330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 msgr2=0x5647f5e96810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e96810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 -- 172.21.15.112:0/1984773330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f5e9e080 con 0x5647f5e9c1b0 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e96810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e8bc40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009fd0 tx=0x7f7ab4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56effff9f197bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ab9ffb700 1 -- 172.21.15.112:0/1984773330 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4012ac0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ab9ffb700 1 -- 172.21.15.112:0/1984773330 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ab40044b0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.283+0000 7f7ab9ffb700 1 -- 172.21.15.112:0/1984773330 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4029970 con 0x5647f5e9c1b0 2024-05-10T10:41:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/1984773330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 msgr2=0x5647f5e8bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e8bc40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009fd0 tx=0x7f7ab4016000 comp rx=0 tx=0).stop 2024-05-10T10:41:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/1984773330 shutdown_connections 2024-05-10T10:41:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e9cad0 0x5647f5eabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e96d50 0x5647f5e96810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/1984773330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5e8bc40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/1984773330 >> 172.21.15.112:0/1984773330 conn(0x5647f5dc2b70 msgr2=0x5647f5eab800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/1984773330 shutdown_connections 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/1984773330 wait complete. 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.284+0000 7f7ad4c37b80 1 -- start start 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 1 Processor -- start 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e96d50 0x5647f5eb6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5eb6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5f1cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41ae530 con 0x5647f5e9c1b0 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41ade60 con 0x5647f5e9cad0 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41ab060 con 0x5647f5e96d50 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ad4c37b80 10 client.0 _ll_register_callbacks cb 0x5647f5d7c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5eb6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5eb6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41360/0 (socket says 172.21.15.112:41360) 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 -- 172.21.15.112:0/2024445274 learned_addr learned my addr 172.21.15.112:0/2024445274 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac8941700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5f1cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e96d50 0x5647f5eb6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 -- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e96d50 msgr2=0x5647f5eb6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e96d50 0x5647f5eb6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 -- 172.21.15.112:0/2024445274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 msgr2=0x5647f5f1cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5f1cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac37fe700 1 -- 172.21.15.112:0/2024445274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f5f1d0b0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.285+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e96d50 0x5647f5eb6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ac37fe700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5eb6e90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4015040 tx=0x7f7ab4010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c316d73e94c28c2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7aa37fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7aa37fe700 1 -- 172.21.15.112:0/2024445274 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4037020 con 0x5647f5e9c1b0 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7aa37fe700 1 -- 172.21.15.112:0/2024445274 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7ab4012d90 con 0x5647f5e9c1b0 2024-05-10T10:41:05.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab40044b0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7aa37fe700 1 -- 172.21.15.112:0/2024445274 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ab40295f0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 10 client.15516 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647f5f1e300 con 0x5647f5e9c1b0 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7aa37fe700 1 -- 172.21.15.112:0/2024445274 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ab40044b0 con 0x5647f5e9c1b0 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7aa37fe700 1 -- 172.21.15.112:0/2024445274 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ab400f040 con 0x5647f5e9c1b0 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 10 client.15516 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 20 client.15516 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5f1e980 0x5647f5f20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 4 client.15516 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.286+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647f5df03c0 con 0x5647f5f1e980 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5f1e980 0x5647f5f20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7aa37fe700 10 client.15516 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ac3fff700 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5f1e980 0x5647f5f20da0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4c1a8565922b9b3 server_cookie=f01bab0e108b6479 in_seq=0 out_seq=0 2024-05-10T10:41:05.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7aa37fe700 1 -- 172.21.15.112:0/2024445274 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647f5df03c0 con 0x5647f5f1e980 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7aa37fe700 10 client.15516 handle_command_reply: tid=0 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 client.15516 shutdown 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 msgr2=0x5647f5eb6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5eb6e90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4015040 tx=0x7f7ab4010a70 comp rx=0 tx=0).stop 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 shutdown_connections 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5647f5f1e980 0x5647f5f20da0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647f5e9cad0 0x5647f5f1cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f5e96d50 0x5647f5eb6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 --2- 172.21.15.112:0/2024445274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647f5e9c1b0 0x5647f5eb6e90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7aa37fe700 0 client.15516 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.287+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 >> 172.21.15.112:0/2024445274 conn(0x5647f5dc2b70 msgr2=0x5647f5eac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 shutdown_connections 2024-05-10T10:41:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7ad4c37b80 1 -- 172.21.15.112:0/2024445274 wait complete. 2024-05-10T10:41:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7ad4c37b80 20 client.15516 trim_cache size 0 max 16384 2024-05-10T10:41:05.295 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7ab00b2880 msgr2=0x7f7ab00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7ab00b2880 0x7f7ab00b4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40799c0 tx=0x7f7ab4009260 comp rx=0 tx=0).stop 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac41023a0 msgr2=0x7f7ac419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac41023a0 0x7f7ac419a960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00d980 tx=0x7f7aac00dc90 comp rx=0 tx=0).stop 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 shutdown_connections 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac4108410 0x7f7ac41a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac41023a0 0x7f7ac419a960 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7ab00b2880 0x7f7ab00b4d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 --2- 172.21.15.112:0/2682610996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ac4102d60 0x7f7ac419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 >> 172.21.15.112:0/2682610996 conn(0x7f7ac4079350 msgr2=0x7f7ac4100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 shutdown_connections 2024-05-10T10:41:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:05.288+0000 7f7aca3a4700 1 -- 172.21.15.112:0/2682610996 wait complete. 2024-05-10T10:41:06.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:41:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:06.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:06 smithi112 ceph-mon[30100]: pgmap v456: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:06 smithi148 ceph-mon[33568]: pgmap v456: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:06 smithi195 ceph-mon[33773]: pgmap v456: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 -- 172.21.15.112:0/860367588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4109540 msgr2=0x7f47e4109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/860367588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4109540 0x7f47e4109920 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0099c0 tx=0x7f47cc009cd0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 -- 172.21.15.112:0/860367588 shutdown_connections 2024-05-10T10:41:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/860367588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4103f20 0x7f47e4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/860367588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4109540 0x7f47e4109920 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/860367588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e4103560 0x7f47e41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 -- 172.21.15.112:0/860367588 >> 172.21.15.112:0/860367588 conn(0x7f47e407a660 msgr2=0x7f47e407aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.024+0000 7f47e8ba1700 1 -- 172.21.15.112:0/860367588 shutdown_connections 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.025+0000 7f47e8ba1700 1 -- 172.21.15.112:0/860367588 wait complete. 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.025+0000 7f47e8ba1700 1 Processor -- start 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.025+0000 7f47e8ba1700 1 -- start start 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.025+0000 7f47e8ba1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4103560 0x7f47e40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.025+0000 7f47e8ba1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4103f20 0x7f47e4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e8ba1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e4109540 0x7f47e4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e8ba1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4113500 con 0x7f47e4103560 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e8ba1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4113380 con 0x7f47e4109540 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e8ba1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4113680 con 0x7f47e4103f20 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4103560 0x7f47e40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4103560 0x7f47e40757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39078/0 (socket says 172.21.15.112:39078) 2024-05-10T10:41:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 -- 172.21.15.112:0/2058210809 learned_addr learned my addr 172.21.15.112:0/2058210809 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e2d9d700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e4109540 0x7f47e4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4103f20 0x7f47e4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 -- 172.21.15.112:0/2058210809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4103f20 msgr2=0x7f47e4075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4103f20 0x7f47e4075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 -- 172.21.15.112:0/2058210809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e4109540 msgr2=0x7f47e4076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e4109540 0x7f47e4076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 -- 172.21.15.112:0/2058210809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e41a7f80 con 0x7f47e4103560 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e2d9d700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e4109540 0x7f47e4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e259c700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4103560 0x7f47e40757a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00f690 tx=0x7f47cc00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47db7fe700 1 -- 172.21.15.112:0/2058210809 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc027070 con 0x7f47e4103560 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e41a8210 con 0x7f47e4103560 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47db7fe700 1 -- 172.21.15.112:0/2058210809 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47cc0056d0 con 0x7f47e4103560 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47db7fe700 1 -- 172.21.15.112:0/2058210809 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0173f0 con 0x7f47e4103560 2024-05-10T10:41:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.026+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e41a86c0 con 0x7f47e4103560 2024-05-10T10:41:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.029+0000 7f47db7fe700 1 -- 172.21.15.112:0/2058210809 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f47cc02a070 con 0x7f47e4103560 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.029+0000 7f47db7fe700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f47d00b2880 0x7f47d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.029+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f47d00b2880 0x7f47d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.029+0000 7f47db7fe700 1 -- 172.21.15.112:0/2058210809 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f47cc0e3380 con 0x7f47e4103560 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f47d00b2880 0x7f47d00b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005ea0 tx=0x7f47d4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 Processor -- start 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 -- start start 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702c4d0 0x561c0702c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702cdf0 0x561c070306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c07031100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c06f7e860 con 0x561c0702cdf0 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561c06f1d7c0 con 0x561c07030c80 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.030+0000 7f47f3434b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561c06f7a820 con 0x561c0702c4d0 2024-05-10T10:41:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702cdf0 0x561c070306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702cdf0 0x561c070306f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39082/0 (socket says 172.21.15.112:39082) 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 -- 172.21.15.112:0/989947315 learned_addr learned my addr 172.21.15.112:0/989947315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e2d9d700 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c07031100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e259c700 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702c4d0 0x561c0702c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 -- 172.21.15.112:0/989947315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702c4d0 msgr2=0x561c0702c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702c4d0 0x561c0702c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 -- 172.21.15.112:0/989947315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 msgr2=0x561c07031100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c07031100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 -- 172.21.15.112:0/989947315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c06ecb8d0 con 0x561c0702cdf0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702cdf0 0x561c070306f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47d400ec90 tx=0x7f47d400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3ea30e7591000 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47d8ff9700 1 -- 172.21.15.112:0/989947315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4021070 con 0x561c0702cdf0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47d8ff9700 1 -- 172.21.15.112:0/989947315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47d4010e40 con 0x561c0702cdf0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.031+0000 7f47d8ff9700 1 -- 172.21.15.112:0/989947315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d400f600 con 0x561c0702cdf0 2024-05-10T10:41:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 -- 172.21.15.112:0/989947315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702cdf0 msgr2=0x561c070306f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702cdf0 0x561c070306f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47d400ec90 tx=0x7f47d400af90 comp rx=0 tx=0).stop 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 -- 172.21.15.112:0/989947315 shutdown_connections 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c07031100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702c4d0 0x561c0702c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 --2- 172.21.15.112:0/989947315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702cdf0 0x561c070306f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 -- 172.21.15.112:0/989947315 >> 172.21.15.112:0/989947315 conn(0x561c06f58100 msgr2=0x561c07026c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 -- 172.21.15.112:0/989947315 shutdown_connections 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 -- 172.21.15.112:0/989947315 wait complete. 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.032+0000 7f47f3434b80 1 -- start start 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 1 Processor -- start 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c0702c4d0 0x561c07038030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07035420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c07030c80 0x561c07035960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c07038e90 con 0x561c07030c80 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561c06ee4c30 con 0x561c0702c4d0 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561c07039010 con 0x561c0702cdf0 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47f3434b80 10 client.0 _ll_register_callbacks cb 0x561c06f16920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c07030c80 0x561c07035960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c07030c80 0x561c07035960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39096/0 (socket says 172.21.15.112:39096) 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 -- 172.21.15.112:0/1583561209 learned_addr learned my addr 172.21.15.112:0/1583561209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e259c700 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c0702c4d0 0x561c07038030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07035420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 -- 172.21.15.112:0/1583561209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 msgr2=0x561c07035420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07035420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 -- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c0702c4d0 msgr2=0x561c07038030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c0702c4d0 0x561c07038030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.033+0000 7f47e2d9d700 1 -- 172.21.15.112:0/1583561209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c06f1a690 con 0x561c07030c80 2024-05-10T10:41:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47e2d9d700 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c07030c80 0x561c07035960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f47dc008f50 tx=0x7f47dc010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bc05dd2d5c7af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47c9ffb700 1 -- 172.21.15.112:0/1583561209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc007e60 con 0x561c07030c80 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c06ff8650 con 0x561c07030c80 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47c9ffb700 1 -- 172.21.15.112:0/1583561209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f47dc010d20 con 0x561c07030c80 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47c9ffb700 1 -- 172.21.15.112:0/1583561209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47dc024050 con 0x561c07030c80 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 10 client.15528 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561c070436e0 con 0x561c07030c80 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47c9ffb700 1 -- 172.21.15.112:0/1583561209 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561c06ff8650 con 0x561c07030c80 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47c9ffb700 1 -- 172.21.15.112:0/1583561209 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47dc0153d0 con 0x561c07030c80 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 10 client.15528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 20 client.15528 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c07045d70 0x561c070481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 4 client.15528 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.034+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c06f8c240 con 0x561c07045d70 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.035+0000 7f47e259c700 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c07045d70 0x561c070481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.035+0000 7f47e259c700 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c07045d70 0x561c070481d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b1a5d76b37815d server_cookie=8dc6bca0adfb04dd in_seq=0 out_seq=0 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.035+0000 7f47c9ffb700 10 client.15528 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47c9ffb700 1 -- 172.21.15.112:0/1583561209 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c06f8c240 con 0x561c07045d70 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47c9ffb700 10 client.15528 handle_command_reply: tid=0 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 client.15528 shutdown 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c07030c80 msgr2=0x561c07035960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c07030c80 0x561c07035960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f47dc008f50 tx=0x7f47dc010430 comp rx=0 tx=0).stop 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 shutdown_connections 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c07045d70 0x561c070481d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c07030c80 0x561c07035960 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c0702c4d0 0x561c07038030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 --2- 172.21.15.112:0/1583561209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07035420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47c9ffb700 0 client.15528 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 >> 172.21.15.112:0/1583561209 conn(0x561c06f58100 msgr2=0x561c07021720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 shutdown_connections 2024-05-10T10:41:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 1 -- 172.21.15.112:0/1583561209 wait complete. 2024-05-10T10:41:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.036+0000 7f47f3434b80 20 client.15528 trim_cache size 0 max 16384 2024-05-10T10:41:08.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 Processor -- start 2024-05-10T10:41:08.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 -- start start 2024-05-10T10:41:08.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07041f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07034d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c070215a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b1220 con 0x561c0702c4d0 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b2d60 con 0x561c07030c80 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47f3434b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b18f0 con 0x561c0702cdf0 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.051+0000 7f47e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07041f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07041f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39106/0 (socket says 172.21.15.112:39106) 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 -- 172.21.15.112:0/3487263075 learned_addr learned my addr 172.21.15.112:0/3487263075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07034d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e2d9d700 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c070215a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 -- 172.21.15.112:0/3487263075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 msgr2=0x561c07034d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07034d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 -- 172.21.15.112:0/3487263075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 msgr2=0x561c070215a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c070215a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 -- 172.21.15.112:0/3487263075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c07048850 con 0x561c0702c4d0 2024-05-10T10:41:08.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47e259c700 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07041f50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0a7a40 tx=0x7f47cc0e0af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fad4367f19947a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47cbfff700 1 -- 172.21.15.112:0/3487263075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0a9f40 con 0x561c0702c4d0 2024-05-10T10:41:08.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47cbfff700 1 -- 172.21.15.112:0/3487263075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47cc00fe00 con 0x561c0702c4d0 2024-05-10T10:41:08.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.052+0000 7f47cbfff700 1 -- 172.21.15.112:0/3487263075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0aeb80 con 0x561c0702c4d0 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 -- 172.21.15.112:0/3487263075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 msgr2=0x561c07041f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07041f50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0a7a40 tx=0x7f47cc0e0af0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 -- 172.21.15.112:0/3487263075 shutdown_connections 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c070215a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07041f50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 --2- 172.21.15.112:0/3487263075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07034d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 -- 172.21.15.112:0/3487263075 >> 172.21.15.112:0/3487263075 conn(0x561c06f58100 msgr2=0x561c07020e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 -- 172.21.15.112:0/3487263075 shutdown_connections 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 -- 172.21.15.112:0/3487263075 wait complete. 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 -- start start 2024-05-10T10:41:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.053+0000 7f47f3434b80 1 Processor -- start 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07039610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07039b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c070ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b2690 con 0x561c0702c4d0 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b1fc0 con 0x561c07030c80 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41af1c0 con 0x561c0702cdf0 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 10 client.0 _ll_register_callbacks cb 0x561c06f16920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07039610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07039610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39122/0 (socket says 172.21.15.112:39122) 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 -- 172.21.15.112:0/2833335703 learned_addr learned my addr 172.21.15.112:0/2833335703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07039b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 -- 172.21.15.112:0/2833335703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 msgr2=0x561c07039b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07039b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 -- 172.21.15.112:0/2833335703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 msgr2=0x561c070ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:08.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c070ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 -- 172.21.15.112:0/2833335703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c070aec30 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07039b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47e259c700 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07039610 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f47cc005f50 tx=0x7f47cc0b1160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7953fa850c28e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47c9ffb700 1 -- 172.21.15.112:0/2833335703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0aaed0 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47c9ffb700 1 -- 172.21.15.112:0/2833335703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f47cc0b3110 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c06f862e0 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47c9ffb700 1 -- 172.21.15.112:0/2833335703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47cc0b35a0 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 10 client.15534 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.054+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561c070aff10 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47c9ffb700 1 -- 172.21.15.112:0/2833335703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561c06f862e0 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47c9ffb700 1 -- 172.21.15.112:0/2833335703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47cc0bfb90 con 0x561c0702c4d0 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47f3434b80 10 client.15534 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47f3434b80 10 client.15534 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47f3434b80 20 client.15534 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47f3434b80 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c070b0570 0x561c070b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47f3434b80 4 client.15534 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c06dfe250 con 0x561c070b0570 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.055+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c070b0570 0x561c070b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47c9ffb700 10 client.15534 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47e1d9b700 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c070b0570 0x561c070b2990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd1bf39c37c0f408 server_cookie=3890961356981975 in_seq=0 out_seq=0 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47c9ffb700 1 -- 172.21.15.112:0/2833335703 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c06dfe250 con 0x561c070b0570 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47c9ffb700 10 client.15534 handle_command_reply: tid=0 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 client.15534 shutdown 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 msgr2=0x561c07039610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07039610 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f47cc005f50 tx=0x7f47cc0b1160 comp rx=0 tx=0).stop 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 shutdown_connections 2024-05-10T10:41:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c07030c80 0x561c070ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x561c070b0570 0x561c070b2990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c0702c4d0 0x561c07039610 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 --2- 172.21.15.112:0/2833335703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561c0702cdf0 0x561c07039b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47c9ffb700 0 client.15534 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 >> 172.21.15.112:0/2833335703 conn(0x561c06f58100 msgr2=0x561c07021720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 shutdown_connections 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 1 -- 172.21.15.112:0/2833335703 wait complete. 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.056+0000 7f47f3434b80 20 client.15534 trim_cache size 0 max 16384 2024-05-10T10:41:08.063 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f47d00b2880 msgr2=0x7f47d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f47d00b2880 0x7f47d00b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005ea0 tx=0x7f47d4005e30 comp rx=0 tx=0).stop 2024-05-10T10:41:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4103560 msgr2=0x7f47e40757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4103560 0x7f47e40757a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00f690 tx=0x7f47cc00f770 comp rx=0 tx=0).stop 2024-05-10T10:41:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 shutdown_connections 2024-05-10T10:41:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e4109540 0x7f47e4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47e4103560 0x7f47e40757a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f47d00b2880 0x7f47d00b4d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 --2- 172.21.15.112:0/2058210809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4103f20 0x7f47e4075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 >> 172.21.15.112:0/2058210809 conn(0x7f47e407a660 msgr2=0x7f47e4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 shutdown_connections 2024-05-10T10:41:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:08.057+0000 7f47e8ba1700 1 -- 172.21.15.112:0/2058210809 wait complete. 2024-05-10T10:41:08.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:41:09.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:08 smithi148 ceph-mon[33568]: pgmap v457: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:08 smithi195 ceph-mon[33773]: pgmap v457: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:09.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:08 smithi112 ceph-mon[30100]: pgmap v457: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:09.086 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:10.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 -- 172.21.15.112:0/137648595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 msgr2=0x7f55c81099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/137648595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c81099b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f55c40099c0 tx=0x7f55c4009cd0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 -- 172.21.15.112:0/137648595 shutdown_connections 2024-05-10T10:41:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/137648595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c8103ef0 0x7f55c81103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/137648595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c81099b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/137648595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55c8103530 0x7f55c81039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 -- 172.21.15.112:0/137648595 >> 172.21.15.112:0/137648595 conn(0x7f55c807a690 msgr2=0x7f55c807aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 -- 172.21.15.112:0/137648595 shutdown_connections 2024-05-10T10:41:10.916 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:10 smithi112 ceph-mon[30100]: pgmap v458: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:10.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.913+0000 7f55cf7a8700 1 -- 172.21.15.112:0/137648595 wait complete. 2024-05-10T10:41:10.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 Processor -- start 2024-05-10T10:41:10.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 -- start start 2024-05-10T10:41:10.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55c8103530 0x7f55c819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c8103ef0 0x7f55c819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55c8113570 con 0x7f55c81095d0 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81133f0 con 0x7f55c8103ef0 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cf7a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81136f0 con 0x7f55c8103530 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cdd45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cdd45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c8199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39130/0 (socket says 172.21.15.112:39130) 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cdd45700 1 -- 172.21.15.112:0/374136195 learned_addr learned my addr 172.21.15.112:0/374136195 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55cd544700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55c8103530 0x7f55c819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.914+0000 7f55ccd43700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c8103ef0 0x7f55c819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cdd45700 1 -- 172.21.15.112:0/374136195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55c8103530 msgr2=0x7f55c819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cdd45700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55c8103530 0x7f55c819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cdd45700 1 -- 172.21.15.112:0/374136195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c8103ef0 msgr2=0x7f55c819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cdd45700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c8103ef0 0x7f55c819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cdd45700 1 -- 172.21.15.112:0/374136195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c819a030 con 0x7f55c81095d0 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cdd45700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c8199850 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007ee0 tx=0x7f55c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55be7fc700 1 -- 172.21.15.112:0/374136195 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0004bb0 con 0x7f55c81095d0 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55be7fc700 1 -- 172.21.15.112:0/374136195 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c0004d50 con 0x7f55c81095d0 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c819a320 con 0x7f55c81095d0 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55be7fc700 1 -- 172.21.15.112:0/374136195 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00058b0 con 0x7f55c81095d0 2024-05-10T10:41:10.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.915+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c81a6530 con 0x7f55c81095d0 2024-05-10T10:41:10.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.918+0000 7f55be7fc700 1 -- 172.21.15.112:0/374136195 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f55c001a020 con 0x7f55c81095d0 2024-05-10T10:41:10.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.918+0000 7f55be7fc700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55b40b2880 0x7f55b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.919+0000 7f55cd544700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55b40b2880 0x7f55b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.919+0000 7f55be7fc700 1 -- 172.21.15.112:0/374136195 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f55c00e1bd0 con 0x7f55c81095d0 2024-05-10T10:41:10.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.919+0000 7f55cd544700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55b40b2880 0x7f55b40b4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009990 tx=0x7f55c4005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 Processor -- start 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 -- start start 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffbf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffc0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ff2ff11940 con 0x55ff2ffbfd50 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ff2fe76da0 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55da03bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ff2ff0d900 con 0x55ff2ffc51b0 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffc0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffc0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39146/0 (socket says 172.21.15.112:39146) 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 -- 172.21.15.112:0/1259765513 learned_addr learned my addr 172.21.15.112:0/1259765513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cd544700 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55ccd43700 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffbf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 -- 172.21.15.112:0/1259765513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 msgr2=0x55ff2ffc5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 -- 172.21.15.112:0/1259765513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 msgr2=0x55ff2ffbf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffbf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 -- 172.21.15.112:0/1259765513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2fe5e8d0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.920+0000 7f55cdd45700 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffc0190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f55c00a4ee0 tx=0x7f55c00a8880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bf6a0a109a8e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55b37fe700 1 -- 172.21.15.112:0/1259765513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00a8e60 con 0x55ff2ffbfd50 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55b37fe700 1 -- 172.21.15.112:0/1259765513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c00ad210 con 0x55ff2ffbfd50 2024-05-10T10:41:10.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55b37fe700 1 -- 172.21.15.112:0/1259765513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00b1830 con 0x55ff2ffbfd50 2024-05-10T10:41:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55da03bb80 1 -- 172.21.15.112:0/1259765513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 msgr2=0x55ff2ffc0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55da03bb80 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffc0190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f55c00a4ee0 tx=0x7f55c00a8880 comp rx=0 tx=0).stop 2024-05-10T10:41:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55da03bb80 1 -- 172.21.15.112:0/1259765513 shutdown_connections 2024-05-10T10:41:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55da03bb80 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffc0190 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55da03bb80 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55da03bb80 1 --2- 172.21.15.112:0/1259765513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffbf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.921+0000 7f55da03bb80 1 -- 172.21.15.112:0/1259765513 >> 172.21.15.112:0/1259765513 conn(0x55ff2feebb70 msgr2=0x55ff2ffba2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.922+0000 7f55da03bb80 1 -- 172.21.15.112:0/1259765513 shutdown_connections 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.922+0000 7f55da03bb80 1 -- 172.21.15.112:0/1259765513 wait complete. 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.922+0000 7f55da03bb80 1 -- start start 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.922+0000 7f55da03bb80 1 Processor -- start 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.922+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffcb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.922+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55da03bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ff2ffcc450 con 0x55ff2ffbfd50 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55da03bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ff2fecaed0 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55da03bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ff2ffcc5d0 con 0x55ff2ffc51b0 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffcb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55da03bb80 10 client.0 _ll_register_callbacks cb 0x55ff2fea55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:10.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffcb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39152/0 (socket says 172.21.15.112:39152) 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 -- 172.21.15.112:0/3880340574 learned_addr learned my addr 172.21.15.112:0/3880340574 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55ccd43700 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cdd45700 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 -- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 msgr2=0x55ff2ffc8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 -- 172.21.15.112:0/3880340574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 msgr2=0x55ff2ffc90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 -- 172.21.15.112:0/3880340574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2fec8760 con 0x55ff2ffbfd50 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55cd544700 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffcb800 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f55c40099c0 tx=0x7f55c4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323d25d6500a6a56 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55b0ff9700 1 -- 172.21.15.112:0/3880340574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c400f460 con 0x55ff2ffbfd50 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55b0ff9700 1 -- 172.21.15.112:0/3880340574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f55c400f600 con 0x55ff2ffbfd50 2024-05-10T10:41:10.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.923+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c00ad210 con 0x55ff2ffbfd50 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55b0ff9700 1 -- 172.21.15.112:0/3880340574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55c403d050 con 0x55ff2ffbfd50 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 10 client.15546 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff2ffd6de0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55b0ff9700 1 -- 172.21.15.112:0/3880340574 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c00ad210 con 0x55ff2ffbfd50 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55b0ff9700 1 -- 172.21.15.112:0/3880340574 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f55c402f7e0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 10 client.15546 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 10 client.15546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 20 client.15546 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff2ffd9470 0x55ff2ffdb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 4 client.15546 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff2ff1f320 con 0x55ff2ffd9470 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.924+0000 7f55ccd43700 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff2ffd9470 0x55ff2ffdb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.925+0000 7f55ccd43700 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff2ffd9470 0x55ff2ffdb8d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78734a99d0822250 server_cookie=b2112cbcc2de697a in_seq=0 out_seq=0 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.925+0000 7f55b0ff9700 10 client.15546 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55b0ff9700 1 -- 172.21.15.112:0/3880340574 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff2ff1f320 con 0x55ff2ffd9470 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55b0ff9700 10 client.15546 handle_command_reply: tid=0 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 client.15546 shutdown 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 msgr2=0x55ff2ffcb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffcb800 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f55c40099c0 tx=0x7f55c4009310 comp rx=0 tx=0).stop 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 shutdown_connections 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff2ffd9470 0x55ff2ffdb8d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffcb800 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55b0ff9700 0 client.15546 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 --2- 172.21.15.112:0/3880340574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 >> 172.21.15.112:0/3880340574 conn(0x55ff2feebb70 msgr2=0x55ff2ffb4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 shutdown_connections 2024-05-10T10:41:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 1 -- 172.21.15.112:0/3880340574 wait complete. 2024-05-10T10:41:10.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.926+0000 7f55da03bb80 20 client.15546 trim_cache size 0 max 16384 2024-05-10T10:41:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 Processor -- start 2024-05-10T10:41:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 -- start start 2024-05-10T10:41:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffbf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81af070 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81b0bb0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55da03bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81af740 con 0x55ff2ffc51b0 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55cdd45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55cdd45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39168/0 (socket says 172.21.15.112:39168) 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55cdd45700 1 -- 172.21.15.112:0/2956900752 learned_addr learned my addr 172.21.15.112:0/2956900752 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55ccd43700 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55cd544700 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffbf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55cdd45700 1 -- 172.21.15.112:0/2956900752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 msgr2=0x55ff2ffc6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55cdd45700 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.945+0000 7f55cdd45700 1 -- 172.21.15.112:0/2956900752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffbfd50 msgr2=0x55ff2ffbf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.946+0000 7f55cdd45700 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffbf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.946+0000 7f55cdd45700 1 -- 172.21.15.112:0/2956900752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2ffc57e0 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.946+0000 7f55cdd45700 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc7d20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009510 tx=0x7f55c00a8700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e91ded47b59892 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.946+0000 7f55b2ffd700 1 -- 172.21.15.112:0/2956900752 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c006d070 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.946+0000 7f55b2ffd700 1 -- 172.21.15.112:0/2956900752 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c00b1cc0 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.946+0000 7f55b2ffd700 1 -- 172.21.15.112:0/2956900752 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00b1830 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 -- 172.21.15.112:0/2956900752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc5ad0 msgr2=0x55ff2ffc7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc7d20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009510 tx=0x7f55c00a8700 comp rx=0 tx=0).stop 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 -- 172.21.15.112:0/2956900752 shutdown_connections 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffc7d20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffbf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 --2- 172.21.15.112:0/2956900752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffc6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 -- 172.21.15.112:0/2956900752 >> 172.21.15.112:0/2956900752 conn(0x55ff2feebb70 msgr2=0x55ff2ffc8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 -- 172.21.15.112:0/2956900752 shutdown_connections 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 -- 172.21.15.112:0/2956900752 wait complete. 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 -- start start 2024-05-10T10:41:10.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.947+0000 7f55da03bb80 1 Processor -- start 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffd1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffd1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55da03bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffe02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55da03bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81b04e0 con 0x55ff2ffc51b0 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55da03bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81afe10 con 0x55ff2ffc5ad0 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55da03bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55c81ad010 con 0x55ff2ffbfd50 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55da03bb80 10 client.0 _ll_register_callbacks cb 0x55ff2fea55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55ccd43700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffd1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cd544700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffd1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cdd45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffe02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55ccd43700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffd1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39176/0 (socket says 172.21.15.112:39176) 2024-05-10T10:41:10.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55ccd43700 1 -- 172.21.15.112:0/219123863 learned_addr learned my addr 172.21.15.112:0/219123863 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cd544700 1 -- 172.21.15.112:0/219123863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 msgr2=0x55ff2ffe02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cd544700 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffe02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cd544700 1 -- 172.21.15.112:0/219123863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc51b0 msgr2=0x55ff2ffd1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cd544700 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffd1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cd544700 1 -- 172.21.15.112:0/219123863 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2ffe0b10 con 0x55ff2ffbfd50 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55cd544700 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffd1a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55c402fdf0 tx=0x7f55c400b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520c245cee1f74a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.948+0000 7f55b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55b0ff9700 1 -- 172.21.15.112:0/219123863 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c402f800 con 0x55ff2ffbfd50 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff2ff193c0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55b0ff9700 1 -- 172.21.15.112:0/219123863 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f55c402f9a0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55b0ff9700 1 -- 172.21.15.112:0/219123863 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55c40175f0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 10 client.25066 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:10.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff30043910 con 0x55ff2ffbfd50 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55b0ff9700 1 -- 172.21.15.112:0/219123863 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ff2ff193c0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55b0ff9700 1 -- 172.21.15.112:0/219123863 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f55c400f7d0 con 0x55ff2ffbfd50 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 10 client.25066 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 10 client.25066 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 20 client.25066 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff30043dd0 0x55ff300461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 4 client.25066 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.949+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff2fd91250 con 0x55ff30043dd0 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55ccd43700 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff30043dd0 0x55ff300461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55b0ff9700 10 client.25066 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55ccd43700 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff30043dd0 0x55ff300461c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd46c6b50e27725f server_cookie=5f24022aed86099e in_seq=0 out_seq=0 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55b0ff9700 1 -- 172.21.15.112:0/219123863 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff2fd91250 con 0x55ff30043dd0 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55b0ff9700 10 client.25066 handle_command_reply: tid=0 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 client.25066 shutdown 2024-05-10T10:41:10.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffbfd50 msgr2=0x55ff2ffd1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffd1a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55c402fdf0 tx=0x7f55c400b9b0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 shutdown_connections 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ff30043dd0 0x55ff300461c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff2ffc5ad0 0x55ff2ffe02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55b0ff9700 0 client.25066 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff2ffbfd50 0x55ff2ffd1a40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 --2- 172.21.15.112:0/219123863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff2ffc51b0 0x55ff2ffd1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.950+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 >> 172.21.15.112:0/219123863 conn(0x55ff2feebb70 msgr2=0x55ff2ffc7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 shutdown_connections 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55da03bb80 1 -- 172.21.15.112:0/219123863 wait complete. 2024-05-10T10:41:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55da03bb80 20 client.25066 trim_cache size 0 max 16384 2024-05-10T10:41:10.959 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:10.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55b40b2880 msgr2=0x7f55b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55b40b2880 0x7f55b40b4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009990 tx=0x7f55c4005a90 comp rx=0 tx=0).stop 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 msgr2=0x7f55c8199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c8199850 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007ee0 tx=0x7f55c000d3b0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 shutdown_connections 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c81095d0 0x7f55c8199850 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f55b40b2880 0x7f55b40b4d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55c8103530 0x7f55c819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 --2- 172.21.15.112:0/374136195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c8103ef0 0x7f55c819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 >> 172.21.15.112:0/374136195 conn(0x7f55c807a690 msgr2=0x7f55c81020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 shutdown_connections 2024-05-10T10:41:10.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:10.951+0000 7f55cf7a8700 1 -- 172.21.15.112:0/374136195 wait complete. 2024-05-10T10:41:11.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:10 smithi148 ceph-mon[33568]: pgmap v458: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:11.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:10 smithi195 ceph-mon[33773]: pgmap v458: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 -- 172.21.15.112:0/1971807083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700101910 msgr2=0x7f8700101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 --2- 172.21.15.112:0/1971807083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700101910 0x7f8700101d30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002a410 comp rx=0 tx=0).stop 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 -- 172.21.15.112:0/1971807083 shutdown_connections 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 --2- 172.21.15.112:0/1971807083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87001034c0 0x7f8700109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 --2- 172.21.15.112:0/1971807083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8700102b00 0x7f8700102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 --2- 172.21.15.112:0/1971807083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700101910 0x7f8700101d30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 -- 172.21.15.112:0/1971807083 >> 172.21.15.112:0/1971807083 conn(0x7f87000fce80 msgr2=0x7f87000ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 -- 172.21.15.112:0/1971807083 shutdown_connections 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.259+0000 7f8706472640 1 -- 172.21.15.112:0/1971807083 wait complete. 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 Processor -- start 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 -- start start 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8700101910 0x7f8700198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700102b00 0x7f87001990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87001034c0 0x7f87001a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f870010e410 con 0x7f8700102b00 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f870010e290 con 0x7f87001034c0 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8706472640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f870010e590 con 0x7f8700101910 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700102b00 0x7f87001990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700102b00 0x7f87001990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39190/0 (socket says 172.21.15.112:39190) 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 -- 172.21.15.112:0/986163684 learned_addr learned my addr 172.21.15.112:0/986163684 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f8704a0f640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87001034c0 0x7f87001a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86fffff640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8700101910 0x7f8700198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 -- 172.21.15.112:0/986163684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8700101910 msgr2=0x7f8700198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8700101910 0x7f8700198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 -- 172.21.15.112:0/986163684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87001034c0 msgr2=0x7f87001a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87001034c0 0x7f87001a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 -- 172.21.15.112:0/986163684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87001a0770 con 0x7f8700102b00 2024-05-10T10:41:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86fffff640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8700101910 0x7f8700198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86f7fff640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700102b00 0x7f87001990e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d770 tx=0x7f86e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.260+0000 7f86fdffb640 1 -- 172.21.15.112:0/986163684 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8017070 con 0x7f8700102b00 2024-05-10T10:41:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.261+0000 7f86fdffb640 1 -- 172.21.15.112:0/986163684 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86e80054b0 con 0x7f8700102b00 2024-05-10T10:41:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.261+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87001a0a60 con 0x7f8700102b00 2024-05-10T10:41:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.261+0000 7f86fdffb640 1 -- 172.21.15.112:0/986163684 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8004c60 con 0x7f8700102b00 2024-05-10T10:41:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.261+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87001a0fb0 con 0x7f8700102b00 2024-05-10T10:41:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f86fdffb640 1 -- 172.21.15.112:0/986163684 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f86e8004e00 con 0x7f8700102b00 2024-05-10T10:41:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 Processor -- start 2024-05-10T10:41:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 -- start start 2024-05-10T10:41:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a572db0 0x55691a5731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a5782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a57ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55691a4d75d0 con 0x55691a578910 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55691a490890 con 0x55691a572db0 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.263+0000 7f870918f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55691a4d3590 con 0x55691a5737a0 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a57ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a572db0 0x55691a5731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a572db0 0x55691a5731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:37558/0 (socket says 172.21.15.112:37558) 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a57ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39192/0 (socket says 172.21.15.112:39192) 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 -- 172.21.15.112:0/2829124998 learned_addr learned my addr 172.21.15.112:0/2829124998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86fffff640 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a5782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 -- 172.21.15.112:0/2829124998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 msgr2=0x55691a5782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a5782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 -- 172.21.15.112:0/2829124998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 msgr2=0x55691a57ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a57ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 -- 172.21.15.112:0/2829124998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691a1e9210 con 0x55691a572db0 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86f7fff640 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a57ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86fdffb640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f86ec0b2920 0x7f86ec0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86fffff640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f86ec0b2920 0x7f86ec0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f8704a0f640 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a572db0 0x55691a5731d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009fd0 tx=0x7f86f80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5498c3862037741b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86fdffb640 1 -- 172.21.15.112:0/986163684 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f86e80e1fb0 con 0x7f8700102b00 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.264+0000 7f86f67fc640 1 -- 172.21.15.112:0/2829124998 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004d20 con 0x55691a572db0 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f86f67fc640 1 -- 172.21.15.112:0/2829124998 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f8004050 con 0x55691a572db0 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f86f67fc640 1 -- 172.21.15.112:0/2829124998 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8005260 con 0x55691a572db0 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 -- 172.21.15.112:0/2829124998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a572db0 msgr2=0x55691a5731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a572db0 0x55691a5731d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009fd0 tx=0x7f86f80118d0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f86fffff640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f86ec0b2920 0x7f86ec0b4de0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f86f0006010 tx=0x7f86f00058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 -- 172.21.15.112:0/2829124998 shutdown_connections 2024-05-10T10:41:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a57ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a5782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 --2- 172.21.15.112:0/2829124998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a572db0 0x55691a5731d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 -- 172.21.15.112:0/2829124998 >> 172.21.15.112:0/2829124998 conn(0x55691a56e420 msgr2=0x55691a56e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 -- 172.21.15.112:0/2829124998 shutdown_connections 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.265+0000 7f870918f740 1 -- 172.21.15.112:0/2829124998 wait complete. 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 -- start start 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 Processor -- start 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a57d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a578910 0x55691a584520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55691a57b8c0 con 0x55691a572db0 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55691a494370 con 0x55691a5737a0 2024-05-10T10:41:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55691a57ba40 con 0x55691a578910 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f870918f740 10 client.0 _ll_register_callbacks cb 0x55691a1e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39202/0 (socket says 172.21.15.112:39202) 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 -- 172.21.15.112:0/665146354 learned_addr learned my addr 172.21.15.112:0/665146354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86f7fff640 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a57d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 -- 172.21.15.112:0/665146354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a578910 msgr2=0x55691a584520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a578910 0x55691a584520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 -- 172.21.15.112:0/665146354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 msgr2=0x55691a57d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a57d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.266+0000 7f86fffff640 1 -- 172.21.15.112:0/665146354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691a2b5620 con 0x55691a572db0 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f86fffff640 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cfa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f86f0000c00 tx=0x7f86f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe0387113b4e542 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f86df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f86df7fe640 1 -- 172.21.15.112:0/665146354 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f004d030 con 0x55691a572db0 2024-05-10T10:41:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f86df7fe640 1 -- 172.21.15.112:0/665146354 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f86f0004310 con 0x55691a572db0 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f8004050 con 0x55691a572db0 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f86df7fe640 1 -- 172.21.15.112:0/665146354 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86f005a050 con 0x55691a572db0 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 10 client.15564 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55691a587570 con 0x55691a572db0 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f86df7fe640 1 -- 172.21.15.112:0/665146354 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f8004050 con 0x55691a572db0 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f86df7fe640 1 -- 172.21.15.112:0/665146354 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f86f003d4c0 con 0x55691a572db0 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 10 client.15564 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 20 client.15564 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a588cd0 0x55691a58b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 4 client.15564 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.267+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55691a4e51e0 con 0x55691a588cd0 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.268+0000 7f86f7fff640 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a588cd0 0x55691a58b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.268+0000 7f86f7fff640 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a588cd0 0x55691a58b140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5378160513fc48a9 server_cookie=db9540dad9584208 in_seq=0 out_seq=0 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.268+0000 7f86df7fe640 10 client.15564 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f86df7fe640 1 -- 172.21.15.112:0/665146354 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55691a4e51e0 con 0x55691a588cd0 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f86df7fe640 10 client.15564 handle_command_reply: tid=0 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 client.15564 shutdown 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 msgr2=0x55691a57cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cfa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f86f0000c00 tx=0x7f86f0009260 comp rx=0 tx=0).stop 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 shutdown_connections 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a578910 0x55691a584520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a588cd0 0x55691a58b140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a57d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 --2- 172.21.15.112:0/665146354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cfa0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f86df7fe640 0 client.15564 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 >> 172.21.15.112:0/665146354 conn(0x55691a56e420 msgr2=0x55691a56d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 shutdown_connections 2024-05-10T10:41:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 1 -- 172.21.15.112:0/665146354 wait complete. 2024-05-10T10:41:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.269+0000 7f870918f740 20 client.15564 trim_cache size 0 max 16384 2024-05-10T10:41:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.284+0000 7f870918f740 1 Processor -- start 2024-05-10T10:41:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f870918f740 1 -- start start 2024-05-10T10:41:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f870918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f870918f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a57d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f870918f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a578910 0x55691a56ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f870918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87001ab3d0 con 0x55691a572db0 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f870918f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87001acf10 con 0x55691a578910 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f870918f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87001abaa0 con 0x55691a5737a0 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39210/0 (socket says 172.21.15.112:39210) 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 -- 172.21.15.112:0/2787761141 learned_addr learned my addr 172.21.15.112:0/2787761141 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86f7fff640 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a57d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 -- 172.21.15.112:0/2787761141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 msgr2=0x55691a57d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a57d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f8704a0f640 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a578910 0x55691a56ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 -- 172.21.15.112:0/2787761141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a578910 msgr2=0x55691a56ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a578910 0x55691a56ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 -- 172.21.15.112:0/2787761141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691a578640 con 0x55691a572db0 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86fffff640 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cbb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f86f003d980 tx=0x7f86f0032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8827bdffb1e28ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86f5ffb640 1 -- 172.21.15.112:0/2787761141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f004d070 con 0x55691a572db0 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86f5ffb640 1 -- 172.21.15.112:0/2787761141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f003db20 con 0x55691a572db0 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.285+0000 7f86f5ffb640 1 -- 172.21.15.112:0/2787761141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0035040 con 0x55691a572db0 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 -- 172.21.15.112:0/2787761141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 msgr2=0x55691a57cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cbb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f86f003d980 tx=0x7f86f0032f70 comp rx=0 tx=0).stop 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 -- 172.21.15.112:0/2787761141 shutdown_connections 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a578910 0x55691a56ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a5737a0 0x55691a57d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 --2- 172.21.15.112:0/2787761141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a572db0 0x55691a57cbb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 -- 172.21.15.112:0/2787761141 >> 172.21.15.112:0/2787761141 conn(0x55691a56e420 msgr2=0x55691a5693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 -- 172.21.15.112:0/2787761141 shutdown_connections 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 -- 172.21.15.112:0/2787761141 wait complete. 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.286+0000 7f870918f740 1 -- start start 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 1 Processor -- start 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a572db0 0x55691a581590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a581ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a582a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87001ac840 con 0x55691a578910 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87001ac170 con 0x55691a5737a0 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87001a93b0 con 0x55691a572db0 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f870918f740 10 client.0 _ll_register_callbacks cb 0x7f86f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a582a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a582a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39222/0 (socket says 172.21.15.112:39222) 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 -- 172.21.15.112:0/1319556820 learned_addr learned my addr 172.21.15.112:0/1319556820 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f86fffff640 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a572db0 0x55691a581590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f86f7fff640 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a581ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 -- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a572db0 msgr2=0x55691a581590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a572db0 0x55691a581590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 -- 172.21.15.112:0/1319556820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 msgr2=0x55691a581ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a581ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 -- 172.21.15.112:0/1319556820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691a583030 con 0x55691a578910 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f8704a0f640 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a582a40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f86f801a000 tx=0x7f86f8003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a087cc3364c1d198 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.287+0000 7f86df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f86df7fe640 1 -- 172.21.15.112:0/1319556820 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8029020 con 0x55691a578910 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f86df7fe640 1 -- 172.21.15.112:0/1319556820 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f86f8005450 con 0x55691a578910 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f003db20 con 0x55691a578910 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f86df7fe640 1 -- 172.21.15.112:0/1319556820 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86f8034050 con 0x55691a578910 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 10 client.15570 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55691a5f3810 con 0x55691a578910 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f86df7fe640 1 -- 172.21.15.112:0/1319556820 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86f003db20 con 0x55691a578910 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f86df7fe640 1 -- 172.21.15.112:0/1319556820 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f86f8025470 con 0x55691a578910 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 10 client.15570 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 20 client.15570 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a5f40a0 0x55691a5f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 4 client.15570 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.288+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55691a4df050 con 0x55691a5f40a0 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.289+0000 7f86fffff640 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a5f40a0 0x55691a5f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.289+0000 7f86df7fe640 10 client.15570 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.289+0000 7f86fffff640 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a5f40a0 0x55691a5f64c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52b220a50e5ab80 server_cookie=af25866b54a49e6c in_seq=0 out_seq=0 2024-05-10T10:41:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f86df7fe640 1 -- 172.21.15.112:0/1319556820 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55691a4df050 con 0x55691a5f40a0 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f86df7fe640 10 client.15570 handle_command_reply: tid=0 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 client.15570 shutdown 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 msgr2=0x55691a582a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a582a40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f86f801a000 tx=0x7f86f8003e10 comp rx=0 tx=0).stop 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 shutdown_connections 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55691a578910 0x55691a582a40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55691a5f40a0 0x55691a5f64c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55691a5737a0 0x55691a581ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 --2- 172.21.15.112:0/1319556820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55691a572db0 0x55691a581590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f86df7fe640 0 client.15570 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 >> 172.21.15.112:0/1319556820 conn(0x55691a56e420 msgr2=0x55691a57cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 shutdown_connections 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 1 -- 172.21.15.112:0/1319556820 wait complete. 2024-05-10T10:41:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f870918f740 20 client.15570 trim_cache size 0 max 16384 2024-05-10T10:41:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:41:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:41:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:41:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f86ec0b2920 msgr2=0x7f86ec0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f8706472640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f86ec0b2920 0x7f86ec0b4de0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f86f0006010 tx=0x7f86f00058b0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700102b00 msgr2=0x7f87001990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.293+0000 7f8706472640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700102b00 0x7f87001990e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d770 tx=0x7f86e800dc30 comp rx=0 tx=0).stop 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 shutdown_connections 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f86ec0b2920 0x7f86ec0b4de0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87001034c0 0x7f87001a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8700102b00 0x7f87001990e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 --2- 172.21.15.112:0/986163684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8700101910 0x7f8700198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 >> 172.21.15.112:0/986163684 conn(0x7f87000fce80 msgr2=0x7f8700105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 shutdown_connections 2024-05-10T10:41:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:11.294+0000 7f8706472640 1 -- 172.21.15.112:0/986163684 wait complete. 2024-05-10T10:41:11.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:41:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:13.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:12 smithi148 ceph-mon[33568]: pgmap v459: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-10T10:41:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:12 smithi195 ceph-mon[33773]: pgmap v459: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-10T10:41:13.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:12 smithi112 ceph-mon[30100]: pgmap v459: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 -- 172.21.15.112:0/1439047575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0106340 msgr2=0x7f37a0106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 --2- 172.21.15.112:0/1439047575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0106340 0x7f37a0106720 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f37900099c0 tx=0x7f3790009cd0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 -- 172.21.15.112:0/1439047575 shutdown_connections 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 --2- 172.21.15.112:0/1439047575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a0108b70 0x7f37a010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 --2- 172.21.15.112:0/1439047575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0106340 0x7f37a0106720 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 --2- 172.21.15.112:0/1439047575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37a01081b0 0x7f37a0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 -- 172.21.15.112:0/1439047575 >> 172.21.15.112:0/1439047575 conn(0x7f37a00776c0 msgr2=0x7f37a0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 -- 172.21.15.112:0/1439047575 shutdown_connections 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.896+0000 7f37a5c33700 1 -- 172.21.15.112:0/1439047575 wait complete. 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 Processor -- start 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 -- start start 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37a0106340 0x7f37a019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01081b0 0x7f37a019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0108b70 0x7f37a0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0112590 con 0x7f37a0108b70 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0112410 con 0x7f37a01081b0 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.897+0000 7f37a5c33700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0112710 con 0x7f37a0106340 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0108b70 0x7f37a0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0108b70 0x7f37a0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39238/0 (socket says 172.21.15.112:39238) 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 -- 172.21.15.112:0/489146129 learned_addr learned my addr 172.21.15.112:0/489146129 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379f7fe700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37a0106340 0x7f37a019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379effd700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01081b0 0x7f37a019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 -- 172.21.15.112:0/489146129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37a0106340 msgr2=0x7f37a019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37a0106340 0x7f37a019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 -- 172.21.15.112:0/489146129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01081b0 msgr2=0x7f37a019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01081b0 0x7f37a019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 -- 172.21.15.112:0/489146129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a0199070 con 0x7f37a0108b70 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379ffff700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0108b70 0x7f37a0198910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f379800b970 tx=0x7f379800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379cff9700 1 -- 172.21.15.112:0/489146129 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379800d860 con 0x7f37a0108b70 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a0199360 con 0x7f37a0108b70 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379cff9700 1 -- 172.21.15.112:0/489146129 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f379800e430 con 0x7f37a0108b70 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.898+0000 7f379cff9700 1 -- 172.21.15.112:0/489146129 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798017660 con 0x7f37a0108b70 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.899+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a0101e60 con 0x7f37a0108b70 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.902+0000 7f379cff9700 1 -- 172.21.15.112:0/489146129 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f379800da00 con 0x7f37a0108b70 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.902+0000 7f37b04c6b80 1 Processor -- start 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.902+0000 7f37b04c6b80 1 -- start start 2024-05-10T10:41:13.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 0x559e653a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e652f0860 con 0x559e653a28e0 2024-05-10T10:41:13.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559e6528f7c0 con 0x559e653a3200 2024-05-10T10:41:13.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e652ec820 con 0x559e653a7040 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39244/0 (socket says 172.21.15.112:39244) 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 -- 172.21.15.112:0/566839356 learned_addr learned my addr 172.21.15.112:0/566839356 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379f7fe700 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379cff9700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f37880b2880 0x7f37880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 0x559e653a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 -- 172.21.15.112:0/566839356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 msgr2=0x559e653a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 0x559e653a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 -- 172.21.15.112:0/566839356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 msgr2=0x559e653a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379ffff700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f37880b2880 0x7f37880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 -- 172.21.15.112:0/566839356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6523d8d0 con 0x559e653a28e0 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379f7fe700 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379cff9700 1 -- 172.21.15.112:0/489146129 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f37980e21d0 con 0x7f37a0108b70 2024-05-10T10:41:13.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.903+0000 7f379effd700 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653a2cc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f379400a590 tx=0x7f379400a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a80e01abbf7be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.904+0000 7f379ffff700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f37880b2880 0x7f37880b4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f37980a93f0 tx=0x7f37980a5b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.904+0000 7f3785ffb700 1 -- 172.21.15.112:0/566839356 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37940131b0 con 0x559e653a28e0 2024-05-10T10:41:13.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.904+0000 7f3785ffb700 1 -- 172.21.15.112:0/566839356 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794004cc0 con 0x559e653a28e0 2024-05-10T10:41:13.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.904+0000 7f3785ffb700 1 -- 172.21.15.112:0/566839356 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794010690 con 0x559e653a28e0 2024-05-10T10:41:13.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 -- 172.21.15.112:0/566839356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 msgr2=0x559e653a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653a2cc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f379400a590 tx=0x7f379400a930 comp rx=0 tx=0).stop 2024-05-10T10:41:13.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 -- 172.21.15.112:0/566839356 shutdown_connections 2024-05-10T10:41:13.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 0x559e653a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653a2cc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/566839356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 -- 172.21.15.112:0/566839356 >> 172.21.15.112:0/566839356 conn(0x559e652ca100 msgr2=0x559e65398c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 -- 172.21.15.112:0/566839356 shutdown_connections 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 -- 172.21.15.112:0/566839356 wait complete. 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.905+0000 7f37b04c6b80 1 -- start start 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 1 Processor -- start 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 0x559e653abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e653af270 con 0x559e653a28e0 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559e65256c30 con 0x559e653a3200 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e653af3f0 con 0x559e653a7040 2024-05-10T10:41:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f37b04c6b80 10 client.0 _ll_register_callbacks cb 0x559e65288920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39260/0 (socket says 172.21.15.112:39260) 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 -- 172.21.15.112:0/3628330849 learned_addr learned my addr 172.21.15.112:0/3628330849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379f7fe700 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 -- 172.21.15.112:0/3628330849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 msgr2=0x559e653abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 0x559e653abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 -- 172.21.15.112:0/3628330849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 msgr2=0x559e653ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.906+0000 7f379effd700 1 -- 172.21.15.112:0/3628330849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6528c690 con 0x559e653a28e0 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f379f7fe700 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f379effd700 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653ae4b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3794005950 tx=0x7f3794005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bee7f091e2e720 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f377effd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f377effd700 1 -- 172.21.15.112:0/3628330849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379402b060 con 0x559e653a28e0 2024-05-10T10:41:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f377effd700 1 -- 172.21.15.112:0/3628330849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3794004960 con 0x559e653a28e0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3794004cc0 con 0x559e653a28e0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f377effd700 1 -- 172.21.15.112:0/3628330849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3794032050 con 0x559e653a28e0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f37b04c6b80 10 client.15582 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e653b9ab0 con 0x559e653a28e0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f377effd700 1 -- 172.21.15.112:0/3628330849 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794004cc0 con 0x559e653a28e0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.907+0000 7f377effd700 1 -- 172.21.15.112:0/3628330849 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3794010430 con 0x559e653a28e0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f37b04c6b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f37b04c6b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f37b04c6b80 20 client.15582 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e653bbfe0 0x559e653be3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f37b04c6b80 4 client.15582 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6536a650 con 0x559e653bbfe0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f379f7fe700 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e653bbfe0 0x559e653be3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f377effd700 10 client.15582 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.908+0000 7f379f7fe700 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e653bbfe0 0x559e653be3d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c934194ad203cde8 server_cookie=5b2dd77ed087abd6 in_seq=0 out_seq=0 2024-05-10T10:41:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.909+0000 7f377effd700 1 -- 172.21.15.112:0/3628330849 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e6536a650 con 0x559e653bbfe0 2024-05-10T10:41:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.909+0000 7f377effd700 10 client.15582 handle_command_reply: tid=0 2024-05-10T10:41:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.909+0000 7f37b04c6b80 1 client.15582 shutdown 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 msgr2=0x559e653ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653ae4b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3794005950 tx=0x7f3794005da0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 shutdown_connections 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e653bbfe0 0x559e653be3d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a7040 0x559e653abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653ae4b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/3628330849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a3200 0x559e653ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f377effd700 0 client.15582 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 >> 172.21.15.112:0/3628330849 conn(0x559e652ca100 msgr2=0x559e65393720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 shutdown_connections 2024-05-10T10:41:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 1 -- 172.21.15.112:0/3628330849 wait complete. 2024-05-10T10:41:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.910+0000 7f37b04c6b80 20 client.15582 trim_cache size 0 max 16384 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 Processor -- start 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 -- start start 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a7040 0x559e653935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01ad0c0 con 0x559e653a28e0 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01aec00 con 0x559e653a7040 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01ad790 con 0x559e653a3200 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39276/0 (socket says 172.21.15.112:39276) 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 -- 172.21.15.112:0/1835547961 learned_addr learned my addr 172.21.15.112:0/1835547961 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379f7fe700 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 -- 172.21.15.112:0/1835547961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 msgr2=0x559e653a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 -- 172.21.15.112:0/1835547961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a7040 msgr2=0x559e653935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a7040 0x559e653935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 -- 172.21.15.112:0/1835547961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e653ac0e0 con 0x559e653a28e0 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.935+0000 7f379effd700 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653b8300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3794010830 tx=0x7f379400a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798dd5e6b77a243 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37857fa700 1 -- 172.21.15.112:0/1835547961 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379402b040 con 0x559e653a28e0 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37857fa700 1 -- 172.21.15.112:0/1835547961 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794010ba0 con 0x559e653a28e0 2024-05-10T10:41:13.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37857fa700 1 -- 172.21.15.112:0/1835547961 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794010690 con 0x559e653a28e0 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37b04c6b80 1 -- 172.21.15.112:0/1835547961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 msgr2=0x559e653b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653b8300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3794010830 tx=0x7f379400a990 comp rx=0 tx=0).stop 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37b04c6b80 1 -- 172.21.15.112:0/1835547961 shutdown_connections 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a7040 0x559e653935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a28e0 0x559e653b8300 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/1835547961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.936+0000 7f37b04c6b80 1 -- 172.21.15.112:0/1835547961 >> 172.21.15.112:0/1835547961 conn(0x559e652ca100 msgr2=0x559e65392e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 -- 172.21.15.112:0/1835547961 shutdown_connections 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 -- 172.21.15.112:0/1835547961 wait complete. 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 -- start start 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 Processor -- start 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a28e0 0x559e653b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a7040 0x559e653c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01ae530 con 0x559e653a7040 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01ade60 con 0x559e653a28e0 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.937+0000 7f37b04c6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01ab060 con 0x559e653a3200 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f37b04c6b80 10 client.0 _ll_register_callbacks cb 0x559e65288920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a28e0 0x559e653b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a28e0 0x559e653b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:37618/0 (socket says 172.21.15.112:37618) 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 -- 172.21.15.112:0/304430410 learned_addr learned my addr 172.21.15.112:0/304430410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:13.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a7040 0x559e653c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379f7fe700 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 -- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 msgr2=0x559e653c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 -- 172.21.15.112:0/304430410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a7040 msgr2=0x559e653c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a7040 0x559e653c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 -- 172.21.15.112:0/304430410 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e653c3a40 con 0x559e653a28e0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f379effd700 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a28e0 0x559e653b4a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3794005b80 tx=0x7f3794020040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dfc849662d9c49a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f377effd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f377effd700 1 -- 172.21.15.112:0/304430410 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379402b070 con 0x559e653a28e0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3794010ba0 con 0x559e653a28e0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.938+0000 7f377effd700 1 -- 172.21.15.112:0/304430410 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3794004960 con 0x559e653a28e0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f377effd700 1 -- 172.21.15.112:0/304430410 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3794013b40 con 0x559e653a28e0 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 10 client.25076 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:13.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e654264d0 con 0x559e653a28e0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f377effd700 1 -- 172.21.15.112:0/304430410 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794010ba0 con 0x559e653a28e0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f377effd700 1 -- 172.21.15.112:0/304430410 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f379401e9e0 con 0x559e653a28e0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 10 client.25076 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 10 client.25076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 20 client.25076 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e65426ad0 0x559e65428ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 4 client.25076 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e652fe240 con 0x559e65426ad0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.939+0000 7f379f7fe700 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e65426ad0 0x559e65428ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f377effd700 10 client.25076 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f379f7fe700 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e65426ad0 0x559e65428ec0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ca4fd8d55afb6 server_cookie=86996b3eea3c92d2 in_seq=0 out_seq=0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f377effd700 1 -- 172.21.15.112:0/304430410 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e652fe240 con 0x559e65426ad0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f377effd700 10 client.25076 handle_command_reply: tid=0 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 client.25076 shutdown 2024-05-10T10:41:13.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a28e0 msgr2=0x559e653b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a28e0 0x559e653b4a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3794005b80 tx=0x7f3794020040 comp rx=0 tx=0).stop 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 shutdown_connections 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e65426ad0 0x559e65428ec0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e653a7040 0x559e653c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f377effd700 0 client.25076 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e653a28e0 0x559e653b4a80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 --2- 172.21.15.112:0/304430410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e653a3200 0x559e653c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 >> 172.21.15.112:0/304430410 conn(0x559e652ca100 msgr2=0x559e6539d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 shutdown_connections 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.940+0000 7f37b04c6b80 1 -- 172.21.15.112:0/304430410 wait complete. 2024-05-10T10:41:13.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37b04c6b80 20 client.25076 trim_cache size 0 max 16384 2024-05-10T10:41:13.947 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:13.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f37880b2880 msgr2=0x7f37880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f37880b2880 0x7f37880b4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f37980a93f0 tx=0x7f37980a5b30 comp rx=0 tx=0).stop 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0108b70 msgr2=0x7f37a0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0108b70 0x7f37a0198910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f379800b970 tx=0x7f379800bc80 comp rx=0 tx=0).stop 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 shutdown_connections 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37a0108b70 0x7f37a0198910 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f37880b2880 0x7f37880b4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37a0106340 0x7f37a019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 --2- 172.21.15.112:0/489146129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01081b0 0x7f37a019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.941+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 >> 172.21.15.112:0/489146129 conn(0x7f37a00776c0 msgr2=0x7f37a00fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.942+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 shutdown_connections 2024-05-10T10:41:13.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:13.942+0000 7f37a5c33700 1 -- 172.21.15.112:0/489146129 wait complete. 2024-05-10T10:41:14.822 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:14 smithi112 ceph-mon[30100]: pgmap v460: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-10T10:41:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:14 smithi148 ceph-mon[33568]: pgmap v460: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-10T10:41:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:14 smithi195 ceph-mon[33773]: pgmap v460: 129 pgs: 129 active+clean; 671 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-10T10:41:15.849 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:41:15.849 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:41:16.111 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:16.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:16 smithi112 ceph-mon[30100]: pgmap v461: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:17.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:16 smithi148 ceph-mon[33568]: pgmap v461: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:17.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:16 smithi195 ceph-mon[33773]: pgmap v461: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 -- 172.21.15.112:0/650492786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc103540 msgr2=0x7f01fc1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 --2- 172.21.15.112:0/650492786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc103540 0x7f01fc1039c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0099c0 tx=0x7f01ec009cd0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 -- 172.21.15.112:0/650492786 shutdown_connections 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 --2- 172.21.15.112:0/650492786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01fc103f00 0x7f01fc1103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 --2- 172.21.15.112:0/650492786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01fc1095e0 0x7f01fc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 --2- 172.21.15.112:0/650492786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc103540 0x7f01fc1039c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 -- 172.21.15.112:0/650492786 >> 172.21.15.112:0/650492786 conn(0x7f01fc07a650 msgr2=0x7f01fc07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 -- 172.21.15.112:0/650492786 shutdown_connections 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.940+0000 7f020403b700 1 -- 172.21.15.112:0/650492786 wait complete. 2024-05-10T10:41:17.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 Processor -- start 2024-05-10T10:41:17.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 -- start start 2024-05-10T10:41:17.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01fc103540 0x7f01fc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01fc103f00 0x7f01fc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc1095e0 0x7f01fc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc113590 con 0x7f01fc1095e0 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc113410 con 0x7f01fc103540 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f020403b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc113710 con 0x7f01fc103f00 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f02025d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc1095e0 0x7f01fc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f02025d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc1095e0 0x7f01fc076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42146/0 (socket says 172.21.15.112:42146) 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f02025d8700 1 -- 172.21.15.112:0/2351546271 learned_addr learned my addr 172.21.15.112:0/2351546271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f0201dd7700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01fc103540 0x7f01fc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.941+0000 7f02015d6700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01fc103f00 0x7f01fc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f02025d8700 1 -- 172.21.15.112:0/2351546271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01fc103f00 msgr2=0x7f01fc075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f02025d8700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01fc103f00 0x7f01fc075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f02025d8700 1 -- 172.21.15.112:0/2351546271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01fc103540 msgr2=0x7f01fc0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f02025d8700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01fc103540 0x7f01fc0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f02025d8700 1 -- 172.21.15.112:0/2351546271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc1a7ff0 con 0x7f01fc1095e0 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f02025d8700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc1095e0 0x7f01fc076220 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f01f400d980 tx=0x7f01f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f01f2ffd700 1 -- 172.21.15.112:0/2351546271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4017070 con 0x7f01fc1095e0 2024-05-10T10:41:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f01f2ffd700 1 -- 172.21.15.112:0/2351546271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01f4005020 con 0x7f01fc1095e0 2024-05-10T10:41:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc1a82e0 con 0x7f01fc1095e0 2024-05-10T10:41:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f01f2ffd700 1 -- 172.21.15.112:0/2351546271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4009990 con 0x7f01fc1095e0 2024-05-10T10:41:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.942+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc1a87a0 con 0x7f01fc1095e0 2024-05-10T10:41:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.944+0000 7f01f2ffd700 1 -- 172.21.15.112:0/2351546271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f01f4004a30 con 0x7f01fc1095e0 2024-05-10T10:41:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f01f2ffd700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01e80b2880 0x7f01e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f0201dd7700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01e80b2880 0x7f01e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f01f2ffd700 1 -- 172.21.15.112:0/2351546271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f01f40e2390 con 0x7f01fc1095e0 2024-05-10T10:41:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 Processor -- start 2024-05-10T10:41:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 -- start start 2024-05-10T10:41:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f0201dd7700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01e80b2880 0x7f01e80b4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007fd0 tx=0x7f01f800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61995610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995b50 0x561b6198f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b61990210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b618e19c0 con 0x561b61995b50 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561b61846e20 con 0x561b6198fdd0 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.946+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561b618dd980 con 0x561b61995230 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995b50 0x561b6198f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995b50 0x561b6198f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42156/0 (socket says 172.21.15.112:42156) 2024-05-10T10:41:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 -- 172.21.15.112:0/3939098227 learned_addr learned my addr 172.21.15.112:0/3939098227 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f0201dd7700 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61995610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02025d8700 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b61990210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 -- 172.21.15.112:0/3939098227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 msgr2=0x561b61995610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61995610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 -- 172.21.15.112:0/3939098227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 msgr2=0x561b61990210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b61990210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 -- 172.21.15.112:0/3939098227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6182e950 con 0x561b61995b50 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02025d8700 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b61990210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f0201dd7700 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61995610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f02015d6700 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995b50 0x561b6198f890 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006010 tx=0x7f01ec009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb33b1d470f4e852 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f01e7fff700 1 -- 172.21.15.112:0/3939098227 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec027070 con 0x561b61995b50 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f01e7fff700 1 -- 172.21.15.112:0/3939098227 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01ec00bca0 con 0x561b61995b50 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.947+0000 7f01e7fff700 1 -- 172.21.15.112:0/3939098227 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec003aa0 con 0x561b61995b50 2024-05-10T10:41:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3939098227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995b50 msgr2=0x561b6198f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995b50 0x561b6198f890 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006010 tx=0x7f01ec009510 comp rx=0 tx=0).stop 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3939098227 shutdown_connections 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b61990210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61995610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3939098227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995b50 0x561b6198f890 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3939098227 >> 172.21.15.112:0/3939098227 conn(0x561b618bbbf0 msgr2=0x561b6198a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3939098227 shutdown_connections 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.948+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3939098227 wait complete. 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 -- start start 2024-05-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 Processor -- start 2024-05-10T10:41:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6198fdd0 0x561b6199b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61998be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 0x561b61999120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b6199c470 con 0x561b6198fdd0 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561b6189af50 con 0x561b61995b50 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561b6199c5f0 con 0x561b61995230 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f020e8ceb80 10 client.0 _ll_register_callbacks cb 0x561b61875630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f0201dd7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6198fdd0 0x561b6199b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f0201dd7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6198fdd0 0x561b6199b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42170/0 (socket says 172.21.15.112:42170) 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f0201dd7700 1 -- 172.21.15.112:0/1987231023 learned_addr learned my addr 172.21.15.112:0/1987231023 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.949+0000 7f02025d8700 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 0x561b61999120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f02015d6700 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61998be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f0201dd7700 1 -- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 msgr2=0x561b61998be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f0201dd7700 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61998be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f0201dd7700 1 -- 172.21.15.112:0/1987231023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 msgr2=0x561b61999120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f0201dd7700 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 0x561b61999120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f0201dd7700 1 -- 172.21.15.112:0/1987231023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b618987e0 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f0201dd7700 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6198fdd0 0x561b6199b840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f01f8014440 tx=0x7f01f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44e283a1d3014e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f01e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f01e57fa700 1 -- 172.21.15.112:0/1987231023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8026030 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f01e57fa700 1 -- 172.21.15.112:0/1987231023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f01f8014c90 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec00bca0 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f01e57fa700 1 -- 172.21.15.112:0/1987231023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01f8030050 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f020e8ceb80 10 client.15600 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.950+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b619a6da0 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f01e57fa700 1 -- 172.21.15.112:0/1987231023 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01ec00bca0 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f01e57fa700 1 -- 172.21.15.112:0/1987231023 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f01f8022b30 con 0x561b6198fdd0 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f020e8ceb80 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:17.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f020e8ceb80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f020e8ceb80 20 client.15600 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b619a93e0 0x561b619ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f020e8ceb80 4 client.15600 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b618ef3a0 con 0x561b619a93e0 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.951+0000 7f02015d6700 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b619a93e0 0x561b619ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.952+0000 7f01e57fa700 10 client.15600 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.952+0000 7f02015d6700 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b619a93e0 0x561b619ab810 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ede80afe82cfcd8 server_cookie=ae5a4ee59c7cfedf in_seq=0 out_seq=0 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.952+0000 7f01e57fa700 1 -- 172.21.15.112:0/1987231023 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b618ef3a0 con 0x561b619a93e0 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.952+0000 7f01e57fa700 10 client.15600 handle_command_reply: tid=0 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.952+0000 7f020e8ceb80 1 client.15600 shutdown 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6198fdd0 msgr2=0x561b6199b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6198fdd0 0x561b6199b840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f01f8014440 tx=0x7f01f8004ad0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 shutdown_connections 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b619a93e0 0x561b619ab810 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 0x561b61999120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b6198fdd0 0x561b6199b840 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1987231023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995230 0x561b61998be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f01e57fa700 0 client.15600 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 >> 172.21.15.112:0/1987231023 conn(0x561b618bbbf0 msgr2=0x561b61984bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 shutdown_connections 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1987231023 wait complete. 2024-05-10T10:41:17.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.953+0000 7f020e8ceb80 20 client.15600 trim_cache size 0 max 16384 2024-05-10T10:41:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 Processor -- start 2024-05-10T10:41:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 -- start start 2024-05-10T10:41:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b6198fdd0 0x561b619a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b61984cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 0x561b619b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc1b12b0 con 0x561b61995230 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc1b2df0 con 0x561b61995b50 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc1b1980 con 0x561b6198fdd0 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b61984cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b61984cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42180/0 (socket says 172.21.15.112:42180) 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 -- 172.21.15.112:0/1929878790 learned_addr learned my addr 172.21.15.112:0/1929878790 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f0201dd7700 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b6198fdd0 0x561b619a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 -- 172.21.15.112:0/1929878790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b6198fdd0 msgr2=0x561b619a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b6198fdd0 0x561b619a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 -- 172.21.15.112:0/1929878790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 msgr2=0x561b619b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 0x561b619b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.977+0000 7f02015d6700 1 -- 172.21.15.112:0/1929878790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6198aef0 con 0x561b61995230 2024-05-10T10:41:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f02015d6700 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b61984cc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b580 tx=0x7f01ec005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e22883f61e0e04 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f01e77fe700 1 -- 172.21.15.112:0/1929878790 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec034020 con 0x561b61995230 2024-05-10T10:41:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f01e77fe700 1 -- 172.21.15.112:0/1929878790 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01ec028040 con 0x561b61995230 2024-05-10T10:41:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f01e77fe700 1 -- 172.21.15.112:0/1929878790 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec027030 con 0x561b61995230 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1929878790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 msgr2=0x561b61984cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b61984cc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b580 tx=0x7f01ec005040 comp rx=0 tx=0).stop 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1929878790 shutdown_connections 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b61995b50 0x561b619b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b6198fdd0 0x561b619a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.978+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/1929878790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b61984cc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1929878790 >> 172.21.15.112:0/1929878790 conn(0x561b618bbbf0 msgr2=0x561b61994560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1929878790 shutdown_connections 2024-05-10T10:41:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 -- 172.21.15.112:0/1929878790 wait complete. 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 -- start start 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 Processor -- start 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b619a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b6199e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995b50 0x561b6199edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc1b2720 con 0x561b61995230 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc1b2050 con 0x561b6198fdd0 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc1af250 con 0x561b61995b50 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.979+0000 7f020e8ceb80 10 client.0 _ll_register_callbacks cb 0x561b61875630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b6199e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b6199e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42198/0 (socket says 172.21.15.112:42198) 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 -- 172.21.15.112:0/3015282277 learned_addr learned my addr 172.21.15.112:0/3015282277 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f0201dd7700 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b619a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02025d8700 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995b50 0x561b6199edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 -- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995b50 msgr2=0x561b6199edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995b50 0x561b6199edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 -- 172.21.15.112:0/3015282277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 msgr2=0x561b619a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b619a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 -- 172.21.15.112:0/3015282277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6199f600 con 0x561b61995230 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f02015d6700 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b6199e880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0098e0 tx=0x7f01ec009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a98f4adf572073 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f01e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f01e57fa700 1 -- 172.21.15.112:0/3015282277 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec034070 con 0x561b61995230 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f01e57fa700 1 -- 172.21.15.112:0/3015282277 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f01ec0052c0 con 0x561b61995230 2024-05-10T10:41:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec028040 con 0x561b61995230 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f01e57fa700 1 -- 172.21.15.112:0/3015282277 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01ec027040 con 0x561b61995230 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.980+0000 7f020e8ceb80 10 client.15606 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b61a14c90 con 0x561b61995230 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f01e57fa700 1 -- 172.21.15.112:0/3015282277 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01ec028040 con 0x561b61995230 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f01e57fa700 1 -- 172.21.15.112:0/3015282277 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f01ec039070 con 0x561b61995230 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f020e8ceb80 10 client.15606 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f020e8ceb80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f020e8ceb80 20 client.15606 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b61a14e60 0x561b61a17250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f020e8ceb80 4 client.15606 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:41:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561b618e9440 con 0x561b61a14e60 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.981+0000 7f0201dd7700 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b61a14e60 0x561b61a17250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.982+0000 7f01e57fa700 10 client.15606 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.982+0000 7f0201dd7700 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b61a14e60 0x561b61a17250 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ec32241befeac4 server_cookie=55da2f8577646bc0 in_seq=0 out_seq=0 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f01e57fa700 1 -- 172.21.15.112:0/3015282277 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561b618e9440 con 0x561b61a14e60 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f01e57fa700 10 client.15606 handle_command_reply: tid=0 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 client.15606 shutdown 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 msgr2=0x561b6199e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b6199e880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0098e0 tx=0x7f01ec009cd0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 shutdown_connections 2024-05-10T10:41:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x561b61a14e60 0x561b61a17250 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b61995b50 0x561b6199edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561b6198fdd0 0x561b619a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f01e57fa700 0 client.15606 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 --2- 172.21.15.112:0/3015282277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b61995230 0x561b6199e880 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 >> 172.21.15.112:0/3015282277 conn(0x561b618bbbf0 msgr2=0x561b6199a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 shutdown_connections 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 1 -- 172.21.15.112:0/3015282277 wait complete. 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.983+0000 7f020e8ceb80 20 client.15606 trim_cache size 0 max 16384 2024-05-10T10:41:17.990 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:41:17.991 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:41:17.991 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "4046f93c-a4d5-451b-902d-a2ef114935ba", 2024-05-10T10:41:17.991 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:41:17.991 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01e80b2880 msgr2=0x7f01e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01e80b2880 0x7f01e80b4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007fd0 tx=0x7f01f800c040 comp rx=0 tx=0).stop 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc1095e0 msgr2=0x7f01fc076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc1095e0 0x7f01fc076220 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f01f400d980 tx=0x7f01f400dc90 comp rx=0 tx=0).stop 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 shutdown_connections 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc1095e0 0x7f01fc076220 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01fc103540 0x7f01fc0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01e80b2880 0x7f01e80b4d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 --2- 172.21.15.112:0/2351546271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01fc103f00 0x7f01fc075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 >> 172.21.15.112:0/2351546271 conn(0x7f01fc07a650 msgr2=0x7f01fc102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 shutdown_connections 2024-05-10T10:41:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:17.984+0000 7f020403b700 1 -- 172.21.15.112:0/2351546271 wait complete. 2024-05-10T10:41:18.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:41:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:18 smithi148 ceph-mon[33568]: pgmap v462: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:18 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:41:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:18 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:18 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:41:19.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:18 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:18 smithi195 ceph-mon[33773]: pgmap v462: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:18 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:41:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:18 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:18 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:41:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:18 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:18 smithi112 ceph-mon[30100]: pgmap v462: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:18 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:41:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:18 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:18 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:41:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:18 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:19.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:19 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:41:20.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:19 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:41:20.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:19 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:41:20.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:20 smithi112 ceph-mon[30100]: pgmap v463: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:21.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/3985168367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4108400 msgr2=0x7f6ed41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/3985168367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4108400 0x7f6ed41087e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009990 tx=0x7f6ec4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/3985168367 shutdown_connections 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/3985168367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ed4102d50 0x7f6ed410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/3985168367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4108400 0x7f6ed41087e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/3985168367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ed4102390 0x7f6ed4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/3985168367 >> 172.21.15.112:0/3985168367 conn(0x7f6ed4079310 msgr2=0x7f6ed4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/3985168367 shutdown_connections 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.021+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/3985168367 wait complete. 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.022+0000 7f6ed8bb5700 1 Processor -- start 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.022+0000 7f6ed8bb5700 1 -- start start 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed8bb5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ed4102390 0x7f6ed419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed8bb5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4102d50 0x7f6ed419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed8bb5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ed4108400 0x7f6ed41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed8bb5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4112450 con 0x7f6ed4102d50 2024-05-10T10:41:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed8bb5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41122d0 con 0x7f6ed4108400 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed8bb5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41125d0 con 0x7f6ed4102390 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4102d50 0x7f6ed419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4102d50 0x7f6ed419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42220/0 (socket says 172.21.15.112:42220) 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/1988716182 learned_addr learned my addr 172.21.15.112:0/1988716182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed259c700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ed4102390 0x7f6ed419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ed4108400 0x7f6ed41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/1988716182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ed4102390 msgr2=0x7f6ed419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ed4102390 0x7f6ed419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/1988716182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ed4108400 msgr2=0x7f6ed41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ed4108400 0x7f6ed41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/1988716182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed419a030 con 0x7f6ed4102d50 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4102d50 0x7f6ed419fc50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b530 tx=0x7f6ec800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ec37fe700 1 -- 172.21.15.112:0/1988716182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8015020 con 0x7f6ed4102d50 2024-05-10T10:41:21.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.023+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed419a320 con 0x7f6ed4102d50 2024-05-10T10:41:21.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.024+0000 7f6ec37fe700 1 -- 172.21.15.112:0/1988716182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ec8011cc0 con 0x7f6ed4102d50 2024-05-10T10:41:21.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.024+0000 7f6ec37fe700 1 -- 172.21.15.112:0/1988716182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8010dc0 con 0x7f6ed4102d50 2024-05-10T10:41:21.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.024+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed41a65e0 con 0x7f6ed4102d50 2024-05-10T10:41:21.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.026+0000 7f6ec37fe700 1 -- 172.21.15.112:0/1988716182 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6ec8018020 con 0x7f6ed4102d50 2024-05-10T10:41:21.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.027+0000 7f6ec37fe700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ebc0b2880 0x7f6ebc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.027+0000 7f6ee3448b80 1 Processor -- start 2024-05-10T10:41:21.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.027+0000 7f6ed259c700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ebc0b2880 0x7f6ebc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.027+0000 7f6ec37fe700 1 -- 172.21.15.112:0/1988716182 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6ec80e1e10 con 0x7f6ed4102d50 2024-05-10T10:41:21.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.027+0000 7f6ee3448b80 1 -- start start 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8fd8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd88a0 0x5635f8fd2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fd3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5635f8f20860 con 0x5635f8fd7f80 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5635f8ebf7c0 con 0x5635f8fd2c90 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5635f8f1c820 con 0x5635f8fd88a0 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8fd8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8fd8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42222/0 (socket says 172.21.15.112:42222) 2024-05-10T10:41:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 -- 172.21.15.112:0/2577686926 learned_addr learned my addr 172.21.15.112:0/2577686926 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ebc0b2880 0x7f6ebc0b4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400b5c0 tx=0x7f6ec401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fd3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd88a0 0x5635f8fd2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 -- 172.21.15.112:0/2577686926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd88a0 msgr2=0x5635f8fd2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd88a0 0x5635f8fd2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 -- 172.21.15.112:0/2577686926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 msgr2=0x5635f8fd3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fd3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 -- 172.21.15.112:0/2577686926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f8e6d8d0 con 0x5635f8fd7f80 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd88a0 0x5635f8fd2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fd3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.028+0000 7f6ed259c700 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8fd8360 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4015000 tx=0x7f6ec401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f30552bff8fed6d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ec0ff9700 1 -- 172.21.15.112:0/2577686926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4003980 con 0x5635f8fd7f80 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ec0ff9700 1 -- 172.21.15.112:0/2577686926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ec400b7d0 con 0x5635f8fd7f80 2024-05-10T10:41:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ec0ff9700 1 -- 172.21.15.112:0/2577686926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec400f820 con 0x5635f8fd7f80 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2577686926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 msgr2=0x5635f8fd8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8fd8360 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4015000 tx=0x7f6ec401f000 comp rx=0 tx=0).stop 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2577686926 shutdown_connections 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fd3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8fd8360 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2577686926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd88a0 0x5635f8fd2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.029+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2577686926 >> 172.21.15.112:0/2577686926 conn(0x5635f8efa100 msgr2=0x5635f8fc8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2577686926 shutdown_connections 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2577686926 wait complete. 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 -- start start 2024-05-10T10:41:21.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 Processor -- start 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fde4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd7f80 0x5635f8fdb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fdbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5635f8fdf260 con 0x5635f8fd88a0 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5635f8e86c30 con 0x5635f8fd7f80 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5635f8fdf3e0 con 0x5635f8fd2c90 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ee3448b80 10 client.0 _ll_register_callbacks cb 0x5635f8eb8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ed2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fdbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ed2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fdbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42238/0 (socket says 172.21.15.112:42238) 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.030+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/2514469075 learned_addr learned my addr 172.21.15.112:0/2514469075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd7f80 0x5635f8fdb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed259c700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fde4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 msgr2=0x5635f8fde4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fde4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/2514469075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd7f80 msgr2=0x5635f8fdb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd7f80 0x5635f8fdb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/2514469075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f8ebc690 con 0x5635f8fd88a0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed259c700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fde4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fdbdd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc007b70 tx=0x7f6ecc007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c495e509878b6b62 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/2514469075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc01a020 con 0x5635f8fd88a0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/2514469075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6ecc011930 con 0x5635f8fd88a0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec400b7d0 con 0x5635f8fd88a0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/2514469075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ecc025050 con 0x5635f8fd88a0 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ee3448b80 10 client.15618 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.031+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5635f8fe9a80 con 0x5635f8fd88a0 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/2514469075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ec400b7d0 con 0x5635f8fd88a0 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/2514469075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6ecc016c70 con 0x5635f8fd88a0 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6ee3448b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6ee3448b80 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6ee3448b80 20 client.15618 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f8fec0c0 0x5635f8fee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6ee3448b80 4 client.15618 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635f8f9a650 con 0x5635f8fec0c0 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.032+0000 7f6ed259c700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f8fec0c0 0x5635f8fee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.033+0000 7f6ed259c700 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f8fec0c0 0x5635f8fee4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8bb522c81614bc server_cookie=1572c129ec04abcb in_seq=0 out_seq=0 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.033+0000 7f6eb9ffb700 10 client.15618 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.033+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/2514469075 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5635f8f9a650 con 0x5635f8fec0c0 2024-05-10T10:41:21.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.033+0000 7f6eb9ffb700 10 client.15618 handle_command_reply: tid=0 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.033+0000 7f6ee3448b80 1 client.15618 shutdown 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 msgr2=0x5635f8fdbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fdbdd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc007b70 tx=0x7f6ecc007e80 comp rx=0 tx=0).stop 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 shutdown_connections 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f8fec0c0 0x5635f8fee4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fdbdd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fde4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/2514469075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd7f80 0x5635f8fdb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 >> 172.21.15.112:0/2514469075 conn(0x5635f8efa100 msgr2=0x5635f8fc3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6eb9ffb700 0 client.15618 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 shutdown_connections 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 1 -- 172.21.15.112:0/2514469075 wait complete. 2024-05-10T10:41:21.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.034+0000 7f6ee3448b80 20 client.15618 trim_cache size 0 max 16384 2024-05-10T10:41:21.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:20 smithi148 ceph-mon[33568]: pgmap v463: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.054+0000 7f6ee3448b80 1 Processor -- start 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.054+0000 7f6ee3448b80 1 -- start start 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fdb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8ff3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd88a0 0x5635f8ff42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41acf70 con 0x5635f8fd7f80 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41aeab0 con 0x5635f8fd88a0 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41ad640 con 0x5635f8fd2c90 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8ff3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8ff3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42250/0 (socket says 172.21.15.112:42250) 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/639963454 learned_addr learned my addr 172.21.15.112:0/639963454 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:21.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd88a0 0x5635f8ff42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed259c700 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fdb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/639963454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 msgr2=0x5635f8fdb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fdb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/639963454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd88a0 msgr2=0x5635f8ff42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd88a0 0x5635f8ff42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 -- 172.21.15.112:0/639963454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f8fc5810 con 0x5635f8fd7f80 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd88a0 0x5635f8ff42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.055+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8ff3da0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80a5120 tx=0x7f6ec8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6772dc972e16bb01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ebbfff700 1 -- 172.21.15.112:0/639963454 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80a97a0 con 0x5635f8fd7f80 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ebbfff700 1 -- 172.21.15.112:0/639963454 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ec80c3070 con 0x5635f8fd7f80 2024-05-10T10:41:21.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ebbfff700 1 -- 172.21.15.112:0/639963454 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80ad0d0 con 0x5635f8fd7f80 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 -- 172.21.15.112:0/639963454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 msgr2=0x5635f8ff3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8ff3da0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80a5120 tx=0x7f6ec8000bc0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 -- 172.21.15.112:0/639963454 shutdown_connections 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd88a0 0x5635f8ff42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd2c90 0x5635f8fdb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/639963454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd7f80 0x5635f8ff3da0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 -- 172.21.15.112:0/639963454 >> 172.21.15.112:0/639963454 conn(0x5635f8efa100 msgr2=0x5635f8fdb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 -- 172.21.15.112:0/639963454 shutdown_connections 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.056+0000 7f6ee3448b80 1 -- 172.21.15.112:0/639963454 wait complete. 2024-05-10T10:41:21.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 -- start start 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 Processor -- start 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fe4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd7f80 0x5635f8fe0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fe07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41ae3e0 con 0x5635f8fd88a0 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41add10 con 0x5635f8fd2c90 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41aaf10 con 0x5635f8fd7f80 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ee3448b80 10 client.0 _ll_register_callbacks cb 0x5635f8eb8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ed2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fe07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ed2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fe07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42262/0 (socket says 172.21.15.112:42262) 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/1368859120 learned_addr learned my addr 172.21.15.112:0/1368859120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ed1d9b700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd7f80 0x5635f8fe0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.057+0000 7f6ed259c700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fe4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd7f80 msgr2=0x5635f8fe0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd7f80 0x5635f8fe0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/1368859120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 msgr2=0x5635f8fe4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fe4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ed2d9d700 1 -- 172.21.15.112:0/1368859120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f9055950 con 0x5635f8fd88a0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ed259c700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fe4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ed2d9d700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fe07d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc007840 tx=0x7f6ecc015a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598b9d75d3f0488f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/1368859120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc02d070 con 0x5635f8fd88a0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/1368859120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6ecc015aa0 con 0x5635f8fd88a0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec80c3070 con 0x5635f8fd88a0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/1368859120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ecc039050 con 0x5635f8fd88a0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ee3448b80 10 client.15624 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.058+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5635f9056af0 con 0x5635f8fd88a0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/1368859120 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ecc0098c0 con 0x5635f8fd88a0 2024-05-10T10:41:21.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/1368859120 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6ecc016450 con 0x5635f8fd88a0 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ee3448b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ee3448b80 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ee3448b80 20 client.15624 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f9057170 0x5635f9059590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ee3448b80 4 client.15624 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6ec80c3070 con 0x5635f9057170 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ed259c700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f9057170 0x5635f9059590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6eb9ffb700 10 client.15624 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.059+0000 7f6ed259c700 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f9057170 0x5635f9059590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d3729beac6ba4d server_cookie=5da4a2ea50f67df0 in_seq=0 out_seq=0 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6eb9ffb700 1 -- 172.21.15.112:0/1368859120 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f6ec80c3070 con 0x5635f9057170 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6eb9ffb700 10 client.15624 handle_command_reply: tid=0 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 client.15624 shutdown 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 msgr2=0x5635f8fe07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fe07d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc007840 tx=0x7f6ecc015a20 comp rx=0 tx=0).stop 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 shutdown_connections 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5635f9057170 0x5635f9059590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635f8fd88a0 0x5635f8fe07d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635f8fd2c90 0x5635f8fe4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 --2- 172.21.15.112:0/1368859120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635f8fd7f80 0x5635f8fe0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 >> 172.21.15.112:0/1368859120 conn(0x5635f8efa100 msgr2=0x5635f8fc2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6eb9ffb700 0 client.15624 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 shutdown_connections 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 1 -- 172.21.15.112:0/1368859120 wait complete. 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.060+0000 7f6ee3448b80 20 client.15624 trim_cache size 0 max 16384 2024-05-10T10:41:21.066 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:41:21.067 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:41:21.067 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:41:21.067 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:41:21.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ebc0b2880 msgr2=0x7f6ebc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ebc0b2880 0x7f6ebc0b4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400b5c0 tx=0x7f6ec401a040 comp rx=0 tx=0).stop 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4102d50 msgr2=0x7f6ed419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4102d50 0x7f6ed419fc50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b530 tx=0x7f6ec800b840 comp rx=0 tx=0).stop 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 shutdown_connections 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ed4108400 0x7f6ed41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f6ebc0b2880 0x7f6ebc0b4d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ed4102390 0x7f6ed419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 --2- 172.21.15.112:0/1988716182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed4102d50 0x7f6ed419fc50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 >> 172.21.15.112:0/1988716182 conn(0x7f6ed4079310 msgr2=0x7f6ed406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 shutdown_connections 2024-05-10T10:41:21.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:21.061+0000 7f6ed8bb5700 1 -- 172.21.15.112:0/1988716182 wait complete. 2024-05-10T10:41:21.069 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:20 smithi195 ceph-mon[33773]: pgmap v463: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:21.877 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:41:21.877 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:41:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:22.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:22 smithi112 ceph-mon[30100]: pgmap v464: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:23.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:22 smithi148 ceph-mon[33568]: pgmap v464: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:22 smithi195 ceph-mon[33773]: pgmap v464: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 -- 172.21.15.112:0/3873070062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102540 msgr2=0x7ffb4010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 --2- 172.21.15.112:0/3873070062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102540 0x7ffb4010e9b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b0d0 tx=0x7ffb3800b3e0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 -- 172.21.15.112:0/3873070062 shutdown_connections 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 --2- 172.21.15.112:0/3873070062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102540 0x7ffb4010e9b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 --2- 172.21.15.112:0/3873070062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb40107be0 0x7ffb40107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 --2- 172.21.15.112:0/3873070062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb40101b80 0x7ffb40102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 -- 172.21.15.112:0/3873070062 >> 172.21.15.112:0/3873070062 conn(0x7ffb40078a10 msgr2=0x7ffb400ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 -- 172.21.15.112:0/3873070062 shutdown_connections 2024-05-10T10:41:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.046+0000 7ffb46649700 1 -- 172.21.15.112:0/3873070062 wait complete. 2024-05-10T10:41:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.047+0000 7ffb46649700 1 Processor -- start 2024-05-10T10:41:24.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.047+0000 7ffb46649700 1 -- start start 2024-05-10T10:41:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.047+0000 7ffb46649700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb40101b80 0x7ffb4019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.047+0000 7ffb46649700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb40102540 0x7ffb4019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.047+0000 7ffb46649700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40107be0 0x7ffb40199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.047+0000 7ffb46649700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40112480 con 0x7ffb40107be0 2024-05-10T10:41:24.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb46649700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40112300 con 0x7ffb40101b80 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb46649700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40112600 con 0x7ffb40102540 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40107be0 0x7ffb40199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40107be0 0x7ffb40199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42280/0 (socket says 172.21.15.112:42280) 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 -- 172.21.15.112:0/2129793032 learned_addr learned my addr 172.21.15.112:0/2129793032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb3f7fe700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb40102540 0x7ffb4019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb3ffff700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb40101b80 0x7ffb4019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 -- 172.21.15.112:0/2129793032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb40102540 msgr2=0x7ffb4019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb40102540 0x7ffb4019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 -- 172.21.15.112:0/2129793032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb40101b80 msgr2=0x7ffb4019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb40101b80 0x7ffb4019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 -- 172.21.15.112:0/2129793032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb40199ff0 con 0x7ffb40107be0 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.048+0000 7ffb44be6700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40107be0 0x7ffb40199810 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b820 tx=0x7ffb38007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.049+0000 7ffb3d7fa700 1 -- 172.21.15.112:0/2129793032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3801f030 con 0x7ffb40107be0 2024-05-10T10:41:24.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.049+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4019a280 con 0x7ffb40107be0 2024-05-10T10:41:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.049+0000 7ffb3d7fa700 1 -- 172.21.15.112:0/2129793032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb380096a0 con 0x7ffb40107be0 2024-05-10T10:41:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.049+0000 7ffb3d7fa700 1 -- 172.21.15.112:0/2129793032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38029060 con 0x7ffb40107be0 2024-05-10T10:41:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.049+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb401a64f0 con 0x7ffb40107be0 2024-05-10T10:41:24.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.050+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb401031e0 con 0x7ffb40107be0 2024-05-10T10:41:24.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.052+0000 7ffb3d7fa700 1 -- 172.21.15.112:0/2129793032 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ffb3801b7d0 con 0x7ffb40107be0 2024-05-10T10:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.053+0000 7ffb3d7fa700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffb280b2880 0x7ffb280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.053+0000 7ffb3d7fa700 1 -- 172.21.15.112:0/2129793032 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ffb380e3050 con 0x7ffb40107be0 2024-05-10T10:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.053+0000 7ffb3ffff700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffb280b2880 0x7ffb280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.054+0000 7ffb3ffff700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffb280b2880 0x7ffb280b4d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005950 tx=0x7ffb3000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.062+0000 7ffb3d7fa700 1 -- 172.21.15.112:0/2129793032 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffb380aaa70 con 0x7ffb40107be0 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 -- 172.21.15.112:0/1799475231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141012e0 msgr2=0x7f0c14101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1799475231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141012e0 0x7f0c14101700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009900 tx=0x7f0c0402a410 comp rx=0 tx=0).stop 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 -- 172.21.15.112:0/1799475231 shutdown_connections 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1799475231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c141025e0 0x7f0c14102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1799475231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c14101c40 0x7f0c141020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1799475231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141012e0 0x7f0c14101700 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 -- 172.21.15.112:0/1799475231 >> 172.21.15.112:0/1799475231 conn(0x7f0c140fce80 msgr2=0x7f0c140ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 -- 172.21.15.112:0/1799475231 shutdown_connections 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.411+0000 7f0c1a789640 1 -- 172.21.15.112:0/1799475231 wait complete. 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 Processor -- start 2024-05-10T10:41:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 -- start start 2024-05-10T10:41:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c14101c40 0x7f0c1419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141025e0 0x7f0c1419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141a0490 0x7f0c141a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141105d0 con 0x7f0c141a0490 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14110450 con 0x7f0c14101c40 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c1a789640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14110750 con 0x7f0c141025e0 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c18d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141a0490 0x7f0c141a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c18d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141a0490 0x7f0c141a2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42302/0 (socket says 172.21.15.112:42302) 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.412+0000 7f0c18d26640 1 -- 172.21.15.112:0/1172441422 learned_addr learned my addr 172.21.15.112:0/1172441422 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c137fe640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141025e0 0x7f0c1419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c13fff640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c14101c40 0x7f0c1419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c18d26640 1 -- 172.21.15.112:0/1172441422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141025e0 msgr2=0x7f0c1419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c18d26640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141025e0 0x7f0c1419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c18d26640 1 -- 172.21.15.112:0/1172441422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c14101c40 msgr2=0x7f0c1419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c18d26640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c14101c40 0x7f0c1419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c18d26640 1 -- 172.21.15.112:0/1172441422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c141a2ef0 con 0x7f0c141a0490 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c18d26640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141a0490 0x7f0c141a2880 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00eb10 tx=0x7f0c0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c117fa640 1 -- 172.21.15.112:0/1172441422 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c004b20 con 0x7f0c141a0490 2024-05-10T10:41:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.413+0000 7f0c117fa640 1 -- 172.21.15.112:0/1172441422 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c0c004cc0 con 0x7f0c141a0490 2024-05-10T10:41:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.414+0000 7f0c117fa640 1 -- 172.21.15.112:0/1172441422 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c010690 con 0x7f0c141a0490 2024-05-10T10:41:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.414+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c141a31e0 con 0x7f0c141a0490 2024-05-10T10:41:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.414+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c141a36e0 con 0x7f0c141a0490 2024-05-10T10:41:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.417+0000 7f0c1d57c740 1 Processor -- start 2024-05-10T10:41:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.417+0000 7f0c117fa640 1 -- 172.21.15.112:0/1172441422 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0c0c005000 con 0x7f0c141a0490 2024-05-10T10:41:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c1d57c740 1 -- start start 2024-05-10T10:41:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b6b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3acf5d0 con 0x55cfe3b70910 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3a88890 con 0x55cfe3b6adb0 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3acb590 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b72da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42308/0 (socket says 172.21.15.112:42308) 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c137fe640 1 -- 172.21.15.112:0/1316566827 learned_addr learned my addr 172.21.15.112:0/1316566827 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.418+0000 7f0c137fe640 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b6b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c13fff640 1 -- 172.21.15.112:0/1316566827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 msgr2=0x55cfe3b6b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c13fff640 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b6b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c13fff640 1 -- 172.21.15.112:0/1316566827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 msgr2=0x55cfe3b72da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c13fff640 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b72da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c13fff640 1 -- 172.21.15.112:0/1316566827 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe37e1210 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c137fe640 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c117fa640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bfc0b29f0 0x7f0bfc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c13fff640 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b702a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c04008000 tx=0x7f0c04031500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f7338544d59f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c137fe640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bfc0b29f0 0x7f0bfc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0c117fa640 1 -- 172.21.15.112:0/1172441422 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c0c0e2be0 con 0x7f0c141a0490 2024-05-10T10:41:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1316566827 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04035030 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1316566827 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c04004020 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.419+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1316566827 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040038e0 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 -- 172.21.15.112:0/1316566827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 msgr2=0x55cfe3b702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c137fe640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bfc0b29f0 0x7f0bfc0b4eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55cfe3b6c000 tx=0x7f0c08009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b702a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c04008000 tx=0x7f0c04031500 comp rx=0 tx=0).stop 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 -- 172.21.15.112:0/1316566827 shutdown_connections 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b702a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/1316566827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b6b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 -- 172.21.15.112:0/1316566827 >> 172.21.15.112:0/1316566827 conn(0x55cfe3b66420 msgr2=0x55cfe3b66820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 -- 172.21.15.112:0/1316566827 shutdown_connections 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.420+0000 7f0c1d57c740 1 -- 172.21.15.112:0/1316566827 wait complete. 2024-05-10T10:41:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.421+0000 7f0c1d57c740 1 -- start start 2024-05-10T10:41:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.421+0000 7f0c1d57c740 1 Processor -- start 2024-05-10T10:41:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.421+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b74f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.421+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b754a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.421+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.421+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3b73860 con 0x55cfe3b70910 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3a8c370 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c18d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c18d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42310/0 (socket says 172.21.15.112:42310) 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c18d26640 1 -- 172.21.15.112:0/2566176451 learned_addr learned my addr 172.21.15.112:0/2566176451 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c13fff640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b74f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c137fe640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3b739e0 con 0x55cfe3b6adb0 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c1d57c740 10 client.0 _ll_register_callbacks cb 0x55cfe37e0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c137fe640 1 -- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6adb0 msgr2=0x55cfe3b74f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c137fe640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b74f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c137fe640 1 -- 172.21.15.112:0/2566176451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 msgr2=0x55cfe3b7c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c137fe640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c137fe640 1 -- 172.21.15.112:0/2566176451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe38ad620 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c18d26640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c13fff640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b74f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0c137fe640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b754a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c08005890 tx=0x7f0c08005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c0c3f299fe73fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.422+0000 7f0bf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0bf37fe640 1 -- 172.21.15.112:0/2566176451 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800ce90 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0bf37fe640 1 -- 172.21.15.112:0/2566176451 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c08013e00 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c04004020 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0bf37fe640 1 -- 172.21.15.112:0/2566176451 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c08003800 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0c1d57c740 10 client.25121 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfe3b7f580 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0bf37fe640 1 -- 172.21.15.112:0/2566176451 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c04004020 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.423+0000 7f0bf37fe640 1 -- 172.21.15.112:0/2566176451 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0c08028070 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c1d57c740 10 client.25121 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c1d57c740 10 client.25121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c1d57c740 20 client.25121 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3b80c60 0x55cfe3b830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c1d57c740 4 client.25121 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfe3add1e0 con 0x55cfe3b80c60 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c13fff640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3b80c60 0x55cfe3b830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0bf37fe640 10 client.25121 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.424+0000 7f0c13fff640 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3b80c60 0x55cfe3b830f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb56ceaf72df50c4 server_cookie=313a216c79e8e4a8 in_seq=0 out_seq=0 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0bf37fe640 1 -- 172.21.15.112:0/2566176451 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cfe3add1e0 con 0x55cfe3b80c60 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0bf37fe640 10 client.25121 handle_command_reply: tid=0 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 client.25121 shutdown 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6b7a0 msgr2=0x55cfe3b754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b754a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c08005890 tx=0x7f0c08005f90 comp rx=0 tx=0).stop 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 shutdown_connections 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3b80c60 0x55cfe3b830f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b754a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/2566176451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b74f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 >> 172.21.15.112:0/2566176451 conn(0x55cfe3b66420 msgr2=0x55cfe3b65580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0bf37fe640 0 client.25121 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 shutdown_connections 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.425+0000 7f0c1d57c740 1 -- 172.21.15.112:0/2566176451 wait complete. 2024-05-10T10:41:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.426+0000 7f0c1d57c740 20 client.25121 trim_cache size 0 max 16384 2024-05-10T10:41:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.440+0000 7f0c1d57c740 1 Processor -- start 2024-05-10T10:41:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.440+0000 7f0c1d57c740 1 -- start start 2024-05-10T10:41:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b69330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b7c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b70910 0x55cfe3b69870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141adab0 con 0x55cfe3b6adb0 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141af5f0 con 0x55cfe3b70910 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141ae180 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b69330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b69330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42326/0 (socket says 172.21.15.112:42326) 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 -- 172.21.15.112:0/3826953680 learned_addr learned my addr 172.21.15.112:0/3826953680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c137fe640 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b7c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 -- 172.21.15.112:0/3826953680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 msgr2=0x55cfe3b7c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b7c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 -- 172.21.15.112:0/3826953680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b70910 msgr2=0x55cfe3b69870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b70910 0x55cfe3b69870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 -- 172.21.15.112:0/3826953680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe3b6fc70 con 0x55cfe3b6adb0 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c137fe640 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b7c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0c13fff640 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b69330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c04000c00 tx=0x7f0c04005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9d36ea168f5526 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0bf9ffb640 1 -- 172.21.15.112:0/3826953680 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04035040 con 0x55cfe3b6adb0 2024-05-10T10:41:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0bf9ffb640 1 -- 172.21.15.112:0/3826953680 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c04044440 con 0x55cfe3b6adb0 2024-05-10T10:41:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.441+0000 7f0bf9ffb640 1 -- 172.21.15.112:0/3826953680 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0403c6a0 con 0x55cfe3b6adb0 2024-05-10T10:41:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 -- 172.21.15.112:0/3826953680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b6adb0 msgr2=0x55cfe3b69330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b69330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c04000c00 tx=0x7f0c04005a20 comp rx=0 tx=0).stop 2024-05-10T10:41:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 -- 172.21.15.112:0/3826953680 shutdown_connections 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b70910 0x55cfe3b69870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b7c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/3826953680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b69330 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 -- 172.21.15.112:0/3826953680 >> 172.21.15.112:0/3826953680 conn(0x55cfe3b66420 msgr2=0x55cfe3b6c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 -- 172.21.15.112:0/3826953680 shutdown_connections 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.442+0000 7f0c1d57c740 1 -- 172.21.15.112:0/3826953680 wait complete. 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 -- start start 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 Processor -- start 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b799f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b79f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141aef20 con 0x55cfe3b70910 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141ae850 con 0x55cfe3b6adb0 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c18d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c18d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42332/0 (socket says 172.21.15.112:42332) 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b799f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c18d26640 1 -- 172.21.15.112:0/193723527 learned_addr learned my addr 172.21.15.112:0/193723527 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.443+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141aba90 con 0x55cfe3b6b7a0 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c1d57c740 10 client.0 _ll_register_callbacks cb 0x7f0c08009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c137fe640 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b79f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c18d26640 1 -- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 msgr2=0x55cfe3b79f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c18d26640 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b79f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c18d26640 1 -- 172.21.15.112:0/193723527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 msgr2=0x55cfe3b799f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c18d26640 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b799f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c18d26640 1 -- 172.21.15.112:0/193723527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe3b7b530 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c13fff640 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c18d26640 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7af40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0a6410 tx=0x7f0c0c0aa410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20f29dbdad0c37e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0bf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0bf37fe640 1 -- 172.21.15.112:0/193723527 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0c9060 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0bf37fe640 1 -- 172.21.15.112:0/193723527 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c0c0a57e0 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.444+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c04044440 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.445+0000 7f0bf37fe640 1 -- 172.21.15.112:0/193723527 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c0c0a5d30 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.445+0000 7f0c1d57c740 10 client.15648 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.445+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfe3b7c8b0 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.445+0000 7f0bf37fe640 1 -- 172.21.15.112:0/193723527 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c04044440 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.446+0000 7f0bf37fe640 1 -- 172.21.15.112:0/193723527 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0c0c0b6030 con 0x55cfe3b70910 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.446+0000 7f0c1d57c740 10 client.15648 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.446+0000 7f0c1d57c740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.446+0000 7f0c1d57c740 20 client.15648 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.446+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3bebdc0 0x55cfe3bee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.446+0000 7f0c1d57c740 4 client.15648 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:41:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.446+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfe3ad7050 con 0x55cfe3bebdc0 2024-05-10T10:41:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.447+0000 7f0c13fff640 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3bebdc0 0x55cfe3bee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.447+0000 7f0c13fff640 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3bebdc0 0x55cfe3bee1e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3bb435ea9fde server_cookie=6334a4761d619a1d in_seq=0 out_seq=0 2024-05-10T10:41:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.447+0000 7f0bf37fe640 10 client.15648 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0bf37fe640 1 -- 172.21.15.112:0/193723527 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cfe3ad7050 con 0x55cfe3bebdc0 2024-05-10T10:41:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0bf37fe640 10 client.15648 handle_command_reply: tid=0 2024-05-10T10:41:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 client.15648 shutdown 2024-05-10T10:41:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 msgr2=0x55cfe3b7af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7af40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0a6410 tx=0x7f0c0c0aa410 comp rx=0 tx=0).stop 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 shutdown_connections 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfe3b70910 0x55cfe3b7af40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfe3b6b7a0 0x55cfe3b79f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cfe3bebdc0 0x55cfe3bee1e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 --2- 172.21.15.112:0/193723527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfe3b6adb0 0x55cfe3b799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 >> 172.21.15.112:0/193723527 conn(0x55cfe3b66420 msgr2=0x55cfe3b64cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0bf37fe640 0 client.15648 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 shutdown_connections 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 1 -- 172.21.15.112:0/193723527 wait complete. 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.451+0000 7f0c1d57c740 20 client.15648 trim_cache size 0 max 16384 2024-05-10T10:41:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:41:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:41:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:41:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:41:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bfc0b29f0 msgr2=0x7f0bfc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bfc0b29f0 0x7f0bfc0b4eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55cfe3b6c000 tx=0x7f0c08009700 comp rx=0 tx=0).stop 2024-05-10T10:41:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141a0490 msgr2=0x7f0c141a2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141a0490 0x7f0c141a2880 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00eb10 tx=0x7f0c0c00c2d0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 shutdown_connections 2024-05-10T10:41:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0bfc0b29f0 0x7f0bfc0b4eb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c141a0490 0x7f0c141a2880 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141025e0 0x7f0c1419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.452+0000 7f0c1a789640 1 --2- 172.21.15.112:0/1172441422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c14101c40 0x7f0c1419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.453+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 >> 172.21.15.112:0/1172441422 conn(0x7f0c140fce80 msgr2=0x7f0c141002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.453+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 shutdown_connections 2024-05-10T10:41:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:24.453+0000 7f0c1a789640 1 -- 172.21.15.112:0/1172441422 wait complete. 2024-05-10T10:41:24.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.460+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb401a66c0 con 0x7ffb40107be0 2024-05-10T10:41:24.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.461+0000 7ffb3d7fa700 1 -- 172.21.15.112:0/2129793032 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7ffb380aa370 con 0x7ffb40107be0 2024-05-10T10:41:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:41:24.466 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:40:06.380474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:41:24.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffb280b2880 msgr2=0x7ffb280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffb280b2880 0x7ffb280b4d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005950 tx=0x7ffb3000b410 comp rx=0 tx=0).stop 2024-05-10T10:41:24.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40107be0 msgr2=0x7ffb40199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40107be0 0x7ffb40199810 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b820 tx=0x7ffb38007d10 comp rx=0 tx=0).stop 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 shutdown_connections 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40107be0 0x7ffb40199810 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb40101b80 0x7ffb4019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ffb280b2880 0x7ffb280b4d40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 --2- 172.21.15.112:0/2129793032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb40102540 0x7ffb4019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.464+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 >> 172.21.15.112:0/2129793032 conn(0x7ffb40078a10 msgr2=0x7ffb40105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.465+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 shutdown_connections 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:24.465+0000 7ffb46649700 1 -- 172.21.15.112:0/2129793032 wait complete. 2024-05-10T10:41:24.468 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-05-10T10:41:24.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:24 smithi112 ceph-mon[30100]: pgmap v465: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:24.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:24 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2129793032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:25.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:24 smithi148 ceph-mon[33568]: pgmap v465: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:25.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:24 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2129793032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:25.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:24 smithi195 ceph-mon[33773]: pgmap v465: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:25.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:24 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2129793032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:25.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:41:25.710 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:27.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:26 smithi148 ceph-mon[33568]: pgmap v466: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-10T10:41:27.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:26 smithi195 ceph-mon[33773]: pgmap v466: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-10T10:41:27.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:26 smithi112 ceph-mon[30100]: pgmap v466: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.491+0000 7f2791860700 1 -- 172.21.15.112:0/3148081327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c103ef0 msgr2=0x7f278c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.491+0000 7f2791860700 1 --2- 172.21.15.112:0/3148081327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c103ef0 0x7f278c1103d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f278400b0a0 tx=0x7f278400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.491+0000 7f2791860700 1 -- 172.21.15.112:0/3148081327 shutdown_connections 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.491+0000 7f2791860700 1 --2- 172.21.15.112:0/3148081327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c103ef0 0x7f278c1103d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.491+0000 7f2791860700 1 --2- 172.21.15.112:0/3148081327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f278c108460 0x7f278c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.491+0000 7f2791860700 1 --2- 172.21.15.112:0/3148081327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c103550 0x7f278c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.491+0000 7f2791860700 1 -- 172.21.15.112:0/3148081327 >> 172.21.15.112:0/3148081327 conn(0x7f278c07a600 msgr2=0x7f278c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.492+0000 7f2791860700 1 -- 172.21.15.112:0/3148081327 shutdown_connections 2024-05-10T10:41:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.492+0000 7f2791860700 1 -- 172.21.15.112:0/3148081327 wait complete. 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.492+0000 7f2791860700 1 Processor -- start 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.492+0000 7f2791860700 1 -- start start 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f2791860700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c103550 0x7f278c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f2791860700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f278c103ef0 0x7f278c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f2791860700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c108460 0x7f278c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f2791860700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f278c113590 con 0x7f278c108460 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f2791860700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f278c113410 con 0x7f278c103550 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f2791860700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f278c113710 con 0x7f278c103ef0 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c108460 0x7f278c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c108460 0x7f278c076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50700/0 (socket says 172.21.15.112:50700) 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 -- 172.21.15.112:0/2254692316 learned_addr learned my addr 172.21.15.112:0/2254692316 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278affd700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c103550 0x7f278c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278a7fc700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f278c103ef0 0x7f278c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 -- 172.21.15.112:0/2254692316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f278c103ef0 msgr2=0x7f278c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f278c103ef0 0x7f278c075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 -- 172.21.15.112:0/2254692316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c103550 msgr2=0x7f278c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c103550 0x7f278c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 -- 172.21.15.112:0/2254692316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c1a7f00 con 0x7f278c108460 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.493+0000 7f278b7fe700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c108460 0x7f278c076220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f278400b830 tx=0x7f2784004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.494+0000 7f279085e700 1 -- 172.21.15.112:0/2254692316 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784021070 con 0x7f278c108460 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.494+0000 7f279085e700 1 -- 172.21.15.112:0/2254692316 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2784007b80 con 0x7f278c108460 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.494+0000 7f279085e700 1 -- 172.21.15.112:0/2254692316 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27840052a0 con 0x7f278c108460 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.494+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c1a8190 con 0x7f278c108460 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.494+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c1a8590 con 0x7f278c108460 2024-05-10T10:41:27.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.497+0000 7f279085e700 1 -- 172.21.15.112:0/2254692316 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2784024070 con 0x7f278c108460 2024-05-10T10:41:27.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.497+0000 7f279085e700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2980 0x7f27740b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f278affd700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2980 0x7f27740b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 Processor -- start 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 -- start start 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279085e700 1 -- 172.21.15.112:0/2254692316 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f27840e2e90 con 0x7f278c108460 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 0x55aeb30ee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30eef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aeb3033940 con 0x55aeb30eeaa0 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2f98da0 con 0x55aeb30ea200 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aeb302f900 con 0x55aeb30eab20 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f278b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f278b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30eef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50710/0 (socket says 172.21.15.112:50710) 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f278b7fe700 1 -- 172.21.15.112:0/1058376622 learned_addr learned my addr 172.21.15.112:0/1058376622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.498+0000 7f278affd700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2980 0x7f27740b4e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f277c005950 tx=0x7f277c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f278b7fe700 1 -- 172.21.15.112:0/1058376622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 msgr2=0x55aeb30ee560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f278b7fe700 1 --2- 172.21.15.112:0/1058376622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 0x55aeb30ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f278b7fe700 1 -- 172.21.15.112:0/1058376622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 msgr2=0x55aeb30ea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f278b7fe700 1 --2- 172.21.15.112:0/1058376622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f278b7fe700 1 -- 172.21.15.112:0/1058376622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb2f808d0 con 0x55aeb30eeaa0 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f278b7fe700 1 --2- 172.21.15.112:0/1058376622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30eef00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f27840a6180 tx=0x7f27840099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d483a8ced0518e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f2771ffb700 1 -- 172.21.15.112:0/1058376622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278402b070 con 0x55aeb30eeaa0 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f2771ffb700 1 -- 172.21.15.112:0/1058376622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27840aa150 con 0x55aeb30eeaa0 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.499+0000 7f2771ffb700 1 -- 172.21.15.112:0/1058376622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27840ae2e0 con 0x55aeb30eeaa0 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 -- 172.21.15.112:0/1058376622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 msgr2=0x55aeb30eef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/1058376622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30eef00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f27840a6180 tx=0x7f27840099e0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 -- 172.21.15.112:0/1058376622 shutdown_connections 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/1058376622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30eef00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/1058376622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/1058376622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 0x55aeb30ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 -- 172.21.15.112:0/1058376622 >> 172.21.15.112:0/1058376622 conn(0x55aeb300db70 msgr2=0x55aeb30dc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 -- 172.21.15.112:0/1058376622 shutdown_connections 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.500+0000 7f279c0f3b80 1 -- 172.21.15.112:0/1058376622 wait complete. 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.501+0000 7f279c0f3b80 1 -- start start 2024-05-10T10:41:27.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.501+0000 7f279c0f3b80 1 Processor -- start 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 0x55aeb30f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30f3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aeb30f69c0 con 0x55aeb30eeaa0 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2feced0 con 0x55aeb30ea200 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aeb30f6b40 con 0x55aeb30eab20 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48928/0 (socket says 172.21.15.112:48928) 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 -- 172.21.15.112:0/3585337761 learned_addr learned my addr 172.21.15.112:0/3585337761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278a7fc700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 0x55aeb30f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278b7fe700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f279c0f3b80 10 client.0 _ll_register_callbacks cb 0x55aeb2fc75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 -- 172.21.15.112:0/3585337761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 msgr2=0x55aeb30f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 0x55aeb30f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 -- 172.21.15.112:0/3585337761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 msgr2=0x55aeb30f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278affd700 1 -- 172.21.15.112:0/3585337761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb2fea760 con 0x55aeb30ea200 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.502+0000 7f278b7fe700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f278affd700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30f5eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f277c00fdb0 tx=0x7f277c00b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc261b666337424 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f276affd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f276affd700 1 -- 172.21.15.112:0/3585337761 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c022070 con 0x55aeb30ea200 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f276affd700 1 -- 172.21.15.112:0/3585337761 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f277c011de0 con 0x55aeb30ea200 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27840aa150 con 0x55aeb30ea200 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f276affd700 1 -- 172.21.15.112:0/3585337761 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f277c02c050 con 0x55aeb30ea200 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f279c0f3b80 10 client.25133 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.503+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aeb3101160 con 0x55aeb30ea200 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f276affd700 1 -- 172.21.15.112:0/3585337761 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27840aa150 con 0x55aeb30ea200 2024-05-10T10:41:27.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f276affd700 1 -- 172.21.15.112:0/3585337761 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f277c027560 con 0x55aeb30ea200 2024-05-10T10:41:27.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f279c0f3b80 10 client.25133 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:27.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f279c0f3b80 10 client.25133 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:41:27.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f279c0f3b80 20 client.25133 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb31037a0 0x55aeb3105c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f279c0f3b80 4 client.25133 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeb3041320 con 0x55aeb31037a0 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.504+0000 7f278a7fc700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb31037a0 0x55aeb3105c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.505+0000 7f276affd700 10 client.25133 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.505+0000 7f278a7fc700 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb31037a0 0x55aeb3105c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=279931e6d6d20c0b server_cookie=5b2cab63369ed962 in_seq=0 out_seq=0 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.505+0000 7f276affd700 1 -- 172.21.15.112:0/3585337761 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aeb3041320 con 0x55aeb31037a0 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.505+0000 7f276affd700 10 client.25133 handle_command_reply: tid=0 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.505+0000 7f279c0f3b80 1 client.25133 shutdown 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 msgr2=0x55aeb30f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30f5eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f277c00fdb0 tx=0x7f277c00b4b0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 shutdown_connections 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb30f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30f5eb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb31037a0 0x55aeb3105c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/3585337761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eab20 0x55aeb30f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f276affd700 0 client.25133 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 >> 172.21.15.112:0/3585337761 conn(0x55aeb300db70 msgr2=0x55aeb30d6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 shutdown_connections 2024-05-10T10:41:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 1 -- 172.21.15.112:0/3585337761 wait complete. 2024-05-10T10:41:27.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.506+0000 7f279c0f3b80 20 client.25133 trim_cache size 0 max 16384 2024-05-10T10:41:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 Processor -- start 2024-05-10T10:41:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 -- start start 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eab20 0x55aeb30eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eeaa0 0x55aeb30edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1acea0 con 0x55aeb30eab20 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1ae9e0 con 0x55aeb30ea200 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1ad570 con 0x55aeb30eeaa0 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f278a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eab20 0x55aeb30eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f278a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eab20 0x55aeb30eeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50738/0 (socket says 172.21.15.112:50738) 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.521+0000 7f278a7fc700 1 -- 172.21.15.112:0/23966291 learned_addr learned my addr 172.21.15.112:0/23966291 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278b7fe700 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eeaa0 0x55aeb30edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278affd700 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278a7fc700 1 -- 172.21.15.112:0/23966291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eeaa0 msgr2=0x55aeb30edb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278a7fc700 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eeaa0 0x55aeb30edb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278a7fc700 1 -- 172.21.15.112:0/23966291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 msgr2=0x55aeb30dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278a7fc700 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30dde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278a7fc700 1 -- 172.21.15.112:0/23966291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb31062b0 con 0x55aeb30eab20 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f278a7fc700 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eab20 0x55aeb30eeea0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f278c07ac00 tx=0x7f2780005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b7efe234ecb299 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f27717fa700 1 -- 172.21.15.112:0/23966291 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278001f030 con 0x55aeb30eab20 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f27717fa700 1 -- 172.21.15.112:0/23966291 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2780004830 con 0x55aeb30eab20 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.522+0000 7f27717fa700 1 -- 172.21.15.112:0/23966291 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800053e0 con 0x55aeb30eab20 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 -- 172.21.15.112:0/23966291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eab20 msgr2=0x55aeb30eeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eab20 0x55aeb30eeea0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f278c07ac00 tx=0x7f2780005910 comp rx=0 tx=0).stop 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 -- 172.21.15.112:0/23966291 shutdown_connections 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30eeaa0 0x55aeb30edb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30ea200 0x55aeb30dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/23966291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eab20 0x55aeb30eeea0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 -- 172.21.15.112:0/23966291 >> 172.21.15.112:0/23966291 conn(0x55aeb300db70 msgr2=0x55aeb30ed460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 -- 172.21.15.112:0/23966291 shutdown_connections 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.523+0000 7f279c0f3b80 1 -- 172.21.15.112:0/23966291 wait complete. 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 -- start start 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 Processor -- start 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30ea200 0x55aeb30f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30eab20 0x55aeb30f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb316c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1ae310 con 0x55aeb30eeaa0 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1adc40 con 0x55aeb30eab20 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1aae70 con 0x55aeb30ea200 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.524+0000 7f279c0f3b80 10 client.0 _ll_register_callbacks cb 0x55aeb2fc75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30eab20 0x55aeb30f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30eab20 0x55aeb30f95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48950/0 (socket says 172.21.15.112:48950) 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 -- 172.21.15.112:0/2646616885 learned_addr learned my addr 172.21.15.112:0/2646616885 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278affd700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30ea200 0x55aeb30f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278b7fe700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb316c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 -- 172.21.15.112:0/2646616885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30ea200 msgr2=0x55aeb30f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30ea200 0x55aeb30f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 -- 172.21.15.112:0/2646616885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 msgr2=0x55aeb316c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb316c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 -- 172.21.15.112:0/2646616885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb316c680 con 0x55aeb30eab20 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278b7fe700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb316c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f278a7fc700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30eab20 0x55aeb30f95e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55aeb3101470 tx=0x7f2780012cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff78fb50c19ddd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.525+0000 7f276affd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f276affd700 1 -- 172.21.15.112:0/2646616885 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780005020 con 0x55aeb30eab20 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f276affd700 1 -- 172.21.15.112:0/2646616885 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f27800051c0 con 0x55aeb30eab20 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2780004830 con 0x55aeb30eab20 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f276affd700 1 -- 172.21.15.112:0/2646616885 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2780037050 con 0x55aeb30eab20 2024-05-10T10:41:27.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 10 client.25139 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:27.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aeb316d980 con 0x55aeb30eab20 2024-05-10T10:41:27.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f276affd700 1 -- 172.21.15.112:0/2646616885 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2780004830 con 0x55aeb30eab20 2024-05-10T10:41:27.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f276affd700 1 -- 172.21.15.112:0/2646616885 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f278000f040 con 0x55aeb30eab20 2024-05-10T10:41:27.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 10 client.25139 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:27.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 10 client.25139 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:41:27.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 20 client.25139 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb316dfb0 0x55aeb31703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 4 client.25139 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.526+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aeb303b3c0 con 0x55aeb316dfb0 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f278affd700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb316dfb0 0x55aeb31703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f276affd700 10 client.25139 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f278affd700 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb316dfb0 0x55aeb31703d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed4ebfe8e4fb79e8 server_cookie=a678508cb001e60f in_seq=0 out_seq=0 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f276affd700 1 -- 172.21.15.112:0/2646616885 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aeb303b3c0 con 0x55aeb316dfb0 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f276affd700 10 client.25139 handle_command_reply: tid=0 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f279c0f3b80 1 client.25139 shutdown 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30eab20 msgr2=0x55aeb30f95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30eab20 0x55aeb30f95e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55aeb3101470 tx=0x7f2780012cb0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 shutdown_connections 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aeb30eeaa0 0x55aeb316c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.527+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55aeb316dfb0 0x55aeb31703d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aeb30ea200 0x55aeb30f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f279c0f3b80 1 --2- 172.21.15.112:0/2646616885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aeb30eab20 0x55aeb30f95e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 >> 172.21.15.112:0/2646616885 conn(0x55aeb300db70 msgr2=0x55aeb30ed460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f276affd700 0 client.25139 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 shutdown_connections 2024-05-10T10:41:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f279c0f3b80 1 -- 172.21.15.112:0/2646616885 wait complete. 2024-05-10T10:41:27.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f279c0f3b80 20 client.25139 trim_cache size 0 max 16384 2024-05-10T10:41:27.533 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:27.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2980 msgr2=0x7f27740b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f2791860700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2980 0x7f27740b4e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f277c005950 tx=0x7f277c0058e0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c108460 msgr2=0x7f278c076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.528+0000 7f2791860700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c108460 0x7f278c076220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f278400b830 tx=0x7f2784004d90 comp rx=0 tx=0).stop 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 shutdown_connections 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f278c108460 0x7f278c076220 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c103550 0x7f278c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f27740b2980 0x7f27740b4e40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 --2- 172.21.15.112:0/2254692316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f278c103ef0 0x7f278c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 >> 172.21.15.112:0/2254692316 conn(0x7f278c07a600 msgr2=0x7f278c101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 shutdown_connections 2024-05-10T10:41:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:27.529+0000 7f2791860700 1 -- 172.21.15.112:0/2254692316 wait complete. 2024-05-10T10:41:28.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:41:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:29.022 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:28 smithi112 ceph-mon[30100]: pgmap v467: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-10T10:41:29.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:28 smithi148 ceph-mon[33568]: pgmap v467: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-10T10:41:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:28 smithi195 ceph-mon[33773]: pgmap v467: 129 pgs: 129 active+clean; 672 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-10T10:41:30.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.591+0000 7f120ce37700 1 -- 172.21.15.112:0/393698021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 msgr2=0x7f120810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.591+0000 7f120ce37700 1 --2- 172.21.15.112:0/393698021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 0x7f120810bac0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009990 tx=0x7f11f8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.592+0000 7f120ce37700 1 -- 172.21.15.112:0/393698021 shutdown_connections 2024-05-10T10:41:30.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.592+0000 7f120ce37700 1 --2- 172.21.15.112:0/393698021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081060a0 0x7f12081125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.592+0000 7f120ce37700 1 --2- 172.21.15.112:0/393698021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 0x7f120810bac0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.592+0000 7f120ce37700 1 --2- 172.21.15.112:0/393698021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12081056e0 0x7f1208105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.592+0000 7f120ce37700 1 -- 172.21.15.112:0/393698021 >> 172.21.15.112:0/393698021 conn(0x7f12081011d0 msgr2=0x7f12081035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.592+0000 7f120ce37700 1 -- 172.21.15.112:0/393698021 shutdown_connections 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.592+0000 7f120ce37700 1 -- 172.21.15.112:0/393698021 wait complete. 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.593+0000 7f120ce37700 1 Processor -- start 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.593+0000 7f120ce37700 1 -- start start 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.593+0000 7f120ce37700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081056e0 0x7f12081124a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.593+0000 7f120ce37700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12081060a0 0x7f120810f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120ce37700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 0x7f120810fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120ce37700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12081157d0 con 0x7f120810b6e0 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120ce37700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1208115650 con 0x7f12081056e0 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081056e0 0x7f12081124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081056e0 0x7f12081124a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48956/0 (socket says 172.21.15.112:48956) 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 -- 172.21.15.112:0/3921744860 learned_addr learned my addr 172.21.15.112:0/3921744860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f1205d9b700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12081060a0 0x7f120810f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1208115950 con 0x7f12081060a0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f1206d9d700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 0x7f120810fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 -- 172.21.15.112:0/3921744860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12081060a0 msgr2=0x7f120810f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12081060a0 0x7f120810f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 -- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 msgr2=0x7f120810fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 0x7f120810fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f120659c700 1 -- 172.21.15.112:0/3921744860 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12081ac4d0 con 0x7f12081056e0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.594+0000 7f1206d9d700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 0x7f120810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.595+0000 7f120659c700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081056e0 0x7f12081124a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f11f800b650 tx=0x7f11f800bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.595+0000 7f11f77fe700 1 -- 172.21.15.112:0/3921744860 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8027070 con 0x7f12081056e0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.595+0000 7f11f77fe700 1 -- 172.21.15.112:0/3921744860 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f8022470 con 0x7f12081056e0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.595+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12081ac6a0 con 0x7f12081056e0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.595+0000 7f11f77fe700 1 -- 172.21.15.112:0/3921744860 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8017650 con 0x7f12081056e0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.595+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081acaa0 con 0x7f12081056e0 2024-05-10T10:41:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.599+0000 7f11f77fe700 1 -- 172.21.15.112:0/3921744860 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f11f802a070 con 0x7f12081056e0 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.600+0000 7f11f77fe700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f11f00b2900 0x7f11f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.600+0000 7f12176cab80 1 Processor -- start 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f12176cab80 1 -- start start 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f1205d9b700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f11f00b2900 0x7f11f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f11f77fe700 1 -- 172.21.15.112:0/3921744860 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f11f80e3370 con 0x7f12081056e0 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f12176cab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e4e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f12176cab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f1205d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f1205d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e48810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50788/0 (socket says 172.21.15.112:50788) 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f1205d9b700 1 -- 172.21.15.112:0/752549103 learned_addr learned my addr 172.21.15.112:0/752549103 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f12176cab80 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e49190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f120659c700 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e4e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5d9a940 con 0x55e8f5e4ead0 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5cffda0 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.601+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5d96900 con 0x55e8f5e48d50 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f120659c700 1 -- 172.21.15.112:0/752549103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 msgr2=0x55e8f5e49190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f1206d9d700 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e49190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f120659c700 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e49190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f120659c700 1 -- 172.21.15.112:0/752549103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 msgr2=0x55e8f5e48810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f120659c700 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e48810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f120659c700 1 -- 172.21.15.112:0/752549103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f5ce78d0 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f1206d9d700 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e49190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f1205d9b700 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e48810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f120659c700 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e4e590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f11f80e0b30 tx=0x7f11f80aa270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b992326f741ce4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f11f4ff9700 1 -- 172.21.15.112:0/752549103 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80aea40 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.602+0000 7f11f4ff9700 1 -- 172.21.15.112:0/752549103 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f80b3460 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 msgr2=0x55e8f5e4e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f12176cab80 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e4e590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f11f80e0b30 tx=0x7f11f80aa270 comp rx=0 tx=0).stop 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f1205d9b700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f11f00b2900 0x7f11f00b4dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55e8f5e45450 tx=0x7f11fc00b820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 shutdown_connections 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f12176cab80 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e49190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f12176cab80 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e4e590 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f12176cab80 1 --2- 172.21.15.112:0/752549103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e48810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.603+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 >> 172.21.15.112:0/752549103 conn(0x55e8f5d74b70 msgr2=0x55e8f5e432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.604+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 shutdown_connections 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.604+0000 7f12176cab80 1 -- 172.21.15.112:0/752549103 wait complete. 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.604+0000 7f12176cab80 1 -- start start 2024-05-10T10:41:30.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.604+0000 7f12176cab80 1 Processor -- start 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.604+0000 7f12176cab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e48d50 0x55e8f5e545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f12176cab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f12176cab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e51ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f12176cab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5e55210 con 0x55e8f5e48d50 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f12176cab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5d53ed0 con 0x55e8f5e4ead0 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f12176cab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5e55390 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f12176cab80 10 client.0 _ll_register_callbacks cb 0x55e8f5d2e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e48d50 0x55e8f5e545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e48d50 0x55e8f5e545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50796/0 (socket says 172.21.15.112:50796) 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 -- 172.21.15.112:0/4045526394 learned_addr learned my addr 172.21.15.112:0/4045526394 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 -- 172.21.15.112:0/4045526394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e4e1b0 msgr2=0x55e8f5e51980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 -- 172.21.15.112:0/4045526394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 msgr2=0x55e8f5e51ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:41:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e51ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 -- 172.21.15.112:0/4045526394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f5d51760 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f120659c700 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e48d50 0x55e8f5e545e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f11f80e0ae0 tx=0x7f11f80aacb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740f5c9fe8d8658b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f11edffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f11edffb700 1 -- 172.21.15.112:0/4045526394 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80afde0 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f11edffb700 1 -- 172.21.15.112:0/4045526394 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f11f80b2fd0 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.605+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f80b3460 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f11edffb700 1 -- 172.21.15.112:0/4045526394 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11f80b31d0 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 10 client.15678 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8f5e55210 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f11edffb700 1 -- 172.21.15.112:0/4045526394 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f80b3460 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f11edffb700 1 -- 172.21.15.112:0/4045526394 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f11f80a9450 con 0x55e8f5e48d50 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 10 client.15678 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 10 client.15678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:41:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 20 client.15678 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5e61990 0x55e8f5e63d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 4 client.15678 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.606+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f11f80922a0 con 0x55e8f5e61990 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.607+0000 7f1205d9b700 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5e61990 0x55e8f5e63d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.607+0000 7f11edffb700 10 client.15678 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.607+0000 7f1205d9b700 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5e61990 0x55e8f5e63d80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3847612bcb5c57bf server_cookie=eec2e99b41e59ed9 in_seq=0 out_seq=0 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f11edffb700 1 -- 172.21.15.112:0/4045526394 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f11f80922a0 con 0x55e8f5e61990 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f11edffb700 10 client.15678 handle_command_reply: tid=0 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 client.15678 shutdown 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e48d50 msgr2=0x55e8f5e545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e48d50 0x55e8f5e545e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f11f80e0ae0 tx=0x7f11f80aacb0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 shutdown_connections 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5e61990 0x55e8f5e63d80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e51ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e48d50 0x55e8f5e545e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 --2- 172.21.15.112:0/4045526394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 >> 172.21.15.112:0/4045526394 conn(0x55e8f5d74b70 msgr2=0x55e8f5e3db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f11edffb700 0 client.15678 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 shutdown_connections 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 1 -- 172.21.15.112:0/4045526394 wait complete. 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.608+0000 7f12176cab80 20 client.15678 trim_cache size 0 max 16384 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 Processor -- start 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 -- start start 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e3dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e51770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b56c0 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b7200 con 0x55e8f5e4ead0 2024-05-10T10:41:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.637+0000 7f12176cab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b5d90 con 0x55e8f5e48d50 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50798/0 (socket says 172.21.15.112:50798) 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 -- 172.21.15.112:0/2037865842 learned_addr learned my addr 172.21.15.112:0/2037865842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1206d9d700 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e51770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f120659c700 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e3dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 -- 172.21.15.112:0/2037865842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 msgr2=0x55e8f5e3dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e3dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 -- 172.21.15.112:0/2037865842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 msgr2=0x55e8f5e51770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e51770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 -- 172.21.15.112:0/2037865842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f5e44070 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.638+0000 7f1205d9b700 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51230 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55e8f5e44ed0 tx=0x7f11fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2a90884248295 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.639+0000 7f11effff700 1 -- 172.21.15.112:0/2037865842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00b9b0 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.639+0000 7f11effff700 1 -- 172.21.15.112:0/2037865842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11fc02c030 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.639+0000 7f11effff700 1 -- 172.21.15.112:0/2037865842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0308f0 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 -- 172.21.15.112:0/2037865842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4e1b0 msgr2=0x55e8f5e51230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51230 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55e8f5e44ed0 tx=0x7f11fc0047c0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 -- 172.21.15.112:0/2037865842 shutdown_connections 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e51770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e3dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 --2- 172.21.15.112:0/2037865842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e51230 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 -- 172.21.15.112:0/2037865842 >> 172.21.15.112:0/2037865842 conn(0x55e8f5d74b70 msgr2=0x55e8f5e509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 -- 172.21.15.112:0/2037865842 shutdown_connections 2024-05-10T10:41:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 -- 172.21.15.112:0/2037865842 wait complete. 2024-05-10T10:41:30.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 -- start start 2024-05-10T10:41:30.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.640+0000 7f12176cab80 1 Processor -- start 2024-05-10T10:41:30.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f12176cab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f12176cab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e69430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f12176cab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e69970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f12176cab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b6b30 con 0x55e8f5e4ead0 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f12176cab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b6460 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f1205d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e69430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f1206d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e69970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f1205d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e69430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:49010/0 (socket says 172.21.15.112:49010) 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f1205d9b700 1 -- 172.21.15.112:0/3390038002 learned_addr learned my addr 172.21.15.112:0/3390038002 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f120659c700 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.641+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b3660 con 0x55e8f5e48d50 2024-05-10T10:41:30.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f1205d9b700 1 -- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 msgr2=0x55e8f5e68ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f1205d9b700 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e68ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f1205d9b700 1 -- 172.21.15.112:0/3390038002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 msgr2=0x55e8f5e69970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f1205d9b700 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e69970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f1205d9b700 1 -- 172.21.15.112:0/3390038002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f5e44d00 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f1206d9d700 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e69970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f12176cab80 10 client.0 _ll_register_callbacks cb 0x55e8f5d2e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc02c030 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f1205d9b700 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e69430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc012510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41360a378116642 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f11edffb700 10 client.25154 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f11edffb700 1 -- 172.21.15.112:0/3390038002 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc028420 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f11edffb700 1 -- 172.21.15.112:0/3390038002 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f11fc0285c0 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f11edffb700 1 -- 172.21.15.112:0/3390038002 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11fc028840 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f12176cab80 10 client.25154 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:30.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.642+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e8f5ecc1d0 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f11edffb700 1 -- 172.21.15.112:0/3390038002 <== mon.2 v2:172.21.15.148:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f11fc030570 con 0x55e8f5e4e1b0 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f12176cab80 10 client.25154 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f12176cab80 10 client.25154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f12176cab80 20 client.25154 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f12176cab80 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5ecc6d0 0x55e8f5eceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f12176cab80 4 client.25154 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8f5da8320 con 0x55e8f5ecc6d0 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.643+0000 7f120659c700 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5ecc6d0 0x55e8f5eceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f11edffb700 10 client.25154 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f120659c700 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5ecc6d0 0x55e8f5eceaf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b31e43e3437fcf2f server_cookie=be542b55168417a3 in_seq=0 out_seq=0 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f11edffb700 1 -- 172.21.15.112:0/3390038002 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8f5da8320 con 0x55e8f5ecc6d0 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f11edffb700 10 client.25154 handle_command_reply: tid=0 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 client.25154 shutdown 2024-05-10T10:41:30.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 msgr2=0x55e8f5e69430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e69430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc012510 comp rx=0 tx=0).stop 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 shutdown_connections 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55e8f5ecc6d0 0x55e8f5eceaf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8f5e4ead0 0x55e8f5e69970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8f5e48d50 0x55e8f5e68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 --2- 172.21.15.112:0/3390038002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8f5e4e1b0 0x55e8f5e69430 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 >> 172.21.15.112:0/3390038002 conn(0x55e8f5d74b70 msgr2=0x55e8f5e47800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f11edffb700 0 client.25154 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 shutdown_connections 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 1 -- 172.21.15.112:0/3390038002 wait complete. 2024-05-10T10:41:30.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.644+0000 7f12176cab80 20 client.25154 trim_cache size 0 max 16384 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.645+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f11f00b2900 msgr2=0x7f11f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f11f00b2900 0x7f11f00b4dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55e8f5e45450 tx=0x7f11fc00b820 comp rx=0 tx=0).stop 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081056e0 msgr2=0x7f12081124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081056e0 0x7f12081124a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f11f800b650 tx=0x7f11f800bfc0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 shutdown_connections 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120810b6e0 0x7f120810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12081056e0 0x7f12081124a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f11f00b2900 0x7f11f00b4dc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 --2- 172.21.15.112:0/3921744860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12081060a0 0x7f120810f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 >> 172.21.15.112:0/3921744860 conn(0x7f12081011d0 msgr2=0x7f1208101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 shutdown_connections 2024-05-10T10:41:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:30.646+0000 7f120ce37700 1 -- 172.21.15.112:0/3921744860 wait complete. 2024-05-10T10:41:31.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:30 smithi148 ceph-mon[33568]: pgmap v468: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:30 smithi195 ceph-mon[33773]: pgmap v468: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:31.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:30 smithi112 ceph-mon[30100]: pgmap v468: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:33.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:32 smithi148 ceph-mon[33568]: pgmap v469: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:32 smithi195 ceph-mon[33773]: pgmap v469: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:33.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:32 smithi112 ceph-mon[30100]: pgmap v469: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:34.466 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:41:34.831 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:35.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:34 smithi148 ceph-mon[33568]: pgmap v470: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:34 smithi195 ceph-mon[33773]: pgmap v470: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:35.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:34 smithi112 ceph-mon[30100]: pgmap v470: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:36.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 -- 172.21.15.112:0/1865234553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec108410 msgr2=0x7f2bec1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 --2- 172.21.15.112:0/1865234553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec108410 0x7f2bec1087f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009990 tx=0x7f2be8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 -- 172.21.15.112:0/1865234553 shutdown_connections 2024-05-10T10:41:36.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 --2- 172.21.15.112:0/1865234553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bec102d60 0x7f2bec10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 --2- 172.21.15.112:0/1865234553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec108410 0x7f2bec1087f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 --2- 172.21.15.112:0/1865234553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bec1023a0 0x7f2bec102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 -- 172.21.15.112:0/1865234553 >> 172.21.15.112:0/1865234553 conn(0x7f2bec079350 msgr2=0x7f2bec079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 -- 172.21.15.112:0/1865234553 shutdown_connections 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.821+0000 7f2bf409a700 1 -- 172.21.15.112:0/1865234553 wait complete. 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.822+0000 7f2bf409a700 1 Processor -- start 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.822+0000 7f2bf409a700 1 -- start start 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.822+0000 7f2bf409a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec1023a0 0x7f2bec19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf409a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bec102d60 0x7f2bec19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec1023a0 0x7f2bec19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec1023a0 0x7f2bec19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38252/0 (socket says 172.21.15.112:38252) 2024-05-10T10:41:36.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 -- 172.21.15.112:0/2409600813 learned_addr learned my addr 172.21.15.112:0/2409600813 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:36.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf409a700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bec108410 0x7f2bec199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1635700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bec102d60 0x7f2bec19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1134f0 con 0x7f2bec1023a0 2024-05-10T10:41:36.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec113370 con 0x7f2bec108410 2024-05-10T10:41:36.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec113670 con 0x7f2bec102d60 2024-05-10T10:41:36.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf2637700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bec108410 0x7f2bec199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 -- 172.21.15.112:0/2409600813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bec102d60 msgr2=0x7f2bec19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bec102d60 0x7f2bec19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 -- 172.21.15.112:0/2409600813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bec108410 msgr2=0x7f2bec199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bec108410 0x7f2bec199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.823+0000 7f2bf1e36700 1 -- 172.21.15.112:0/2409600813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec199ff0 con 0x7f2bec1023a0 2024-05-10T10:41:36.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.824+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec1023a0 0x7f2bec19f6d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2be8006010 tx=0x7f2be800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:36.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.824+0000 7f2be2ffd700 1 -- 172.21.15.112:0/2409600813 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be800fb00 con 0x7f2bec1023a0 2024-05-10T10:41:36.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.824+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec19a280 con 0x7f2bec1023a0 2024-05-10T10:41:36.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.824+0000 7f2be2ffd700 1 -- 172.21.15.112:0/2409600813 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be800fca0 con 0x7f2bec1023a0 2024-05-10T10:41:36.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.824+0000 7f2be2ffd700 1 -- 172.21.15.112:0/2409600813 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8028630 con 0x7f2bec1023a0 2024-05-10T10:41:36.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.824+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec19a6a0 con 0x7f2bec1023a0 2024-05-10T10:41:36.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.828+0000 7f2be2ffd700 1 -- 172.21.15.112:0/2409600813 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2be8009d10 con 0x7f2bec1023a0 2024-05-10T10:41:36.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.829+0000 7f2be2ffd700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2bd80b2900 0x7f2bd80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.829+0000 7f2be2ffd700 1 -- 172.21.15.112:0/2409600813 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2be80e7930 con 0x7f2bec1023a0 2024-05-10T10:41:36.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.829+0000 7f2bf1635700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2bd80b2900 0x7f2bd80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.830+0000 7f2bf1635700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2bd80b2900 0x7f2bd80b4dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2bec079a30 tx=0x7f2bdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:36.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.830+0000 7f2bfe92db80 1 Processor -- start 2024-05-10T10:41:36.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.830+0000 7f2bfe92db80 1 -- start start 2024-05-10T10:41:36.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.830+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e63590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.830+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e5d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e5dd50 0x556722e5e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e63590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556722daf940 con 0x556722e631b0 2024-05-10T10:41:36.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556722d14da0 con 0x556722e5dd50 2024-05-10T10:41:36.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e63590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38262/0 (socket says 172.21.15.112:38262) 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556722dab900 con 0x556722e63ad0 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1e36700 1 -- 172.21.15.112:0/3081632224 learned_addr learned my addr 172.21.15.112:0/3081632224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1635700 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e5d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf2637700 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e5dd50 0x556722e5e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1635700 1 -- 172.21.15.112:0/3081632224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e5dd50 msgr2=0x556722e5e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1635700 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e5dd50 0x556722e5e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1635700 1 -- 172.21.15.112:0/3081632224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 msgr2=0x556722e63590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1635700 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e63590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1635700 1 -- 172.21.15.112:0/3081632224 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556722cfc8d0 con 0x556722e63ad0 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1635700 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e5d810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc011c90 tx=0x7f2bdc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbd66bb1a54c58e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.831+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e63590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.832+0000 7f2bd7fff700 1 -- 172.21.15.112:0/3081632224 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004230 con 0x556722e63ad0 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.832+0000 7f2bd7fff700 1 -- 172.21.15.112:0/3081632224 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bdc0043d0 con 0x556722e63ad0 2024-05-10T10:41:36.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.832+0000 7f2bd7fff700 1 -- 172.21.15.112:0/3081632224 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc003cc0 con 0x556722e63ad0 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 -- 172.21.15.112:0/3081632224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 msgr2=0x556722e5d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e5d810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc011c90 tx=0x7f2bdc014000 comp rx=0 tx=0).stop 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 -- 172.21.15.112:0/3081632224 shutdown_connections 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e5dd50 0x556722e5e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e63590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/3081632224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e5d810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 -- 172.21.15.112:0/3081632224 >> 172.21.15.112:0/3081632224 conn(0x556722d89b70 msgr2=0x556722e582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 -- 172.21.15.112:0/3081632224 shutdown_connections 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 -- 172.21.15.112:0/3081632224 wait complete. 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.833+0000 7f2bfe92db80 1 -- start start 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 1 Processor -- start 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e69770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e66b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e67050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556722e72b70 con 0x556722e5dd50 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556722d68ed0 con 0x556722e631b0 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556722e72cf0 con 0x556722e63ad0 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e69770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bfe92db80 10 client.0 _ll_register_callbacks cb 0x556722d435b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e69770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38264/0 (socket says 172.21.15.112:38264) 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bf1e36700 1 -- 172.21.15.112:0/505304705 learned_addr learned my addr 172.21.15.112:0/505304705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bf2637700 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e67050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.834+0000 7f2bf1635700 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e66b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bf1e36700 1 -- 172.21.15.112:0/505304705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 msgr2=0x556722e67050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e67050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bf1e36700 1 -- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 msgr2=0x556722e66b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:36.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e66b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bf1e36700 1 -- 172.21.15.112:0/505304705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556722d66760 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e69770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ae8f0 tx=0x7f2be80ab110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4569696281672bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bd57fa700 1 -- 172.21.15.112:0/505304705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80aa380 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bd57fa700 1 -- 172.21.15.112:0/505304705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2be80b7740 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.835+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bdc0043d0 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bd57fa700 1 -- 172.21.15.112:0/505304705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2be80b7300 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 10 client.15696 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556722e7d370 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bd57fa700 1 -- 172.21.15.112:0/505304705 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bdc0043d0 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bd57fa700 1 -- 172.21.15.112:0/505304705 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2be8005110 con 0x556722e5dd50 2024-05-10T10:41:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 10 client.15696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 20 client.15696 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722e7f930 0x556722e81d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 4 client.15696 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556722dbd320 con 0x556722e7f930 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.836+0000 7f2bf1635700 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722e7f930 0x556722e81d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.837+0000 7f2bd57fa700 10 client.15696 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.837+0000 7f2bf1635700 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722e7f930 0x556722e81d70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81464e5b586d8773 server_cookie=3aca1907693ad245 in_seq=0 out_seq=0 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.838+0000 7f2bd57fa700 1 -- 172.21.15.112:0/505304705 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556722dbd320 con 0x556722e7f930 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.838+0000 7f2bd57fa700 10 client.15696 handle_command_reply: tid=0 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.838+0000 7f2bfe92db80 1 client.15696 shutdown 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 msgr2=0x556722e69770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e69770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ae8f0 tx=0x7f2be80ab110 comp rx=0 tx=0).stop 2024-05-10T10:41:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 shutdown_connections 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e67050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722e7f930 0x556722e81d70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e69770 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/505304705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e66b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bd57fa700 0 client.15696 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 >> 172.21.15.112:0/505304705 conn(0x556722d89b70 msgr2=0x556722e52b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 shutdown_connections 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 1 -- 172.21.15.112:0/505304705 wait complete. 2024-05-10T10:41:36.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.839+0000 7f2bfe92db80 20 client.15696 trim_cache size 0 max 16384 2024-05-10T10:41:36.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.868+0000 7f2bfe92db80 1 Processor -- start 2024-05-10T10:41:36.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.868+0000 7f2bfe92db80 1 -- start start 2024-05-10T10:41:36.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e5d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e5d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e5d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38274/0 (socket says 172.21.15.112:38274) 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e7bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 -- 172.21.15.112:0/1444367225 learned_addr learned my addr 172.21.15.112:0/1444367225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e52c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1ace90 con 0x556722e5dd50 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1ae9d0 con 0x556722e631b0 2024-05-10T10:41:36.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1ad560 con 0x556722e63ad0 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1635700 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e7bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 -- 172.21.15.112:0/1444367225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 msgr2=0x556722e52c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e52c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 -- 172.21.15.112:0/1444367225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 msgr2=0x556722e7bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e7bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1e36700 1 -- 172.21.15.112:0/1444367225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556722e59d00 con 0x556722e5dd50 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.869+0000 7f2bf1635700 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e7bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:36.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.870+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e5d810 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009580 tx=0x7f2be8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a7b222dea3780f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:36.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.870+0000 7f2bd77fe700 1 -- 172.21.15.112:0/1444367225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80aa380 con 0x556722e5dd50 2024-05-10T10:41:36.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.870+0000 7f2bd77fe700 1 -- 172.21.15.112:0/1444367225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be80ae700 con 0x556722e5dd50 2024-05-10T10:41:36.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.870+0000 7f2bd77fe700 1 -- 172.21.15.112:0/1444367225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80b7a10 con 0x556722e5dd50 2024-05-10T10:41:36.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 msgr2=0x556722e5d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e5d810 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009580 tx=0x7f2be8009310 comp rx=0 tx=0).stop 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 shutdown_connections 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e63ad0 0x556722e52c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e5dd50 0x556722e5d810 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1444367225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e631b0 0x556722e7bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 >> 172.21.15.112:0/1444367225 conn(0x556722d89b70 msgr2=0x556722e65400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 shutdown_connections 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.871+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1444367225 wait complete. 2024-05-10T10:41:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.872+0000 7f2bfe92db80 1 -- start start 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.872+0000 7f2bfe92db80 1 Processor -- start 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.872+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e5dd50 0x556722e73070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.872+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bfe92db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e63ad0 0x556722e73af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1ae300 con 0x556722e631b0 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1adc30 con 0x556722e63ad0 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bfe92db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1aae30 con 0x556722e5dd50 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bfe92db80 10 client.0 _ll_register_callbacks cb 0x556722d435b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf1635700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf1635700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38284/0 (socket says 172.21.15.112:38284) 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf1635700 1 -- 172.21.15.112:0/1794429913 learned_addr learned my addr 172.21.15.112:0/1794429913 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf2637700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e63ad0 0x556722e73af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e5dd50 0x556722e73070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf2637700 1 -- 172.21.15.112:0/1794429913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e5dd50 msgr2=0x556722e73070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf2637700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e5dd50 0x556722e73070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf2637700 1 -- 172.21.15.112:0/1794429913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 msgr2=0x556722e735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf2637700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf2637700 1 -- 172.21.15.112:0/1794429913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556722ee8440 con 0x556722e63ad0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.873+0000 7f2bf1635700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bf2637700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e63ad0 0x556722e73af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2be4007af0 tx=0x7f2be4007eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633b58458d54c29f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bd57fa700 1 -- 172.21.15.112:0/1794429913 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be401b030 con 0x556722e63ad0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be80ae700 con 0x556722e63ad0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bd57fa700 1 -- 172.21.15.112:0/1794429913 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2be4012040 con 0x556722e63ad0 2024-05-10T10:41:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bd57fa700 1 -- 172.21.15.112:0/1794429913 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2be4027050 con 0x556722e63ad0 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bfe92db80 10 client.25169 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556722ee9760 con 0x556722e63ad0 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bd57fa700 1 -- 172.21.15.112:0/1794429913 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be80ae700 con 0x556722e63ad0 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.874+0000 7f2bd57fa700 1 -- 172.21.15.112:0/1794429913 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2be4017a60 con 0x556722e63ad0 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bfe92db80 10 client.25169 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bfe92db80 10 client.25169 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bfe92db80 20 client.25169 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722ee9e50 0x556722eec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bfe92db80 4 client.25169 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556722db73c0 con 0x556722ee9e50 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722ee9e50 0x556722eec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bd57fa700 10 client.25169 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bf1e36700 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722ee9e50 0x556722eec270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17c3b5c00497fc5 server_cookie=9fa05d49341a0307 in_seq=0 out_seq=0 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bd57fa700 1 -- 172.21.15.112:0/1794429913 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556722db73c0 con 0x556722ee9e50 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bd57fa700 10 client.25169 handle_command_reply: tid=0 2024-05-10T10:41:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.875+0000 7f2bfe92db80 1 client.25169 shutdown 2024-05-10T10:41:36.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e63ad0 msgr2=0x556722e73af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e63ad0 0x556722e73af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2be4007af0 tx=0x7f2be4007eb0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 shutdown_connections 2024-05-10T10:41:36.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556722e63ad0 0x556722e73af0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556722e5dd50 0x556722e73070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556722ee9e50 0x556722eec270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 --2- 172.21.15.112:0/1794429913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556722e631b0 0x556722e735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bd57fa700 0 client.25169 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 >> 172.21.15.112:0/1794429913 conn(0x556722d89b70 msgr2=0x556722e624f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 shutdown_connections 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.876+0000 7f2bfe92db80 1 -- 172.21.15.112:0/1794429913 wait complete. 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.878+0000 7f2bfe92db80 20 client.25169 trim_cache size 0 max 16384 2024-05-10T10:41:36.883 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.878+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2bd80b2900 msgr2=0x7f2bd80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.878+0000 7f2bf409a700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2bd80b2900 0x7f2bd80b4dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2bec079a30 tx=0x7f2bdc009260 comp rx=0 tx=0).stop 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec1023a0 msgr2=0x7f2bec19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec1023a0 0x7f2bec19f6d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2be8006010 tx=0x7f2be800bdb0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 shutdown_connections 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bec108410 0x7f2bec199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bec1023a0 0x7f2bec19f6d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2bd80b2900 0x7f2bd80b4dc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 --2- 172.21.15.112:0/2409600813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bec102d60 0x7f2bec19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 >> 172.21.15.112:0/2409600813 conn(0x7f2bec079350 msgr2=0x7f2bec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 shutdown_connections 2024-05-10T10:41:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:36.879+0000 7f2bf409a700 1 -- 172.21.15.112:0/2409600813 wait complete. 2024-05-10T10:41:37.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:36 smithi148 ceph-mon[33568]: pgmap v471: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:37.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:36 smithi195 ceph-mon[33773]: pgmap v471: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:37.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:36 smithi112 ceph-mon[30100]: pgmap v471: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.590+0000 7fa59e32b640 1 -- 172.21.15.112:0/134504589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 msgr2=0x7fa598105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.590+0000 7fa59e32b640 1 --2- 172.21.15.112:0/134504589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 0x7fa598105880 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa588009900 tx=0x7fa58802a3f0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.590+0000 7fa59e32b640 1 -- 172.21.15.112:0/134504589 shutdown_connections 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.590+0000 7fa59e32b640 1 --2- 172.21.15.112:0/134504589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5981086f0 0x7fa59810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.590+0000 7fa59e32b640 1 --2- 172.21.15.112:0/134504589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa598105dc0 0x7fa5981081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.590+0000 7fa59e32b640 1 --2- 172.21.15.112:0/134504589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 0x7fa598105880 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.590+0000 7fa59e32b640 1 -- 172.21.15.112:0/134504589 >> 172.21.15.112:0/134504589 conn(0x7fa5980fce80 msgr2=0x7fa5980ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.591+0000 7fa59e32b640 1 -- 172.21.15.112:0/134504589 shutdown_connections 2024-05-10T10:41:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.591+0000 7fa59e32b640 1 -- 172.21.15.112:0/134504589 wait complete. 2024-05-10T10:41:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.591+0000 7fa59e32b640 1 Processor -- start 2024-05-10T10:41:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.591+0000 7fa59e32b640 1 -- start start 2024-05-10T10:41:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59e32b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 0x7fa598198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59e32b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa598105dc0 0x7fa598199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59e32b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5981086f0 0x7fa5981a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59e32b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa59810e420 con 0x7fa598103490 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59e32b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa59810e2a0 con 0x7fa598105dc0 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59e32b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa59810e5a0 con 0x7fa5981086f0 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa597fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 0x7fa598198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59c8c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5981086f0 0x7fa5981a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59c8c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5981086f0 0x7fa5981a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:34464/0 (socket says 172.21.15.112:34464) 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1090079069 learned_addr learned my addr 172.21.15.112:0/1090079069 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa5977fe640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa598105dc0 0x7fa598199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1090079069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa598105dc0 msgr2=0x7fa598199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59c8c8640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa598105dc0 0x7fa598199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.592+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 msgr2=0x7fa598198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa59c8c8640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 0x7fa598198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1090079069 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5981a06e0 con 0x7fa5981086f0 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa5977fe640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa598105dc0 0x7fa598199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa597fff640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 0x7fa598198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa59c8c8640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5981086f0 0x7fa5981a0070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa59000cce0 tx=0x7fa59000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa5957fa640 1 -- 172.21.15.112:0/1090079069 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59000f450 con 0x7fa5981086f0 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa5957fa640 1 -- 172.21.15.112:0/1090079069 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa59000fa80 con 0x7fa5981086f0 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5981a09d0 con 0x7fa5981086f0 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa5957fa640 1 -- 172.21.15.112:0/1090079069 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590018ad0 con 0x7fa5981086f0 2024-05-10T10:41:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.593+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5981a0e70 con 0x7fa5981086f0 2024-05-10T10:41:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 Processor -- start 2024-05-10T10:41:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5957fa640 1 -- 172.21.15.112:0/1090079069 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa59000f5f0 con 0x7fa5981086f0 2024-05-10T10:41:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 -- start start 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215df030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215df600 0x564c215e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215e4150 0x564c215e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c2153a5d0 con 0x564c215df600 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564c214f3890 con 0x564c215e4150 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa5a1047740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c21536590 con 0x564c215dec10 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa597fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215df600 0x564c215e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa597fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215df600 0x564c215e3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38302/0 (socket says 172.21.15.112:38302) 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.597+0000 7fa597fff640 1 -- 172.21.15.112:0/123055740 learned_addr learned my addr 172.21.15.112:0/123055740 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa597fff640 1 -- 172.21.15.112:0/123055740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 msgr2=0x564c215df030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa597fff640 1 --2- 172.21.15.112:0/123055740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa597fff640 1 -- 172.21.15.112:0/123055740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215e4150 msgr2=0x564c215e65a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa597fff640 1 --2- 172.21.15.112:0/123055740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215e4150 0x564c215e65a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa597fff640 1 -- 172.21.15.112:0/123055740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c2124c210 con 0x564c215df600 2024-05-10T10:41:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa5957fa640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5800b28a0 0x7fa5800b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa597fff640 1 --2- 172.21.15.112:0/123055740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215df600 0x564c215e3ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa58802a910 tx=0x7fa588005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32f43eb7b6de990 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa5957fa640 1 -- 172.21.15.112:0/1090079069 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa5900e2d70 con 0x7fa5981086f0 2024-05-10T10:41:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa5977fe640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5800b28a0 0x7fa5800b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa57e7fc640 1 -- 172.21.15.112:0/123055740 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588035030 con 0x564c215df600 2024-05-10T10:41:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.598+0000 7fa57e7fc640 1 -- 172.21.15.112:0/123055740 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa588040070 con 0x564c215df600 2024-05-10T10:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.599+0000 7fa5a1047740 1 -- 172.21.15.112:0/123055740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215df600 msgr2=0x564c215e3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.599+0000 7fa5a1047740 1 --2- 172.21.15.112:0/123055740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215df600 0x564c215e3ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa58802a910 tx=0x7fa588005e80 comp rx=0 tx=0).stop 2024-05-10T10:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.599+0000 7fa5977fe640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5800b28a0 0x7fa5800b4d60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x564c215d4f60 tx=0x7fa58c009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- 172.21.15.112:0/123055740 shutdown_connections 2024-05-10T10:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 --2- 172.21.15.112:0/123055740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215e4150 0x564c215e65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 --2- 172.21.15.112:0/123055740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215df600 0x564c215e3ae0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 --2- 172.21.15.112:0/123055740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215df030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- 172.21.15.112:0/123055740 >> 172.21.15.112:0/123055740 conn(0x564c215d13a0 msgr2=0x564c215d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- 172.21.15.112:0/123055740 shutdown_connections 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- 172.21.15.112:0/123055740 wait complete. 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- start start 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 Processor -- start 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 0x564c215f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c215e70b0 con 0x564c215e4150 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564c214f7370 con 0x564c215edf10 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.600+0000 7fa5a1047740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c215e7230 con 0x564c215dec10 2024-05-10T10:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5a1047740 10 client.0 _ll_register_callbacks cb 0x564c2124ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215e8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38310/0 (socket says 172.21.15.112:38310) 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 -- 172.21.15.112:0/2393892222 learned_addr learned my addr 172.21.15.112:0/2393892222 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa597fff640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa59c8c8640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 0x564c215f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 -- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 msgr2=0x564c215e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 -- 172.21.15.112:0/2393892222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 msgr2=0x564c215f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 0x564c215f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 -- 172.21.15.112:0/2393892222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c21318620 con 0x564c215e4150 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa597fff640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5977fe640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215e8f50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa58c012780 tx=0x7fa58c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee76b3afac7740 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5777fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5777fe640 1 -- 172.21.15.112:0/2393892222 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00fb50 con 0x564c215e4150 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5777fe640 1 -- 172.21.15.112:0/2393892222 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa58c00fcf0 con 0x564c215e4150 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.601+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa588040070 con 0x564c215e4150 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5777fe640 1 -- 172.21.15.112:0/2393892222 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa58c01f420 con 0x564c215e4150 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 10 client.15714 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564c215f33b0 con 0x564c215e4150 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5777fe640 1 -- 172.21.15.112:0/2393892222 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa58c01fa50 con 0x564c215e4150 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5777fe640 1 -- 172.21.15.112:0/2393892222 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa58c0275e0 con 0x564c215e4150 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 10 client.15714 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 20 client.15714 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c215f4ec0 0x564c215f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 4 client.15714 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa588040070 con 0x564c215f4ec0 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.602+0000 7fa597fff640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c215f4ec0 0x564c215f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.603+0000 7fa5777fe640 10 client.15714 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.603+0000 7fa597fff640 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c215f4ec0 0x564c215f7360 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b1f9c62245f1f2 server_cookie=8339b6b4f72fc178 in_seq=0 out_seq=0 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5777fe640 1 -- 172.21.15.112:0/2393892222 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fa588040070 con 0x564c215f4ec0 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5777fe640 10 client.15714 handle_command_reply: tid=0 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 client.15714 shutdown 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 msgr2=0x564c215e8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215e8f50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa58c012780 tx=0x7fa58c005f90 comp rx=0 tx=0).stop 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 shutdown_connections 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c215f4ec0 0x564c215f7360 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 0x564c215f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215e8f50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 --2- 172.21.15.112:0/2393892222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 >> 172.21.15.112:0/2393892222 conn(0x564c215d13a0 msgr2=0x564c215de2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5777fe640 0 client.15714 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 shutdown_connections 2024-05-10T10:41:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 1 -- 172.21.15.112:0/2393892222 wait complete. 2024-05-10T10:41:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.604+0000 7fa5a1047740 20 client.15714 trim_cache size 0 max 16384 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 Processor -- start 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 -- start start 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215dfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38324/0 (socket says 172.21.15.112:38324) 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5977fe640 1 -- 172.21.15.112:0/891757095 learned_addr learned my addr 172.21.15.112:0/891757095 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 0x564c215cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981a9270 con 0x564c215e4150 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981aadb0 con 0x564c215edf10 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.619+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981a9940 con 0x564c215dec10 2024-05-10T10:41:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa597fff640 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa5977fe640 1 -- 172.21.15.112:0/891757095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 msgr2=0x564c215e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa5977fe640 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa5977fe640 1 -- 172.21.15.112:0/891757095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 msgr2=0x564c215cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa5977fe640 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 0x564c215cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa5977fe640 1 -- 172.21.15.112:0/891757095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c215d11d0 con 0x564c215e4150 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa597fff640 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa5977fe640 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215dfa60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa58c012750 tx=0x7fa58c012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fec4713a2c3c1e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa57dffb640 1 -- 172.21.15.112:0/891757095 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c03a070 con 0x564c215e4150 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa57dffb640 1 -- 172.21.15.112:0/891757095 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa58c005030 con 0x564c215e4150 2024-05-10T10:41:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.620+0000 7fa57dffb640 1 -- 172.21.15.112:0/891757095 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00fa00 con 0x564c215e4150 2024-05-10T10:41:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 msgr2=0x564c215dfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215dfa60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa58c012750 tx=0x7fa58c012f10 comp rx=0 tx=0).stop 2024-05-10T10:41:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 shutdown_connections 2024-05-10T10:41:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215edf10 0x564c215cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215e4150 0x564c215dfa60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 --2- 172.21.15.112:0/891757095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215dec10 0x564c215e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 >> 172.21.15.112:0/891757095 conn(0x564c215d13a0 msgr2=0x564c215d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 shutdown_connections 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.621+0000 7fa5a1047740 1 -- 172.21.15.112:0/891757095 wait complete. 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.622+0000 7fa5a1047740 1 -- start start 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.622+0000 7fa5a1047740 1 Processor -- start 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215dec10 0x564c21605f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215e4150 0x564c216064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215edf10 0x564c216074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981aa6e0 con 0x564c215edf10 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981aa010 con 0x564c215dec10 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981a7250 con 0x564c215e4150 2024-05-10T10:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215edf10 0x564c216074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215edf10 0x564c216074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38336/0 (socket says 172.21.15.112:38336) 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1188739412 learned_addr learned my addr 172.21.15.112:0/1188739412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa597fff640 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215dec10 0x564c21605f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5977fe640 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215e4150 0x564c216064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 10 client.0 _ll_register_callbacks cb 0x7fa58c011f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215e4150 msgr2=0x564c216064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215e4150 0x564c216064d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1188739412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215dec10 msgr2=0x564c21605f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215dec10 0x564c21605f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 -- 172.21.15.112:0/1188739412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c21607ad0 con 0x564c215edf10 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c005030 con 0x564c215edf10 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa59c8c8640 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215edf10 0x564c216074e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa5900a9f90 tx=0x7fa5900a9830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c507160694550bf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.623+0000 7fa5777fe640 10 client.15720 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5777fe640 1 -- 172.21.15.112:0/1188739412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900c3070 con 0x564c215edf10 2024-05-10T10:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5777fe640 1 -- 172.21.15.112:0/1188739412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa5900b2be0 con 0x564c215edf10 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5777fe640 1 -- 172.21.15.112:0/1188739412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5900aa000 con 0x564c215edf10 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5a1047740 10 client.15720 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=30}) -- 0x564c21608ce0 con 0x564c215edf10 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5777fe640 1 -- 172.21.15.112:0/1188739412 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa5900a9260 con 0x564c215edf10 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5a1047740 10 client.15720 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5a1047740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5a1047740 20 client.15720 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.624+0000 7fa5a1047740 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c2165f8d0 0x564c21661cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.625+0000 7fa5a1047740 4 client.15720 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.625+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c215481e0 con 0x564c2165f8d0 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.625+0000 7fa597fff640 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c2165f8d0 0x564c21661cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.625+0000 7fa597fff640 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c2165f8d0 0x564c21661cf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7867284797a44f4 server_cookie=35ec48917f70292b in_seq=0 out_seq=0 2024-05-10T10:41:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.625+0000 7fa5777fe640 10 client.15720 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5777fe640 1 -- 172.21.15.112:0/1188739412 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c215481e0 con 0x564c2165f8d0 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5777fe640 10 client.15720 handle_command_reply: tid=0 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 client.15720 shutdown 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215edf10 msgr2=0x564c216074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215edf10 0x564c216074e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa5900a9f90 tx=0x7fa5900a9830 comp rx=0 tx=0).stop 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 shutdown_connections 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c215edf10 0x564c216074e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c215e4150 0x564c216064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564c2165f8d0 0x564c21661cf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 --2- 172.21.15.112:0/1188739412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c215dec10 0x564c21605f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 >> 172.21.15.112:0/1188739412 conn(0x564c215d13a0 msgr2=0x564c215cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5777fe640 0 client.15720 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 shutdown_connections 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 1 -- 172.21.15.112:0/1188739412 wait complete. 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.629+0000 7fa5a1047740 20 client.15720 trim_cache size 0 max 16384 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:41:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5800b28a0 msgr2=0x7fa5800b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5800b28a0 0x7fa5800b4d60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x564c215d4f60 tx=0x7fa58c009530 comp rx=0 tx=0).stop 2024-05-10T10:41:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5981086f0 msgr2=0x7fa5981a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5981086f0 0x7fa5981a0070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa59000cce0 tx=0x7fa59000a620 comp rx=0 tx=0).stop 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 shutdown_connections 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa5800b28a0 0x7fa5800b4d60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5981086f0 0x7fa5981a0070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa598105dc0 0x7fa598199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 --2- 172.21.15.112:0/1090079069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa598103490 0x7fa598198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 >> 172.21.15.112:0/1090079069 conn(0x7fa5980fce80 msgr2=0x7fa598103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 shutdown_connections 2024-05-10T10:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:37.630+0000 7fa59e32b640 1 -- 172.21.15.112:0/1090079069 wait complete. 2024-05-10T10:41:37.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:41:37.998 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:38.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:38 smithi112 ceph-mon[30100]: pgmap v472: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:38.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:38 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:41:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:38 smithi148 ceph-mon[33568]: pgmap v472: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:38 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:41:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:38 smithi195 ceph-mon[33773]: pgmap v472: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:39.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:38 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 -- 172.21.15.112:0/1711011013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7120109540 msgr2=0x7f7120109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 --2- 172.21.15.112:0/1711011013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7120109540 0x7f7120109920 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7110009990 tx=0x7f7110009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 -- 172.21.15.112:0/1711011013 shutdown_connections 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 --2- 172.21.15.112:0/1711011013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f712006c530 0x7f7120110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 --2- 172.21.15.112:0/1711011013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7120109540 0x7f7120109920 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 --2- 172.21.15.112:0/1711011013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 0x7f712006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 -- 172.21.15.112:0/1711011013 >> 172.21.15.112:0/1711011013 conn(0x7f71200793e0 msgr2=0x7f71200797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 -- 172.21.15.112:0/1711011013 shutdown_connections 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.858+0000 7f71251ba700 1 -- 172.21.15.112:0/1711011013 wait complete. 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.859+0000 7f71251ba700 1 Processor -- start 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.859+0000 7f71251ba700 1 -- start start 2024-05-10T10:41:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.859+0000 7f71251ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 0x7f71201a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.859+0000 7f71251ba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f712006c530 0x7f71201a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f71251ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7120109540 0x7f712010e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f71251ba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7120113520 con 0x7f712006c530 2024-05-10T10:41:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f71251ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71201133a0 con 0x7f7120109540 2024-05-10T10:41:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f71251ba700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f71201136a0 con 0x7f712006bb70 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 0x7f71201a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 0x7f71201a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:34518/0 (socket says 172.21.15.112:34518) 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 -- 172.21.15.112:0/1004760211 learned_addr learned my addr 172.21.15.112:0/1004760211 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711f59e700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7120109540 0x7f712010e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711e59c700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f712006c530 0x7f71201a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 -- 172.21.15.112:0/1004760211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7120109540 msgr2=0x7f712010e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7120109540 0x7f712010e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 -- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f712006c530 msgr2=0x7f71201a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f712006c530 0x7f71201a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 -- 172.21.15.112:0/1004760211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712010ee80 con 0x7f712006bb70 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711f59e700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7120109540 0x7f712010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.860+0000 7f711ed9d700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 0x7f71201a0780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7110006010 tx=0x7f711000bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.861+0000 7f7107fff700 1 -- 172.21.15.112:0/1004760211 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110027070 con 0x7f712006bb70 2024-05-10T10:41:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.861+0000 7f7107fff700 1 -- 172.21.15.112:0/1004760211 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7110022470 con 0x7f712006bb70 2024-05-10T10:41:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.861+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712010f110 con 0x7f712006bb70 2024-05-10T10:41:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.861+0000 7f7107fff700 1 -- 172.21.15.112:0/1004760211 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110017650 con 0x7f712006bb70 2024-05-10T10:41:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.861+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712019c330 con 0x7f712006bb70 2024-05-10T10:41:39.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.864+0000 7f7107fff700 1 -- 172.21.15.112:0/1004760211 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7110009d10 con 0x7f712006bb70 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.865+0000 7f7107fff700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f71080b2980 0x7f71080b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.865+0000 7f7107fff700 1 -- 172.21.15.112:0/1004760211 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f71100e3240 con 0x7f712006bb70 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.865+0000 7f712fa4db80 1 Processor -- start 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f711e59c700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f71080b2980 0x7f71080b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f712fa4db80 1 -- start start 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00faad0 0x55e7e00f4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00f4d50 0x55e7e00f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f711ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f711ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38364/0 (socket says 172.21.15.112:38364) 2024-05-10T10:41:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f711f59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00f4d50 0x55e7e00f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f711ed9d700 1 -- 172.21.15.112:0/623807401 learned_addr learned my addr 172.21.15.112:0/623807401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:39.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e7e0046940 con 0x55e7e00fa1b0 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e7dffabda0 con 0x55e7e00faad0 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7e0042900 con 0x55e7e00f4d50 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f711e59c700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f71080b2980 0x7f71080b4e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f712010f650 tx=0x7f7114009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.866+0000 7f711e59c700 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00faad0 0x55e7e00f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f711ed9d700 1 -- 172.21.15.112:0/623807401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00f4d50 msgr2=0x55e7e00f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f711ed9d700 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00f4d50 0x55e7e00f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f711ed9d700 1 -- 172.21.15.112:0/623807401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00faad0 msgr2=0x55e7e00f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f711ed9d700 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00faad0 0x55e7e00f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f711ed9d700 1 -- 172.21.15.112:0/623807401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7dff938d0 con 0x55e7e00fa1b0 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f711f59e700 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00f4d50 0x55e7e00f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f711ed9d700 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fa590 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f71100aa490 tx=0x7f71100a9f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc12cc4ebbfeee11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f71057fa700 1 -- 172.21.15.112:0/623807401 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100aa500 con 0x55e7e00fa1b0 2024-05-10T10:41:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.867+0000 7f71057fa700 1 -- 172.21.15.112:0/623807401 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71100ae8b0 con 0x55e7e00fa1b0 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f71057fa700 1 -- 172.21.15.112:0/623807401 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100aa9c0 con 0x55e7e00fa1b0 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 msgr2=0x55e7e00fa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fa590 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f71100aa490 tx=0x7f71100a9f60 comp rx=0 tx=0).stop 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 shutdown_connections 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00f4d50 0x55e7e00f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fa590 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 --2- 172.21.15.112:0/623807401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00faad0 0x55e7e00f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 >> 172.21.15.112:0/623807401 conn(0x55e7e0020b70 msgr2=0x55e7e00ef2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 shutdown_connections 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.868+0000 7f712fa4db80 1 -- 172.21.15.112:0/623807401 wait complete. 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 -- start start 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 Processor -- start 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0100630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00faad0 0x55e7e00fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e7e0101210 con 0x55e7e00faad0 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e7dffffed0 con 0x55e7e00f4d50 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7e0101390 con 0x55e7e00fa1b0 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f712fa4db80 10 client.0 _ll_register_callbacks cb 0x55e7dffda5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f711f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00faad0 0x55e7e00fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f711f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00faad0 0x55e7e00fdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38376/0 (socket says 172.21.15.112:38376) 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.869+0000 7f711f59e700 1 -- 172.21.15.112:0/3245775125 learned_addr learned my addr 172.21.15.112:0/3245775125 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711e59c700 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711ed9d700 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711f59e700 1 -- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00fa1b0 msgr2=0x55e7e00fd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711f59e700 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711f59e700 1 -- 172.21.15.112:0/3245775125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 msgr2=0x55e7e0100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711f59e700 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711f59e700 1 -- 172.21.15.112:0/3245775125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7dfffd760 con 0x55e7e00faad0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f711f59e700 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00faad0 0x55e7e00fdf10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7118007b20 tx=0x7f7118007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42da4d1d08916c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f7102ff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f7102ff5700 1 -- 172.21.15.112:0/3245775125 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711801a020 con 0x55e7e00faad0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71100ae8b0 con 0x55e7e00faad0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.870+0000 7f7102ff5700 1 -- 172.21.15.112:0/3245775125 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7118011930 con 0x55e7e00faad0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f7102ff5700 1 -- 172.21.15.112:0/3245775125 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7118025050 con 0x55e7e00faad0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 10 client.15732 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7e011ca80 con 0x55e7e00faad0 2024-05-10T10:41:39.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f7102ff5700 1 -- 172.21.15.112:0/3245775125 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71100ae8b0 con 0x55e7e00faad0 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f7102ff5700 1 -- 172.21.15.112:0/3245775125 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7118016c00 con 0x55e7e00faad0 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 10 client.15732 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 10 client.15732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 20 client.15732 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e011f470 0x55e7e01218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 4 client.15732 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.871+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7e0054320 con 0x55e7e011f470 2024-05-10T10:41:39.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.872+0000 7f711ed9d700 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e011f470 0x55e7e01218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.872+0000 7f7102ff5700 10 client.15732 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.872+0000 7f711ed9d700 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e011f470 0x55e7e01218f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a38d887db3c461bb server_cookie=cd0a9bb5b2c245c6 in_seq=0 out_seq=0 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f7102ff5700 1 -- 172.21.15.112:0/3245775125 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7e0054320 con 0x55e7e011f470 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f7102ff5700 10 client.15732 handle_command_reply: tid=0 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 client.15732 shutdown 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00faad0 msgr2=0x55e7e00fdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00faad0 0x55e7e00fdf10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7118007b20 tx=0x7f7118007ee0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 shutdown_connections 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e011f470 0x55e7e01218f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00faad0 0x55e7e00fdf10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f7102ff5700 0 client.15732 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 --2- 172.21.15.112:0/3245775125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00fa1b0 0x55e7e00fd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 >> 172.21.15.112:0/3245775125 conn(0x55e7e0020b70 msgr2=0x55e7e00e9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 shutdown_connections 2024-05-10T10:41:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 1 -- 172.21.15.112:0/3245775125 wait complete. 2024-05-10T10:41:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.873+0000 7f712fa4db80 20 client.15732 trim_cache size 0 max 16384 2024-05-10T10:41:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.902+0000 7f712fa4db80 1 Processor -- start 2024-05-10T10:41:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.902+0000 7f712fa4db80 1 -- start start 2024-05-10T10:41:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.902+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e00fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00e9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38386/0 (socket says 172.21.15.112:38386) 2024-05-10T10:41:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 -- 172.21.15.112:0/1260655814 learned_addr learned my addr 172.21.15.112:0/1260655814 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f712fa4db80 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e01271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7120074c90 con 0x55e7e00fa1b0 2024-05-10T10:41:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b7340 con 0x55e7e00f4d50 2024-05-10T10:41:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7120075360 con 0x55e7e00faad0 2024-05-10T10:41:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711ed9d700 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e00fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711f59e700 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e01271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 -- 172.21.15.112:0/1260655814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 msgr2=0x55e7e01271c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e01271c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 -- 172.21.15.112:0/1260655814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 msgr2=0x55e7e00fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e00fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711e59c700 1 -- 172.21.15.112:0/1260655814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e00fbf80 con 0x55e7e00fa1b0 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.903+0000 7f711ed9d700 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e00fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.904+0000 7f711e59c700 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00e9c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7120198f20 tx=0x7f7114014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebb10293b585a15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.904+0000 7f7104ff9700 1 -- 172.21.15.112:0/1260655814 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114003880 con 0x55e7e00fa1b0 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.904+0000 7f7104ff9700 1 -- 172.21.15.112:0/1260655814 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7114012450 con 0x55e7e00fa1b0 2024-05-10T10:41:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.904+0000 7f7104ff9700 1 -- 172.21.15.112:0/1260655814 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114020690 con 0x55e7e00fa1b0 2024-05-10T10:41:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 msgr2=0x55e7e00e9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00e9c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7120198f20 tx=0x7f7114014000 comp rx=0 tx=0).stop 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 shutdown_connections 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e01271c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e00fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 --2- 172.21.15.112:0/1260655814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e00fa1b0 0x55e7e00e9c40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 >> 172.21.15.112:0/1260655814 conn(0x55e7e0020b70 msgr2=0x55e7e00faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 shutdown_connections 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 -- 172.21.15.112:0/1260655814 wait complete. 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 -- start start 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.905+0000 7f712fa4db80 1 Processor -- start 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0117b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e0126590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f712fa4db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e0126ad0 0x55e7e0126f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f711f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e0126ad0 0x55e7e0126f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f711ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0117b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f711ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0117b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:39704/0 (socket says 172.21.15.112:39704) 2024-05-10T10:41:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f711ed9d700 1 -- 172.21.15.112:0/2010124949 learned_addr learned my addr 172.21.15.112:0/2010124949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:39.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b6c70 con 0x55e7e0126ad0 2024-05-10T10:41:39.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71201b65a0 con 0x55e7e00f4d50 2024-05-10T10:41:39.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7120072c30 con 0x55e7e00faad0 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.906+0000 7f711e59c700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e0126590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711ed9d700 1 -- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 msgr2=0x55e7e0126590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711ed9d700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e0126590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711ed9d700 1 -- 172.21.15.112:0/2010124949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e0126ad0 msgr2=0x55e7e0126f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711ed9d700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e0126ad0 0x55e7e0126f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711ed9d700 1 -- 172.21.15.112:0/2010124949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e0188df0 con 0x55e7e00f4d50 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711e59c700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e0126590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711f59e700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e0126ad0 0x55e7e0126f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f712fa4db80 10 client.0 _ll_register_callbacks cb 0x55e7dffda5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f711ed9d700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0117b90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71100a6580 tx=0x7f7110009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20d330978f2ca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f7102ff5700 10 client.25193 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7114012450 con 0x55e7e00f4d50 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f7102ff5700 1 -- 172.21.15.112:0/2010124949 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100c3070 con 0x55e7e00f4d50 2024-05-10T10:41:39.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f7102ff5700 1 -- 172.21.15.112:0/2010124949 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f71100cb330 con 0x55e7e00f4d50 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f7102ff5700 1 -- 172.21.15.112:0/2010124949 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71100aa500 con 0x55e7e00f4d50 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.907+0000 7f712fa4db80 10 client.25193 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e7e0189ec0 con 0x55e7e00f4d50 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f7102ff5700 1 -- 172.21.15.112:0/2010124949 <== mon.2 v2:172.21.15.148:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f71100aa760 con 0x55e7e00f4d50 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f712fa4db80 10 client.25193 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f712fa4db80 10 client.25193 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f712fa4db80 20 client.25193 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f712fa4db80 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e018a2f0 0x55e7e018c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f712fa4db80 4 client.25193 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7114012450 con 0x55e7e018a2f0 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.908+0000 7f711e59c700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e018a2f0 0x55e7e018c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f7102ff5700 10 client.25193 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f711e59c700 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e018a2f0 0x55e7e018c710 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=111fd0fd3b243359 server_cookie=8bdefdeb8a674397 in_seq=0 out_seq=0 2024-05-10T10:41:39.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f7102ff5700 1 -- 172.21.15.112:0/2010124949 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7114012450 con 0x55e7e018a2f0 2024-05-10T10:41:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f7102ff5700 10 client.25193 handle_command_reply: tid=0 2024-05-10T10:41:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 client.25193 shutdown 2024-05-10T10:41:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 msgr2=0x55e7e0117b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0117b90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71100a6580 tx=0x7f7110009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 shutdown_connections 2024-05-10T10:41:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55e7e018a2f0 0x55e7e018c710 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7e0126ad0 0x55e7e0126f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f7102ff5700 0 client.25193 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7e00f4d50 0x55e7e0117b90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 --2- 172.21.15.112:0/2010124949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7e00faad0 0x55e7e0126590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 >> 172.21.15.112:0/2010124949 conn(0x55e7e0020b70 msgr2=0x55e7e00ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.909+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 shutdown_connections 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f712fa4db80 1 -- 172.21.15.112:0/2010124949 wait complete. 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f712fa4db80 20 client.25193 trim_cache size 0 max 16384 2024-05-10T10:41:39.914 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f71080b2980 msgr2=0x7f71080b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f71251ba700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f71080b2980 0x7f71080b4e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f712010f650 tx=0x7f7114009260 comp rx=0 tx=0).stop 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 msgr2=0x7f71201a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f71251ba700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 0x7f71201a0780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7110006010 tx=0x7f711000bfc0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 shutdown_connections 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f71251ba700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7120109540 0x7f712010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.910+0000 7f71251ba700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f71080b2980 0x7f71080b4e40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.911+0000 7f71251ba700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f712006bb70 0x7f71201a0780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.911+0000 7f71251ba700 1 --2- 172.21.15.112:0/1004760211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f712006c530 0x7f71201a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:39.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.911+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 >> 172.21.15.112:0/1004760211 conn(0x7f71200793e0 msgr2=0x7f7120102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:39.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.911+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 shutdown_connections 2024-05-10T10:41:39.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:39.911+0000 7f71251ba700 1 -- 172.21.15.112:0/1004760211 wait complete. 2024-05-10T10:41:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:41:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:41:40.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:41:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:41:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:41:40.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:41:40.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:41:40.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:41:40.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:41:40.572 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:41:40.794 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:40 smithi112 ceph-mon[30100]: pgmap v473: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:41:40.946 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:41.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:40 smithi148 ceph-mon[33568]: pgmap v473: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:41:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:40 smithi195 ceph-mon[33773]: pgmap v473: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:41:42.797 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:42 smithi112 ceph-mon[30100]: pgmap v474: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 -- 172.21.15.112:0/1410970728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0107760 msgr2=0x7fefe010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 --2- 172.21.15.112:0/1410970728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0107760 0x7fefe010f2e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fefd800b0d0 tx=0x7fefd800b3e0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 -- 172.21.15.112:0/1410970728 shutdown_connections 2024-05-10T10:41:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 --2- 172.21.15.112:0/1410970728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0107760 0x7fefe010f2e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 --2- 172.21.15.112:0/1410970728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefe0102e00 0x7fefe01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 --2- 172.21.15.112:0/1410970728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe01037b0 0x7fefe0107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 -- 172.21.15.112:0/1410970728 >> 172.21.15.112:0/1410970728 conn(0x7fefe0078a10 msgr2=0x7fefe00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 -- 172.21.15.112:0/1410970728 shutdown_connections 2024-05-10T10:41:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.849+0000 7fefe6f49700 1 -- 172.21.15.112:0/1410970728 wait complete. 2024-05-10T10:41:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.850+0000 7fefe6f49700 1 Processor -- start 2024-05-10T10:41:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe6f49700 1 -- start start 2024-05-10T10:41:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe6f49700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0102e00 0x7fefe019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe6f49700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefe01037b0 0x7fefe019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe6f49700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe0107760 0x7fefe01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe6f49700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0112400 con 0x7fefe0102e00 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe6f49700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0112280 con 0x7fefe0107760 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe6f49700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0112580 con 0x7fefe01037b0 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe54e6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe0107760 0x7fefe01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe54e6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe0107760 0x7fefe01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:39722/0 (socket says 172.21.15.112:39722) 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe54e6700 1 -- 172.21.15.112:0/4256629076 learned_addr learned my addr 172.21.15.112:0/4256629076 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefdffff700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefe01037b0 0x7fefe019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.851+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0102e00 0x7fefe019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe54e6700 1 -- 172.21.15.112:0/4256629076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefe01037b0 msgr2=0x7fefe019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe54e6700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefe01037b0 0x7fefe019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe54e6700 1 -- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0102e00 msgr2=0x7fefe019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe54e6700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0102e00 0x7fefe019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe54e6700 1 -- 172.21.15.112:0/4256629076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe0198ef0 con 0x7fefe0107760 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0102e00 0x7fefe019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe54e6700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe0107760 0x7fefe01986e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009850 tx=0x7fefd8007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefddffb700 1 -- 172.21.15.112:0/4256629076 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd801f070 con 0x7fefe0107760 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefddffb700 1 -- 172.21.15.112:0/4256629076 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefd8005500 con 0x7fefe0107760 2024-05-10T10:41:42.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe0199180 con 0x7fefe0107760 2024-05-10T10:41:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.852+0000 7fefddffb700 1 -- 172.21.15.112:0/4256629076 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd801b9a0 con 0x7fefe0107760 2024-05-10T10:41:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.853+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe0101a90 con 0x7fefe0107760 2024-05-10T10:41:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.856+0000 7fefddffb700 1 -- 172.21.15.112:0/4256629076 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fefd8022070 con 0x7fefe0107760 2024-05-10T10:41:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.857+0000 7fefddffb700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fefd00b2980 0x7fefd00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.858+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fefd00b2980 0x7fefd00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.858+0000 7fefddffb700 1 -- 172.21.15.112:0/4256629076 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fefd800f020 con 0x7fefe0107760 2024-05-10T10:41:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.858+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fefd00b2980 0x7fefd00b4e40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fefd400bcf0 tx=0x7fefd400b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.859+0000 7feff17dcb80 1 Processor -- start 2024-05-10T10:41:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7feff17dcb80 1 -- start start 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e19d0 0x5623e47e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47e7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7feff17dcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623e4736940 con 0x5623e47e19d0 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7feff17dcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5623e469bda0 con 0x5623e47e74a0 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7feff17dcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623e4732900 con 0x5623e47e22f0 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e19d0 0x5623e47e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e19d0 0x5623e47e1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38430/0 (socket says 172.21.15.112:38430) 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 -- 172.21.15.112:0/2449153922 learned_addr learned my addr 172.21.15.112:0/2449153922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefdffff700 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe54e6700 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 -- 172.21.15.112:0/2449153922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 msgr2=0x5623e47e6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47e6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 -- 172.21.15.112:0/2449153922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 msgr2=0x5623e47e7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47e7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.860+0000 7fefe4ce5700 1 -- 172.21.15.112:0/2449153922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e46838d0 con 0x5623e47e19d0 2024-05-10T10:41:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.861+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e19d0 0x5623e47e1db0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fefd400ff30 tx=0x7fefd4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62f07687a6609e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.861+0000 7fefcaffd700 1 -- 172.21.15.112:0/2449153922 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4044070 con 0x5623e47e19d0 2024-05-10T10:41:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.862+0000 7fefcaffd700 1 -- 172.21.15.112:0/2449153922 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefd4011bb0 con 0x5623e47e19d0 2024-05-10T10:41:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.862+0000 7fefcaffd700 1 -- 172.21.15.112:0/2449153922 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd403f620 con 0x5623e47e19d0 2024-05-10T10:41:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 -- 172.21.15.112:0/2449153922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e19d0 msgr2=0x5623e47e1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e19d0 0x5623e47e1db0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fefd400ff30 tx=0x7fefd4012000 comp rx=0 tx=0).stop 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 -- 172.21.15.112:0/2449153922 shutdown_connections 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47e7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e19d0 0x5623e47e1db0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 --2- 172.21.15.112:0/2449153922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47e6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 -- 172.21.15.112:0/2449153922 >> 172.21.15.112:0/2449153922 conn(0x5623e4710b70 msgr2=0x5623e47df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 -- 172.21.15.112:0/2449153922 shutdown_connections 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 -- 172.21.15.112:0/2449153922 wait complete. 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.863+0000 7feff17dcb80 1 -- start start 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7feff17dcb80 1 Processor -- start 2024-05-10T10:41:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47f3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47ee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7feff17dcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623e47ed260 con 0x5623e47ee3a0 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7feff17dcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5623e46efed0 con 0x5623e47e74a0 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7fefe54e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47ee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7feff17dcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623e47ed3e0 con 0x5623e47e22f0 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7fefe54e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47ee820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38444/0 (socket says 172.21.15.112:38444) 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7fefe54e6700 1 -- 172.21.15.112:0/104801557 learned_addr learned my addr 172.21.15.112:0/104801557 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7feff17dcb80 10 client.0 _ll_register_callbacks cb 0x5623e46ca5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.864+0000 7fefdffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefe4ce5700 1 -- 172.21.15.112:0/104801557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 msgr2=0x5623e47f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefe4ce5700 1 -- 172.21.15.112:0/104801557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 msgr2=0x5623e47ee820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47ee820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefe4ce5700 1 -- 172.21.15.112:0/104801557 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e46ed760 con 0x5623e47e22f0 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefe54e6700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47ee820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47f3030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5623e47e06c0 tx=0x7fefd400b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694691fc0b1bdb13 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefbffff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefdffff700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefbffff700 1 -- 172.21.15.112:0/104801557 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4044050 con 0x5623e47e22f0 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.865+0000 7fefbffff700 1 -- 172.21.15.112:0/104801557 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fefd4040670 con 0x5623e47e22f0 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd4011bb0 con 0x5623e47e22f0 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7fefbffff700 1 -- 172.21.15.112:0/104801557 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefd4040870 con 0x5623e47e22f0 2024-05-10T10:41:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7feff17dcb80 10 client.25228 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5623e47fc1c0 con 0x5623e47e22f0 2024-05-10T10:41:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7fefbffff700 1 -- 172.21.15.112:0/104801557 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefd40054b0 con 0x5623e47e22f0 2024-05-10T10:41:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7fefbffff700 1 -- 172.21.15.112:0/104801557 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fefd40049d0 con 0x5623e47e22f0 2024-05-10T10:41:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7feff17dcb80 10 client.25228 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7feff17dcb80 10 client.25228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.866+0000 7feff17dcb80 20 client.25228 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.867+0000 7feff17dcb80 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e47fe6f0 0x5623e4800b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.867+0000 7feff17dcb80 4 client.25228 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.867+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fefd4011bb0 con 0x5623e47fe6f0 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.867+0000 7fefdffff700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e47fe6f0 0x5623e4800b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.867+0000 7fefbffff700 10 client.25228 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.867+0000 7fefdffff700 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e47fe6f0 0x5623e4800b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b871eee19f4473 server_cookie=af840937cce5136a in_seq=0 out_seq=0 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7fefbffff700 1 -- 172.21.15.112:0/104801557 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fefd4011bb0 con 0x5623e47fe6f0 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7fefbffff700 10 client.25228 handle_command_reply: tid=0 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 client.25228 shutdown 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 msgr2=0x5623e47f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47f3030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5623e47e06c0 tx=0x7fefd400b8e0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 shutdown_connections 2024-05-10T10:41:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e47fe6f0 0x5623e4800b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47ee820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47f3030 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 --2- 172.21.15.112:0/104801557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e47f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.869+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 >> 172.21.15.112:0/104801557 conn(0x5623e4710b70 msgr2=0x5623e47d9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.870+0000 7fefbffff700 0 client.25228 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.870+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 shutdown_connections 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.870+0000 7feff17dcb80 1 -- 172.21.15.112:0/104801557 wait complete. 2024-05-10T10:41:42.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.870+0000 7feff17dcb80 20 client.25228 trim_cache size 0 max 16384 2024-05-10T10:41:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.899+0000 7feff17dcb80 1 Processor -- start 2024-05-10T10:41:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.899+0000 7feff17dcb80 1 -- start start 2024-05-10T10:41:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.899+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e4801920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47e8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38456/0 (socket says 172.21.15.112:38456) 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe4ce5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefdffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e4801920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe4ce5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47d9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:34596/0 (socket says 172.21.15.112:34596) 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 -- 172.21.15.112:0/3050837574 learned_addr learned my addr 172.21.15.112:0/3050837574 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefdffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e4801920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:39750/0 (socket says 172.21.15.112:39750) 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01acea0 con 0x5623e47ee3a0 2024-05-10T10:41:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01ae9e0 con 0x5623e47e74a0 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01ad570 con 0x5623e47e22f0 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 -- 172.21.15.112:0/3050837574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 msgr2=0x5623e47d9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 --2- 172.21.15.112:0/3050837574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47d9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 -- 172.21.15.112:0/3050837574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 msgr2=0x5623e4801920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 --2- 172.21.15.112:0/3050837574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e4801920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.900+0000 7fefe54e6700 1 -- 172.21.15.112:0/3050837574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e47ed140 con 0x5623e47ee3a0 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.901+0000 7fefe54e6700 1 --2- 172.21.15.112:0/3050837574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47e8f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fefd80a7fc0 tx=0x7fefd801b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0701579bb6719f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.901+0000 7fefca7fc700 1 -- 172.21.15.112:0/3050837574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80aad50 con 0x5623e47ee3a0 2024-05-10T10:41:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.901+0000 7fefca7fc700 1 -- 172.21.15.112:0/3050837574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefd80af100 con 0x5623e47ee3a0 2024-05-10T10:41:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.901+0000 7fefca7fc700 1 -- 172.21.15.112:0/3050837574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80aafb0 con 0x5623e47ee3a0 2024-05-10T10:41:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.902+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 msgr2=0x5623e47e8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.902+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3050837574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47e8f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fefd80a7fc0 tx=0x7fefd801b430 comp rx=0 tx=0).stop 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.902+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 shutdown_connections 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.902+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3050837574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47ee3a0 0x5623e47e8f30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.902+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3050837574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e47d9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.902+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3050837574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47e74a0 0x5623e4801920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.902+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 >> 172.21.15.112:0/3050837574 conn(0x5623e4710b70 msgr2=0x5623e4801210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.903+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 shutdown_connections 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.903+0000 7feff17dcb80 1 -- 172.21.15.112:0/3050837574 wait complete. 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.903+0000 7feff17dcb80 1 -- start start 2024-05-10T10:41:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.903+0000 7feff17dcb80 1 Processor -- start 2024-05-10T10:41:42.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e4807220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e74a0 0x5623e4807760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7feff17dcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47ee3a0 0x5623e48041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7fefdffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e74a0 0x5623e4807760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7feff17dcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01ae310 con 0x5623e47e74a0 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7fefdffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e74a0 0x5623e4807760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38458/0 (socket says 172.21.15.112:38458) 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7fefdffff700 1 -- 172.21.15.112:0/3865830398 learned_addr learned my addr 172.21.15.112:0/3865830398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7fefe54e6700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47ee3a0 0x5623e48041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7feff17dcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01adc40 con 0x5623e47ee3a0 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01aae40 con 0x5623e47e22f0 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.904+0000 7feff17dcb80 10 client.0 _ll_register_callbacks cb 0x5623e46ca5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:42.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e4807220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefe54e6700 1 -- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 msgr2=0x5623e4807220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefe54e6700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e4807220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefe54e6700 1 -- 172.21.15.112:0/3865830398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e74a0 msgr2=0x5623e4807760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefe54e6700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e74a0 0x5623e4807760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefe54e6700 1 -- 172.21.15.112:0/3865830398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e4804970 con 0x5623e47ee3a0 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefdffff700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e74a0 0x5623e4807760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefe54e6700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47ee3a0 0x5623e48041f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefd80a7fc0 tx=0x7fefd80ab320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6658889ce38143f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefbffff700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefbffff700 1 -- 172.21.15.112:0/3865830398 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd808a060 con 0x5623e47ee3a0 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefbffff700 1 -- 172.21.15.112:0/3865830398 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fefd80c5350 con 0x5623e47ee3a0 2024-05-10T10:41:42.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd80af100 con 0x5623e47ee3a0 2024-05-10T10:41:42.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7fefbffff700 1 -- 172.21.15.112:0/3865830398 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefd8091050 con 0x5623e47ee3a0 2024-05-10T10:41:42.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7feff17dcb80 10 client.25205 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.905+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5623e4868860 con 0x5623e47ee3a0 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7fefbffff700 1 -- 172.21.15.112:0/3865830398 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefd80aad50 con 0x5623e47ee3a0 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7fefbffff700 1 -- 172.21.15.112:0/3865830398 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fefd8011070 con 0x5623e47ee3a0 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7feff17dcb80 10 client.25205 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7feff17dcb80 10 client.25205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7feff17dcb80 20 client.25205 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e4868e00 0x5623e486b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7feff17dcb80 4 client.25205 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.906+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fefd80af100 con 0x5623e4868e00 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e4868e00 0x5623e486b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7fefbffff700 10 client.25205 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7fefe4ce5700 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e4868e00 0x5623e486b1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f078dafbdb0209a5 server_cookie=d5a4e586a3219803 in_seq=0 out_seq=0 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7fefbffff700 1 -- 172.21.15.112:0/3865830398 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fefd80af100 con 0x5623e4868e00 2024-05-10T10:41:42.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7fefbffff700 10 client.25205 handle_command_reply: tid=0 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7feff17dcb80 1 client.25205 shutdown 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47ee3a0 msgr2=0x5623e48041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.907+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47ee3a0 0x5623e48041f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefd80a7fc0 tx=0x7fefd80ab320 comp rx=0 tx=0).stop 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 shutdown_connections 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5623e4868e00 0x5623e486b1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623e47ee3a0 0x5623e48041f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e47e22f0 0x5623e4807220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.912 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 --2- 172.21.15.112:0/3865830398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623e47e74a0 0x5623e4807760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7fefbffff700 0 client.25205 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 >> 172.21.15.112:0/3865830398 conn(0x5623e4710b70 msgr2=0x5623e47f1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 shutdown_connections 2024-05-10T10:41:42.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 1 -- 172.21.15.112:0/3865830398 wait complete. 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.908+0000 7feff17dcb80 20 client.25205 trim_cache size 0 max 16384 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fefd00b2980 msgr2=0x7fefd00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fefd00b2980 0x7fefd00b4e40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fefd400bcf0 tx=0x7fefd400b2c0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe0107760 msgr2=0x7fefe01986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe0107760 0x7fefe01986e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009850 tx=0x7fefd8007d90 comp rx=0 tx=0).stop 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 shutdown_connections 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe0107760 0x7fefe01986e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe0102e00 0x7fefe019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fefd00b2980 0x7fefd00b4e40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 --2- 172.21.15.112:0/4256629076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefe01037b0 0x7fefe019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 >> 172.21.15.112:0/4256629076 conn(0x7fefe0078a10 msgr2=0x7fefe00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 shutdown_connections 2024-05-10T10:41:42.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:42.909+0000 7fefe6f49700 1 -- 172.21.15.112:0/4256629076 wait complete. 2024-05-10T10:41:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:42 smithi195 ceph-mon[33773]: pgmap v474: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:42 smithi148 ceph-mon[33568]: pgmap v474: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:44.641 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:41:44.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:41:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:45.041 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:44 smithi112 ceph-mon[30100]: pgmap v475: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:45.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:44 smithi195 ceph-mon[33773]: pgmap v475: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:44 smithi148 ceph-mon[33568]: pgmap v475: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 -- 172.21.15.112:0/834806253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06ba40 msgr2=0x7f3c9c06be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/834806253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06ba40 0x7f3c9c06be20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009990 tx=0x7f3c8c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 -- 172.21.15.112:0/834806253 shutdown_connections 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/834806253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c9c10ff40 0x7f3c9c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/834806253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06ba40 0x7f3c9c06be20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/834806253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c9c06c360 0x7f3c9c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 -- 172.21.15.112:0/834806253 >> 172.21.15.112:0/834806253 conn(0x7f3c9c079310 msgr2=0x7f3c9c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:46.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 -- 172.21.15.112:0/834806253 shutdown_connections 2024-05-10T10:41:46.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.824+0000 7f3ca12ca700 1 -- 172.21.15.112:0/834806253 wait complete. 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.825+0000 7f3ca12ca700 1 Processor -- start 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.825+0000 7f3ca12ca700 1 -- start start 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3ca12ca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c9c06ba40 0x7f3c9c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3ca12ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06c360 0x7f3c9c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3ca12ca700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c9c10ff40 0x7f3c9c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3ca12ca700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c115720 con 0x7f3c9c06c360 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3ca12ca700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1155a0 con 0x7f3c9c06ba40 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3ca12ca700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1158a0 con 0x7f3c9c10ff40 2024-05-10T10:41:46.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3c9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06c360 0x7f3c9c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3c9affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c9c06ba40 0x7f3c9c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3c9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06c360 0x7f3c9c19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60870/0 (socket says 172.21.15.112:60870) 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3c9a7fc700 1 -- 172.21.15.112:0/504361219 learned_addr learned my addr 172.21.15.112:0/504361219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.826+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c9c10ff40 0x7f3c9c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c9affd700 1 -- 172.21.15.112:0/504361219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c9c10ff40 msgr2=0x7f3c9c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c9affd700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c9c10ff40 0x7f3c9c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c9affd700 1 -- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06c360 msgr2=0x7f3c9c19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c9affd700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06c360 0x7f3c9c19c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c9affd700 1 -- 172.21.15.112:0/504361219 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c1a0d60 con 0x7f3c9c06ba40 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c9a7fc700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06c360 0x7f3c9c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c9affd700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c9c06ba40 0x7f3c9c19bd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0059d0 tx=0x7f3c8c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c83fff700 1 -- 172.21.15.112:0/504361219 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c027070 con 0x7f3c9c06ba40 2024-05-10T10:41:46.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c1a0ff0 con 0x7f3c9c06ba40 2024-05-10T10:41:46.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c83fff700 1 -- 172.21.15.112:0/504361219 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8c00fb00 con 0x7f3c9c06ba40 2024-05-10T10:41:46.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3c83fff700 1 -- 172.21.15.112:0/504361219 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0229d0 con 0x7f3c9c06ba40 2024-05-10T10:41:46.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.827+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1a88d0 con 0x7f3c9c06ba40 2024-05-10T10:41:46.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3c83fff700 1 -- 172.21.15.112:0/504361219 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3c8c02a070 con 0x7f3c9c06ba40 2024-05-10T10:41:46.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 Processor -- start 2024-05-10T10:41:46.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 -- start start 2024-05-10T10:41:46.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220facd60 0x558220fad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220fb14c0 0x558220fb1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558220ef68e0 con 0x558220fad680 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558220e95840 con 0x558220fb14c0 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.831+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558220ef28a0 con 0x558220facd60 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220facd60 0x558220fad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220facd60 0x558220fad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:48630/0 (socket says 172.21.15.112:48630) 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/787106016 learned_addr learned my addr 172.21.15.112:0/787106016 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9a7fc700 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220fb14c0 0x558220fb1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9affd700 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9affd700 1 -- 172.21.15.112:0/787106016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220facd60 msgr2=0x558220fad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9affd700 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220facd60 0x558220fad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9affd700 1 -- 172.21.15.112:0/787106016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220fb14c0 msgr2=0x558220fb1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9affd700 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220fb14c0 0x558220fb1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9affd700 1 -- 172.21.15.112:0/787106016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558220e43950 con 0x558220fad680 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c9affd700 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb0f80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0e0970 tx=0x7f3c8c0eb760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8927f6f698efdefc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.832+0000 7f3c83fff700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c840b2980 0x7f3c840b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.833+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c840b2980 0x7f3c840b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.833+0000 7f3c83fff700 1 -- 172.21.15.112:0/504361219 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3c8c0e32c0 con 0x7f3c9c06ba40 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.833+0000 7f3c817fa700 1 -- 172.21.15.112:0/787106016 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0a6440 con 0x558220fad680 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.833+0000 7f3c817fa700 1 -- 172.21.15.112:0/787106016 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8c0f0030 con 0x558220fad680 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.833+0000 7f3c817fa700 1 -- 172.21.15.112:0/787106016 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0a9cf0 con 0x558220fad680 2024-05-10T10:41:46.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 -- 172.21.15.112:0/787106016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 msgr2=0x558220fb0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb0f80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0e0970 tx=0x7f3c8c0eb760 comp rx=0 tx=0).stop 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c840b2980 0x7f3c840b4e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009790 tx=0x7f3c94009130 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 -- 172.21.15.112:0/787106016 shutdown_connections 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220fb14c0 0x558220fb1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220facd60 0x558220fad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/787106016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb0f80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 -- 172.21.15.112:0/787106016 >> 172.21.15.112:0/787106016 conn(0x558220ed0180 msgr2=0x558220f9ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 -- 172.21.15.112:0/787106016 shutdown_connections 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 -- 172.21.15.112:0/787106016 wait complete. 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 -- start start 2024-05-10T10:41:46.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 Processor -- start 2024-05-10T10:41:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.834+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fb88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fb14c0 0x558220fb61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558220fb9490 con 0x558220fad680 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558220e5ccb0 con 0x558220facd60 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558220fb9610 con 0x558220fb14c0 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3cabb5db80 10 client.0 _ll_register_callbacks cb 0x558220e8e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60890/0 (socket says 172.21.15.112:60890) 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9a7fc700 1 -- 172.21.15.112:0/521526003 learned_addr learned my addr 172.21.15.112:0/521526003 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fb14c0 0x558220fb61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9affd700 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fb88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/521526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 msgr2=0x558220fb88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fb88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/521526003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 msgr2=0x558220fb5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/521526003 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558220e92710 con 0x558220fb14c0 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fb14c0 0x558220fb61d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c94003e30 tx=0x7f3c94004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95fc59e57d2aac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3c7eff5700 1 -- 172.21.15.112:0/521526003 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94027020 con 0x558220fb14c0 2024-05-10T10:41:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3c7eff5700 1 -- 172.21.15.112:0/521526003 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3c94023440 con 0x558220fb14c0 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.835+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8c0f0030 con 0x558220fb14c0 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3c7eff5700 1 -- 172.21.15.112:0/521526003 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c94032050 con 0x558220fb14c0 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 10 client.25243 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558220fa2720 con 0x558220fb14c0 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3c7eff5700 1 -- 172.21.15.112:0/521526003 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8c0f0030 con 0x558220fb14c0 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3c7eff5700 1 -- 172.21.15.112:0/521526003 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3c940166c0 con 0x558220fb14c0 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 10 client.25243 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 10 client.25243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 20 client.25243 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fa4db0 0x558220fa71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 4 client.25243 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.836+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558220f706d0 con 0x558220fa4db0 2024-05-10T10:41:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.837+0000 7f3c9affd700 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fa4db0 0x558220fa71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.837+0000 7f3c9affd700 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fa4db0 0x558220fa71f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51291b7cfef442fb server_cookie=eb270ba1f441062c in_seq=0 out_seq=0 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.837+0000 7f3c7eff5700 10 client.25243 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3c7eff5700 1 -- 172.21.15.112:0/521526003 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558220f706d0 con 0x558220fa4db0 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3c7eff5700 10 client.25243 handle_command_reply: tid=0 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 client.25243 shutdown 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fb14c0 msgr2=0x558220fb61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fb14c0 0x558220fb61d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c94003e30 tx=0x7f3c94004ad0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 shutdown_connections 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fa4db0 0x558220fa71f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3c7eff5700 0 client.25243 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fb14c0 0x558220fb61d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fb88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/521526003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fad680 0x558220fb5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 >> 172.21.15.112:0/521526003 conn(0x558220ed0180 msgr2=0x558220f9eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 shutdown_connections 2024-05-10T10:41:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 1 -- 172.21.15.112:0/521526003 wait complete. 2024-05-10T10:41:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.838+0000 7f3cabb5db80 20 client.25243 trim_cache size 0 max 16384 2024-05-10T10:41:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 Processor -- start 2024-05-10T10:41:46.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 -- start start 2024-05-10T10:41:46.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fb4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fc7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558220faedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ace70 con 0x558220fa4db0 2024-05-10T10:41:46.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ae9b0 con 0x558220facd60 2024-05-10T10:41:46.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ad540 con 0x558220fad680 2024-05-10T10:41:46.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.859+0000 7f3c9affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fb4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558220faedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fb4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60906/0 (socket says 172.21.15.112:60906) 2024-05-10T10:41:46.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558220faedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:48646/0 (socket says 172.21.15.112:48646) 2024-05-10T10:41:46.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9affd700 1 -- 172.21.15.112:0/1614641970 learned_addr learned my addr 172.21.15.112:0/1614641970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:46.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9a7fc700 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fc7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/1614641970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 msgr2=0x558220fc7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fc7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/1614641970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 msgr2=0x558220fb4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fb4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/1614641970 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558220f9b480 con 0x558220fad680 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9affd700 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fb4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558220faedb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c94016ae0 tx=0x7f3c94000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=548d39fd3580d91b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:46.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c80ff9700 1 -- 172.21.15.112:0/1614641970 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94027040 con 0x558220fad680 2024-05-10T10:41:46.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c80ff9700 1 -- 172.21.15.112:0/1614641970 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c9401c030 con 0x558220fad680 2024-05-10T10:41:46.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.860+0000 7f3c80ff9700 1 -- 172.21.15.112:0/1614641970 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94005040 con 0x558220fad680 2024-05-10T10:41:46.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 -- 172.21.15.112:0/1614641970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 msgr2=0x558220faedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558220faedb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c94016ae0 tx=0x7f3c94000f40 comp rx=0 tx=0).stop 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 -- 172.21.15.112:0/1614641970 shutdown_connections 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558220faedb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fb4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/1614641970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fc7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 -- 172.21.15.112:0/1614641970 >> 172.21.15.112:0/1614641970 conn(0x558220ed0180 msgr2=0x558220fc7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 -- 172.21.15.112:0/1614641970 shutdown_connections 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.861+0000 7f3cabb5db80 1 -- 172.21.15.112:0/1614641970 wait complete. 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 -- start start 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 Processor -- start 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fbc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fbcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558221038470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ae2e0 con 0x558220fa4db0 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1adc10 con 0x558220facd60 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1aaed0 con 0x558220fad680 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3cabb5db80 10 client.0 _ll_register_callbacks cb 0x558220e8e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3c9b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558221038470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.862+0000 7f3c9affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fbc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558221038470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:48656/0 (socket says 172.21.15.112:48656) 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fbc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60908/0 (socket says 172.21.15.112:60908) 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/367273994 learned_addr learned my addr 172.21.15.112:0/367273994 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9a7fc700 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fbcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/367273994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 msgr2=0x558220fbcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fbcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/367273994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 msgr2=0x558220fbc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fbc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 -- 172.21.15.112:0/367273994 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558220fbd510 con 0x558220fad680 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9affd700 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fbc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c9b7fe700 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558221038470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c94003e00 tx=0x7f3c94007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32394e6dd7b3ac9d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.863+0000 7f3c7eff5700 1 -- 172.21.15.112:0/367273994 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94042070 con 0x558220fad680 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3c7eff5700 1 -- 172.21.15.112:0/367273994 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3c94004830 con 0x558220fad680 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9401c030 con 0x558220fad680 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3c7eff5700 1 -- 172.21.15.112:0/367273994 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c9404e050 con 0x558220fad680 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 10 client.25249 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558221039ac0 con 0x558220fad680 2024-05-10T10:41:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3c7eff5700 1 -- 172.21.15.112:0/367273994 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c94005040 con 0x558220fad680 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3c7eff5700 1 -- 172.21.15.112:0/367273994 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3c94023440 con 0x558220fad680 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 10 client.25249 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 10 client.25249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 20 client.25249 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fb14c0 0x55822103c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 4 client.25249 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.864+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3c9401c030 con 0x558220fb14c0 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.865+0000 7f3c9affd700 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fb14c0 0x55822103c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.865+0000 7f3c7eff5700 10 client.25249 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.865+0000 7f3c9affd700 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fb14c0 0x55822103c0e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47d53b1ee11a17e server_cookie=6658a314f114944c in_seq=0 out_seq=0 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3c7eff5700 1 -- 172.21.15.112:0/367273994 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3c9401c030 con 0x558220fb14c0 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3c7eff5700 10 client.25249 handle_command_reply: tid=0 2024-05-10T10:41:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 client.25249 shutdown 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 msgr2=0x558221038470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558221038470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c94003e00 tx=0x7f3c94007e10 comp rx=0 tx=0).stop 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 shutdown_connections 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x558220fb14c0 0x55822103c0e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558220fad680 0x558221038470 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558220fa4db0 0x558220fbc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 --2- 172.21.15.112:0/367273994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558220facd60 0x558220fbcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.866+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 >> 172.21.15.112:0/367273994 conn(0x558220ed0180 msgr2=0x558220fc7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.867+0000 7f3c7eff5700 0 client.25249 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.867+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 shutdown_connections 2024-05-10T10:41:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.867+0000 7f3cabb5db80 1 -- 172.21.15.112:0/367273994 wait complete. 2024-05-10T10:41:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.867+0000 7f3cabb5db80 20 client.25249 trim_cache size 0 max 16384 2024-05-10T10:41:46.877 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:41:46.878 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:41:46.878 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "c1a0e23b-c357-4124-b63a-9831a0bb4a72", 2024-05-10T10:41:46.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:41:46.878 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c840b2980 msgr2=0x7f3c840b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c840b2980 0x7f3c840b4e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009790 tx=0x7f3c94009130 comp rx=0 tx=0).stop 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c9c06ba40 msgr2=0x7f3c9c19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c9c06ba40 0x7f3c9c19bd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0059d0 tx=0x7f3c8c0047b0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 shutdown_connections 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c840b2980 0x7f3c840b4e40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c9c10ff40 0x7f3c9c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c9c06ba40 0x7f3c9c19bd50 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0059d0 tx=0x7f3c8c0047b0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 --2- 172.21.15.112:0/504361219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c9c06c360 0x7f3c9c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.868+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 >> 172.21.15.112:0/504361219 conn(0x7f3c9c079310 msgr2=0x7f3c9c1116d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.869+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 shutdown_connections 2024-05-10T10:41:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:46.869+0000 7f3ca12ca700 1 -- 172.21.15.112:0/504361219 wait complete. 2024-05-10T10:41:47.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:46 smithi195 ceph-mon[33773]: pgmap v476: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:47.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:46 smithi112 ceph-mon[30100]: pgmap v476: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:46 smithi148 ceph-mon[33568]: pgmap v476: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:47.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:41:47.916 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:48.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:47 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:41:48.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:47 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:48.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:47 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:41:48.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:47 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:48.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:47 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:41:48.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:47 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:48.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:47 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:41:48.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:47 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:48.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:47 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:41:48.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:47 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:48.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:47 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:41:48.196 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:47 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:41:48.812 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:48 smithi112 ceph-mon[30100]: pgmap v477: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:48 smithi195 ceph-mon[33773]: pgmap v477: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:48 smithi148 ceph-mon[33568]: pgmap v477: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:41:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 -- 172.21.15.112:0/51361923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0101a90 msgr2=0x7f22f0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 --2- 172.21.15.112:0/51361923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0101a90 0x7f22f0101f10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009990 tx=0x7f22e4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 -- 172.21.15.112:0/51361923 shutdown_connections 2024-05-10T10:41:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 --2- 172.21.15.112:0/51361923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 0x7f22f0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 --2- 172.21.15.112:0/51361923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0109520 0x7f22f0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 --2- 172.21.15.112:0/51361923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0101a90 0x7f22f0101f10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 -- 172.21.15.112:0/51361923 >> 172.21.15.112:0/51361923 conn(0x7f22f0078a10 msgr2=0x7f22f00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 -- 172.21.15.112:0/51361923 shutdown_connections 2024-05-10T10:41:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.840+0000 7f22f5ead700 1 -- 172.21.15.112:0/51361923 wait complete. 2024-05-10T10:41:49.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.841+0000 7f22f5ead700 1 Processor -- start 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.841+0000 7f22f5ead700 1 -- start start 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22f5ead700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0101a90 0x7f22f019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22f5ead700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 0x7f22f019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22f5ead700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0109520 0x7f22f0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22f5ead700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0113500 con 0x7f22f0109520 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22f5ead700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0113380 con 0x7f22f0101a90 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22f5ead700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0113680 con 0x7f22f0102450 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 0x7f22f019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22ef7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0101a90 0x7f22f019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 0x7f22f019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:48680/0 (socket says 172.21.15.112:48680) 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22ef7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0101a90 0x7f22f019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:53816/0 (socket says 172.21.15.112:53816) 2024-05-10T10:41:49.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.842+0000 7f22eeffd700 1 -- 172.21.15.112:0/4006690907 learned_addr learned my addr 172.21.15.112:0/4006690907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22effff700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0109520 0x7f22f0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22eeffd700 1 -- 172.21.15.112:0/4006690907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0101a90 msgr2=0x7f22f019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22eeffd700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0101a90 0x7f22f019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22eeffd700 1 -- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0109520 msgr2=0x7f22f0199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22eeffd700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0109520 0x7f22f0199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22eeffd700 1 -- 172.21.15.112:0/4006690907 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f0199fc0 con 0x7f22f0102450 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22eeffd700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 0x7f22f019fbe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22e4006010 tx=0x7f22e400bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.843+0000 7f22ecff9700 1 -- 172.21.15.112:0/4006690907 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4027070 con 0x7f22f0102450 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.844+0000 7f22ecff9700 1 -- 172.21.15.112:0/4006690907 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e4022470 con 0x7f22f0102450 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.844+0000 7f22effff700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0109520 0x7f22f0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.844+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f019a250 con 0x7f22f0102450 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.844+0000 7f22ecff9700 1 -- 172.21.15.112:0/4006690907 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4017650 con 0x7f22f0102450 2024-05-10T10:41:49.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.844+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f01a6570 con 0x7f22f0102450 2024-05-10T10:41:49.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.847+0000 7f22ecff9700 1 -- 172.21.15.112:0/4006690907 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f22e402a070 con 0x7f22f0102450 2024-05-10T10:41:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.849+0000 7f22ecff9700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f22d80b2980 0x7f22d80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.849+0000 7f22ef7fe700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f22d80b2980 0x7f22d80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.849+0000 7f22ecff9700 1 -- 172.21.15.112:0/4006690907 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f22e40e3310 con 0x7f22f0102450 2024-05-10T10:41:49.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.849+0000 7f2300740b80 1 Processor -- start 2024-05-10T10:41:49.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f2300740b80 1 -- start start 2024-05-10T10:41:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f2300740b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 0x564f16c61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f2300740b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c61ad0 0x564f16c5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f2300740b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c5bd50 0x564f16c5c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f2300740b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f16bad940 con 0x564f16c61ad0 2024-05-10T10:41:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f2300740b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f16b12da0 con 0x564f16c5bd50 2024-05-10T10:41:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c61ad0 0x564f16c5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f22effff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c5bd50 0x564f16c5c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f22effff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c5bd50 0x564f16c5c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:53826/0 (socket says 172.21.15.112:53826) 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f22effff700 1 -- 172.21.15.112:0/2443980609 learned_addr learned my addr 172.21.15.112:0/2443980609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.850+0000 7f2300740b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f16ba9900 con 0x564f16c611b0 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22ef7fe700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f22d80b2980 0x7f22d80b4e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22f019a9a0 tx=0x7f22e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22effff700 1 -- 172.21.15.112:0/2443980609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 msgr2=0x564f16c61590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22effff700 1 --2- 172.21.15.112:0/2443980609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 0x564f16c61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22effff700 1 -- 172.21.15.112:0/2443980609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c61ad0 msgr2=0x564f16c5b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22effff700 1 --2- 172.21.15.112:0/2443980609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c61ad0 0x564f16c5b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22effff700 1 -- 172.21.15.112:0/2443980609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f16afa8d0 con 0x564f16c5bd50 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22effff700 1 --2- 172.21.15.112:0/2443980609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c5bd50 0x564f16c5c190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22e80078d0 tx=0x7f22e8007be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725162c7b11461ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22d5ffb700 1 -- 172.21.15.112:0/2443980609 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8019030 con 0x564f16c5bd50 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22d5ffb700 1 -- 172.21.15.112:0/2443980609 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e8004bb0 con 0x564f16c5bd50 2024-05-10T10:41:49.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.851+0000 7f22d5ffb700 1 -- 172.21.15.112:0/2443980609 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e80052c0 con 0x564f16c5bd50 2024-05-10T10:41:49.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.852+0000 7f2300740b80 1 -- 172.21.15.112:0/2443980609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c5bd50 msgr2=0x564f16c5c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.852+0000 7f2300740b80 1 --2- 172.21.15.112:0/2443980609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c5bd50 0x564f16c5c190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22e80078d0 tx=0x7f22e8007be0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 -- 172.21.15.112:0/2443980609 shutdown_connections 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 --2- 172.21.15.112:0/2443980609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c5bd50 0x564f16c5c190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 --2- 172.21.15.112:0/2443980609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 0x564f16c61590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 --2- 172.21.15.112:0/2443980609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c61ad0 0x564f16c5b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 -- 172.21.15.112:0/2443980609 >> 172.21.15.112:0/2443980609 conn(0x564f16b87b70 msgr2=0x564f16c562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 -- 172.21.15.112:0/2443980609 shutdown_connections 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 -- 172.21.15.112:0/2443980609 wait complete. 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.853+0000 7f2300740b80 1 -- start start 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 1 Processor -- start 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c67700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c611b0 0x564f16c64aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c61ad0 0x564f16c64fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f22ef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f16c68450 con 0x564f16c5bd50 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f16b66ed0 con 0x564f16c611b0 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f16c685d0 con 0x564f16c61ad0 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f2300740b80 10 client.0 _ll_register_callbacks cb 0x564f16b415b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f22effff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c61ad0 0x564f16c64fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c611b0 0x564f16c64aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f22effff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c61ad0 0x564f16c64fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:48694/0 (socket says 172.21.15.112:48694) 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f22effff700 1 -- 172.21.15.112:0/1323859802 learned_addr learned my addr 172.21.15.112:0/1323859802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:49.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.854+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c611b0 0x564f16c64aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:53834/0 (socket says 172.21.15.112:53834) 2024-05-10T10:41:49.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22effff700 1 -- 172.21.15.112:0/1323859802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c611b0 msgr2=0x564f16c64aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22effff700 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c611b0 0x564f16c64aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22effff700 1 -- 172.21.15.112:0/1323859802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 msgr2=0x564f16c67700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22effff700 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c67700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22effff700 1 -- 172.21.15.112:0/1323859802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f16b64760 con 0x564f16c61ad0 2024-05-10T10:41:49.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22eeffd700 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c611b0 0x564f16c64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:49.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22effff700 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c61ad0 0x564f16c64fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22e8015e30 tx=0x7f22e8021a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f791ea113afe11f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22ceffd700 1 -- 172.21.15.112:0/1323859802 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8003c00 con 0x564f16c61ad0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e8004bb0 con 0x564f16c61ad0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.855+0000 7f22ceffd700 1 -- 172.21.15.112:0/1323859802 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f22e8003da0 con 0x564f16c61ad0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.856+0000 7f22ceffd700 1 -- 172.21.15.112:0/1323859802 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22e8015c00 con 0x564f16c61ad0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.856+0000 7f2300740b80 10 client.25258 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.856+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f16c72cc0 con 0x564f16c61ad0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.856+0000 7f22ceffd700 1 -- 172.21.15.112:0/1323859802 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e8004bb0 con 0x564f16c61ad0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f22ceffd700 1 -- 172.21.15.112:0/1323859802 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22e8028070 con 0x564f16c61ad0 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f2300740b80 10 client.25258 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f2300740b80 10 client.25258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f2300740b80 20 client.25258 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f2300740b80 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16c751f0 0x564f16c775e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f2300740b80 4 client.25258 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:49.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f16bbb320 con 0x564f16c751f0 2024-05-10T10:41:49.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.857+0000 7f22ef7fe700 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16c751f0 0x564f16c775e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.858+0000 7f22ceffd700 10 client.25258 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:49.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.858+0000 7f22ef7fe700 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16c751f0 0x564f16c775e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcedc8ed4a906c23 server_cookie=1d7e02b001d10f9 in_seq=0 out_seq=0 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.858+0000 7f22ceffd700 1 -- 172.21.15.112:0/1323859802 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f16bbb320 con 0x564f16c751f0 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.858+0000 7f22ceffd700 10 client.25258 handle_command_reply: tid=0 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 client.25258 shutdown 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c61ad0 msgr2=0x564f16c64fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c61ad0 0x564f16c64fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22e8015e30 tx=0x7f22e8021a30 comp rx=0 tx=0).stop 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 shutdown_connections 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16c751f0 0x564f16c775e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c61ad0 0x564f16c64fe0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c67700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f22ceffd700 0 client.25258 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:49.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 --2- 172.21.15.112:0/1323859802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c611b0 0x564f16c64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 >> 172.21.15.112:0/1323859802 conn(0x564f16b87b70 msgr2=0x564f16c50b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:49.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 shutdown_connections 2024-05-10T10:41:49.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 1 -- 172.21.15.112:0/1323859802 wait complete. 2024-05-10T10:41:49.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.859+0000 7f2300740b80 20 client.25258 trim_cache size 0 max 16384 2024-05-10T10:41:49.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.896+0000 7f2300740b80 1 Processor -- start 2024-05-10T10:41:49.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f2300740b80 1 -- start start 2024-05-10T10:41:49.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f2300740b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c5bd50 0x564f16c5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f2300740b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c611b0 0x564f16c714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f2300740b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c50c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f2300740b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01ace20 con 0x564f16c611b0 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f22ef7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c5bd50 0x564f16c5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c611b0 0x564f16c714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f22ef7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c5bd50 0x564f16c5b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:48706/0 (socket says 172.21.15.112:48706) 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.897+0000 7f22ef7fe700 1 -- 172.21.15.112:0/877018817 learned_addr learned my addr 172.21.15.112:0/877018817 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f2300740b80 1 -- 172.21.15.112:0/877018817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01ae960 con 0x564f16c61ad0 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f2300740b80 1 -- 172.21.15.112:0/877018817 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01ad4f0 con 0x564f16c5bd50 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22effff700 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c50c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22eeffd700 1 -- 172.21.15.112:0/877018817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c5bd50 msgr2=0x564f16c5b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22eeffd700 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c5bd50 0x564f16c5b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22eeffd700 1 -- 172.21.15.112:0/877018817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 msgr2=0x564f16c50c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22eeffd700 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c50c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22eeffd700 1 -- 172.21.15.112:0/877018817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f16c51180 con 0x564f16c611b0 2024-05-10T10:41:49.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22effff700 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c50c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.898+0000 7f22eeffd700 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c611b0 0x564f16c714e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f22e40a7840 tx=0x7f22e4005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a109ee44a1ea101 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.899+0000 7f22d57fa700 1 -- 172.21.15.112:0/877018817 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e40aa5d0 con 0x564f16c611b0 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.899+0000 7f22d57fa700 1 -- 172.21.15.112:0/877018817 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e40ae980 con 0x564f16c611b0 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.899+0000 7f22d57fa700 1 -- 172.21.15.112:0/877018817 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e40b2f10 con 0x564f16c611b0 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 -- 172.21.15.112:0/877018817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c611b0 msgr2=0x564f16c714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c611b0 0x564f16c714e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f22e40a7840 tx=0x7f22e4005ad0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 -- 172.21.15.112:0/877018817 shutdown_connections 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c50c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c5bd50 0x564f16c5b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 --2- 172.21.15.112:0/877018817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c611b0 0x564f16c714e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 -- 172.21.15.112:0/877018817 >> 172.21.15.112:0/877018817 conn(0x564f16b87b70 msgr2=0x564f16c64350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:49.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 -- 172.21.15.112:0/877018817 shutdown_connections 2024-05-10T10:41:49.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.900+0000 7f2300740b80 1 -- 172.21.15.112:0/877018817 wait complete. 2024-05-10T10:41:49.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.901+0000 7f2300740b80 1 -- start start 2024-05-10T10:41:49.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.901+0000 7f2300740b80 1 Processor -- start 2024-05-10T10:41:49.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.901+0000 7f2300740b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c7c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.901+0000 7f22ef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c7c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.901+0000 7f22ef7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c7c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60970/0 (socket says 172.21.15.112:60970) 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.901+0000 7f22ef7fe700 1 -- 172.21.15.112:0/455089109 learned_addr learned my addr 172.21.15.112:0/455089109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.901+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 0x564f16c7ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c7cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01ae290 con 0x564f16c5bd50 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01adbc0 con 0x564f16c61ad0 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01aade0 con 0x564f16c611b0 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f2300740b80 10 client.0 _ll_register_callbacks cb 0x564f16b415b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22eeffd700 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 0x564f16c7ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22effff700 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c7cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22ef7fe700 1 -- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 msgr2=0x564f16c7ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22ef7fe700 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 0x564f16c7ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22ef7fe700 1 -- 172.21.15.112:0/455089109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 msgr2=0x564f16c7cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:49.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22ef7fe700 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c7cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22ef7fe700 1 -- 172.21.15.112:0/455089109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f16cde8b0 con 0x564f16c5bd50 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22ef7fe700 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c7c4c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f22e001f550 tx=0x7f22e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b321de0e203569 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.902+0000 7f22ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f22ceffd700 1 -- 172.21.15.112:0/455089109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0004830 con 0x564f16c5bd50 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f22ceffd700 1 -- 172.21.15.112:0/455089109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f22e00049d0 con 0x564f16c5bd50 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e40ae980 con 0x564f16c5bd50 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f22ceffd700 1 -- 172.21.15.112:0/455089109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22e0038050 con 0x564f16c5bd50 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f2300740b80 10 client.15792 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f16cdfbd0 con 0x564f16c5bd50 2024-05-10T10:41:49.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f22ceffd700 1 -- 172.21.15.112:0/455089109 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e002c030 con 0x564f16c5bd50 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f22ceffd700 1 -- 172.21.15.112:0/455089109 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22e0028070 con 0x564f16c5bd50 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f2300740b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f2300740b80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.903+0000 7f2300740b80 20 client.15792 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16ce0310 0x564f16ce2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f2300740b80 4 client.15792 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f22e40ae980 con 0x564f16ce0310 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f22eeffd700 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16ce0310 0x564f16ce2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f22eeffd700 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16ce0310 0x564f16ce2730 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7106a00f1277e9 server_cookie=2b174c1e60b04bd1 in_seq=0 out_seq=0 2024-05-10T10:41:49.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f22ceffd700 10 client.15792 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f22ceffd700 1 -- 172.21.15.112:0/455089109 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f22e40ae980 con 0x564f16ce0310 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f22ceffd700 10 client.15792 handle_command_reply: tid=0 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.904+0000 7f2300740b80 1 client.15792 shutdown 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 msgr2=0x564f16c7c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c7c4c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f22e001f550 tx=0x7f22e00047c0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 shutdown_connections 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564f16ce0310 0x564f16ce2730 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f16c61ad0 0x564f16c7cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f16c5bd50 0x564f16c7c4c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 --2- 172.21.15.112:0/455089109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f16c611b0 0x564f16c7ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f22ceffd700 0 client.15792 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 >> 172.21.15.112:0/455089109 conn(0x564f16b87b70 msgr2=0x564f16c625c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 shutdown_connections 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 1 -- 172.21.15.112:0/455089109 wait complete. 2024-05-10T10:41:49.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.905+0000 7f2300740b80 20 client.15792 trim_cache size 0 max 16384 2024-05-10T10:41:49.910 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:41:49.910 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:41:49.910 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:41:49.910 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f22d80b2980 msgr2=0x7f22d80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f22d80b2980 0x7f22d80b4e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22f019a9a0 tx=0x7f22e0009260 comp rx=0 tx=0).stop 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 msgr2=0x7f22f019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 0x7f22f019fbe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22e4006010 tx=0x7f22e400bfc0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 shutdown_connections 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f0109520 0x7f22f0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0101a90 0x7f22f019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f22d80b2980 0x7f22d80b4e40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 --2- 172.21.15.112:0/4006690907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f0102450 0x7f22f019fbe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.906+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 >> 172.21.15.112:0/4006690907 conn(0x7f22f0078a10 msgr2=0x7f22f0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:49.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.907+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 shutdown_connections 2024-05-10T10:41:49.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:49.907+0000 7f22f5ead700 1 -- 172.21.15.112:0/4006690907 wait complete. 2024-05-10T10:41:50.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:49 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:41:50.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:49 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:41:50.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:49 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:41:50.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:41:50.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:41:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 -- 172.21.15.112:0/2152448069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 msgr2=0x7f4d481014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2152448069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d481014a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800aa10 tx=0x7f4d3802ea20 comp rx=0 tx=0).stop 2024-05-10T10:41:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 -- 172.21.15.112:0/2152448069 shutdown_connections 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2152448069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d48102c30 0x7f4d481094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2152448069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d48102270 0x7f4d481026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2152448069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d481014a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 -- 172.21.15.112:0/2152448069 >> 172.21.15.112:0/2152448069 conn(0x7f4d480fc610 msgr2=0x7f4d480fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 -- 172.21.15.112:0/2152448069 shutdown_connections 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.766+0000 7f4d4df46640 1 -- 172.21.15.112:0/2152448069 wait complete. 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.767+0000 7f4d4df46640 1 Processor -- start 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.767+0000 7f4d4df46640 1 -- start start 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.767+0000 7f4d4df46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d48196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d4df46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d48102270 0x7f4d48196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d48196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d48196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60994/0 (socket says 172.21.15.112:60994) 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 -- 172.21.15.112:0/2375863390 learned_addr learned my addr 172.21.15.112:0/2375863390 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d48102c30 0x7f4d4819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4810c240 con 0x7f4d48101080 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4810c0c0 con 0x7f4d48102270 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4810c3c0 con 0x7f4d48102c30 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d47fff640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d48102c30 0x7f4d4819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 -- 172.21.15.112:0/2375863390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d48102c30 msgr2=0x7f4d4819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d48102c30 0x7f4d4819dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 -- 172.21.15.112:0/2375863390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d48102270 msgr2=0x7f4d48196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d48102270 0x7f4d48196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d48102270 0x7f4d48196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d477fe640 1 -- 172.21.15.112:0/2375863390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4819e360 con 0x7f4d48101080 2024-05-10T10:41:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d47fff640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d48102c30 0x7f4d4819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.768+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d48102270 0x7f4d48196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.769+0000 7f4d477fe640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d48196740 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a960 tx=0x7f4d38034f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.769+0000 7f4d44ff9640 1 -- 172.21.15.112:0/2375863390 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3804a070 con 0x7f4d48101080 2024-05-10T10:41:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.769+0000 7f4d44ff9640 1 -- 172.21.15.112:0/2375863390 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d38003c90 con 0x7f4d48101080 2024-05-10T10:41:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.769+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4819e5f0 con 0x7f4d48101080 2024-05-10T10:41:50.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.770+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4819eae0 con 0x7f4d48101080 2024-05-10T10:41:50.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.771+0000 7f4d44ff9640 1 -- 172.21.15.112:0/2375863390 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38037050 con 0x7f4d48101080 2024-05-10T10:41:50.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 Processor -- start 2024-05-10T10:41:50.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 -- start start 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d44ff9640 1 -- 172.21.15.112:0/2375863390 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4d3804d070 con 0x7f4d48101080 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0f9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0fa2910 0x55acf0fa4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55acf0f015d0 con 0x55acf0fa2910 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acf0eba890 con 0x55acf0f9d7a0 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.773+0000 7f4d50c65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55acf0efd590 con 0x55acf0f9cdb0 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0fa2910 0x55acf0fa4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0fa2910 0x55acf0fa4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32770/0 (socket says 172.21.15.112:32770) 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 -- 172.21.15.112:0/421660689 learned_addr learned my addr 172.21.15.112:0/421660689 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d47fff640 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0f9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d477fe640 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 -- 172.21.15.112:0/421660689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 msgr2=0x55acf0f9d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0f9d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 -- 172.21.15.112:0/421660689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 msgr2=0x55acf0fa22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 -- 172.21.15.112:0/421660689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf0c13210 con 0x55acf0fa2910 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d47fff640 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0f9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0fa2910 0x55acf0fa4da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00db70 tx=0x7f4d3c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72e28607204ccaa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.774+0000 7f4d44ff9640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d300b2960 0x7f4d300b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.775+0000 7f4d47fff640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d300b2960 0x7f4d300b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.775+0000 7f4d2dffb640 1 -- 172.21.15.112:0/421660689 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00de60 con 0x55acf0fa2910 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.775+0000 7f4d44ff9640 1 -- 172.21.15.112:0/2375863390 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f4d381072b0 con 0x7f4d48101080 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.775+0000 7f4d2dffb640 1 -- 172.21.15.112:0/421660689 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d3c0035f0 con 0x55acf0fa2910 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.775+0000 7f4d2dffb640 1 -- 172.21.15.112:0/421660689 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0044c0 con 0x55acf0fa2910 2024-05-10T10:41:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 -- 172.21.15.112:0/421660689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0fa2910 msgr2=0x55acf0fa4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d47fff640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d300b2960 0x7f4d300b4e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55acf0f9e000 tx=0x7f4d40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0fa2910 0x55acf0fa4da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00db70 tx=0x7f4d3c005f90 comp rx=0 tx=0).stop 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 -- 172.21.15.112:0/421660689 shutdown_connections 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0fa2910 0x55acf0fa4da0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 --2- 172.21.15.112:0/421660689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0f9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 -- 172.21.15.112:0/421660689 >> 172.21.15.112:0/421660689 conn(0x55acf0f98420 msgr2=0x55acf0f98820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 -- 172.21.15.112:0/421660689 shutdown_connections 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 -- 172.21.15.112:0/421660689 wait complete. 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 -- start start 2024-05-10T10:41:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.776+0000 7f4d50c65740 1 Processor -- start 2024-05-10T10:41:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d50c65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55acf0fa58c0 con 0x55acf0f9d7a0 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d50c65740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acf0ebe370 con 0x55acf0fa2910 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d50c65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55acf0fa5a40 con 0x55acf0f9cdb0 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d50c65740 10 client.0 _ll_register_callbacks cb 0x55acf0c12a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32776/0 (socket says 172.21.15.112:32776) 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 -- 172.21.15.112:0/445041137 learned_addr learned my addr 172.21.15.112:0/445041137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d477fe640 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d47fff640 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 -- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 msgr2=0x55acf0fa6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 -- 172.21.15.112:0/445041137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 msgr2=0x55acf0fae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 -- 172.21.15.112:0/445041137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf0cdf620 con 0x55acf0f9d7a0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.777+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000c00 tx=0x7f4d3c00d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f778e5cce68e243d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d26ffd640 1 -- 172.21.15.112:0/445041137 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c025070 con 0x55acf0f9d7a0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d26ffd640 1 -- 172.21.15.112:0/445041137 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f4d3c004200 con 0x55acf0f9d7a0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c0035f0 con 0x55acf0f9d7a0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d26ffd640 1 -- 172.21.15.112:0/445041137 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c031050 con 0x55acf0f9d7a0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d50c65740 10 client.15804 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55acf0fb1580 con 0x55acf0f9d7a0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.778+0000 7f4d26ffd640 1 -- 172.21.15.112:0/445041137 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d3c004400 con 0x55acf0f9d7a0 2024-05-10T10:41:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d26ffd640 1 -- 172.21.15.112:0/445041137 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d3c004c20 con 0x55acf0f9d7a0 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d50c65740 10 client.15804 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d50c65740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d50c65740 20 client.15804 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d50c65740 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf0fb4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d50c65740 4 client.15804 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4d3c0035f0 con 0x55acf0fb2930 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d477fe640 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf0fb4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d477fe640 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf0fb4d20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd366cf34b7f8c49 server_cookie=d03bc99b49763f89 in_seq=0 out_seq=0 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.779+0000 7f4d26ffd640 10 client.15804 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.780+0000 7f4d26ffd640 1 -- 172.21.15.112:0/445041137 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4d3c0035f0 con 0x55acf0fb2930 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.780+0000 7f4d26ffd640 10 client.15804 handle_command_reply: tid=0 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.780+0000 7f4d50c65740 1 client.15804 shutdown 2024-05-10T10:41:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9d7a0 msgr2=0x55acf0fa7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000c00 tx=0x7f4d3c00d8c0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 shutdown_connections 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf0fb4d20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d26ffd640 0 client.15804 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 --2- 172.21.15.112:0/445041137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 >> 172.21.15.112:0/445041137 conn(0x55acf0f98420 msgr2=0x55acf0f97580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 shutdown_connections 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 1 -- 172.21.15.112:0/445041137 wait complete. 2024-05-10T10:41:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.781+0000 7f4d50c65740 20 client.15804 trim_cache size 0 max 16384 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 Processor -- start 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 -- start start 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32790/0 (socket says 172.21.15.112:32790) 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d477fe640 1 -- 172.21.15.112:0/2532521008 learned_addr learned my addr 172.21.15.112:0/2532521008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0fa2910 0x55acf0fae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3400fbd0 con 0x55acf0f9cdb0 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34011710 con 0x55acf0f9d7a0 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.813+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4d340102a0 con 0x55acf0fa2910 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d47fff640 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0fa2910 0x55acf0fae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d46ffd640 1 -- 172.21.15.112:0/2532521008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0fa2910 msgr2=0x55acf0fae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0fa2910 0x55acf0fae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d46ffd640 1 -- 172.21.15.112:0/2532521008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 msgr2=0x55acf0fa6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d46ffd640 1 -- 172.21.15.112:0/2532521008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf0f9e030 con 0x55acf0f9d7a0 2024-05-10T10:41:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d477fe640 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00a010 tx=0x7f4d3c00d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29478060855d2b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.814+0000 7f4d2d7fa640 1 -- 172.21.15.112:0/2532521008 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c025030 con 0x55acf0f9d7a0 2024-05-10T10:41:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.815+0000 7f4d2d7fa640 1 -- 172.21.15.112:0/2532521008 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d3c003ce0 con 0x55acf0f9d7a0 2024-05-10T10:41:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.815+0000 7f4d2d7fa640 1 -- 172.21.15.112:0/2532521008 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c02a440 con 0x55acf0f9d7a0 2024-05-10T10:41:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 msgr2=0x55acf0fa7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00a010 tx=0x7f4d3c00d490 comp rx=0 tx=0).stop 2024-05-10T10:41:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 shutdown_connections 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0fa2910 0x55acf0fae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0f9d7a0 0x55acf0fa7530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 --2- 172.21.15.112:0/2532521008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fa6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 >> 172.21.15.112:0/2532521008 conn(0x55acf0f98420 msgr2=0x55acf0f933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 shutdown_connections 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 -- 172.21.15.112:0/2532521008 wait complete. 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.816+0000 7f4d50c65740 1 -- start start 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.817+0000 7f4d50c65740 1 Processor -- start 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.817+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fabc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.817+0000 7f4d50c65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9d7a0 0x55acf0fac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.817+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fabc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fabc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32796/0 (socket says 172.21.15.112:32796) 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 -- 172.21.15.112:0/839819541 learned_addr learned my addr 172.21.15.112:0/839819541 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d50c65740 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34011040 con 0x55acf0f9cdb0 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34010970 con 0x55acf0fa2910 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3400dbb0 con 0x55acf0f9d7a0 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d47fff640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9d7a0 0x55acf0fac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 -- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9d7a0 msgr2=0x55acf0fac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9d7a0 0x55acf0fac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 -- 172.21.15.112:0/839819541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 msgr2=0x55acf0fad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 -- 172.21.15.112:0/839819541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf0fad7b0 con 0x55acf0f9cdb0 2024-05-10T10:41:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9d7a0 0x55acf0fac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d47fff640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.818+0000 7f4d477fe640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fabc70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d380ca3c0 tx=0x7f4d38005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4667cbaf7777b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 10 client.0 _ll_register_callbacks cb 0x7f4d400099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c003ce0 con 0x55acf0f9cdb0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d26ffd640 1 -- 172.21.15.112:0/839819541 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380cdff0 con 0x55acf0f9cdb0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d26ffd640 1 -- 172.21.15.112:0/839819541 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f4d38045470 con 0x55acf0f9cdb0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d26ffd640 1 -- 172.21.15.112:0/839819541 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d380cebe0 con 0x55acf0f9cdb0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 10 client.15810 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55acf1025950 con 0x55acf0f9cdb0 2024-05-10T10:41:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d26ffd640 1 -- 172.21.15.112:0/839819541 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d3800b040 con 0x55acf0f9cdb0 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 10 client.15810 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 20 client.15810 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf1028220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 4 client.15810 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.819+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acf0f0f1e0 con 0x55acf0fb2930 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.820+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf1028220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.820+0000 7f4d26ffd640 10 client.15810 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.820+0000 7f4d46ffd640 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf1028220 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61bc68c12a12b749 server_cookie=1d38e4f08f99e04b in_seq=0 out_seq=0 2024-05-10T10:41:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.825+0000 7f4d26ffd640 1 -- 172.21.15.112:0/839819541 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acf0f0f1e0 con 0x55acf0fb2930 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.825+0000 7f4d26ffd640 10 client.15810 handle_command_reply: tid=0 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.825+0000 7f4d50c65740 1 client.15810 shutdown 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 msgr2=0x55acf0fabc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fabc70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d380ca3c0 tx=0x7f4d38005fd0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 shutdown_connections 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55acf0fb2930 0x55acf1028220 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d26ffd640 0 client.15810 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf0fa2910 0x55acf0fad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acf0f9d7a0 0x55acf0fac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 --2- 172.21.15.112:0/839819541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acf0f9cdb0 0x55acf0fabc70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 >> 172.21.15.112:0/839819541 conn(0x55acf0f98420 msgr2=0x55acf0f9e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 shutdown_connections 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 1 -- 172.21.15.112:0/839819541 wait complete. 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.826+0000 7f4d50c65740 20 client.15810 trim_cache size 0 max 16384 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:41:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:41:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d300b2960 msgr2=0x7f4d300b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d300b2960 0x7f4d300b4e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55acf0f9e000 tx=0x7f4d40009500 comp rx=0 tx=0).stop 2024-05-10T10:41:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 msgr2=0x7f4d48196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d48196740 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a960 tx=0x7f4d38034f60 comp rx=0 tx=0).stop 2024-05-10T10:41:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 shutdown_connections 2024-05-10T10:41:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f4d300b2960 0x7f4d300b4e20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d48102c30 0x7f4d4819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d48102270 0x7f4d48196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 --2- 172.21.15.112:0/2375863390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d48101080 0x7f4d48196740 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.827+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 >> 172.21.15.112:0/2375863390 conn(0x7f4d480fc610 msgr2=0x7f4d481054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.828+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 shutdown_connections 2024-05-10T10:41:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:41:50.828+0000 7f4d4df46640 1 -- 172.21.15.112:0/2375863390 wait complete. 2024-05-10T10:41:51.048 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:51.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:50 smithi112 ceph-mon[30100]: pgmap v478: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:50 smithi148 ceph-mon[33568]: pgmap v478: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:50 smithi195 ceph-mon[33773]: pgmap v478: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:41:52.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 -- 172.21.15.112:0/453544860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb91406bb70 msgr2=0x7fb91406bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 --2- 172.21.15.112:0/453544860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb91406bb70 0x7fb91406bff0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb904009990 tx=0x7fb904009ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 -- 172.21.15.112:0/453544860 shutdown_connections 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 --2- 172.21.15.112:0/453544860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb91406c530 0x7fb914110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 --2- 172.21.15.112:0/453544860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb914109540 0x7fb914109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 --2- 172.21.15.112:0/453544860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb91406bb70 0x7fb91406bff0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 -- 172.21.15.112:0/453544860 >> 172.21.15.112:0/453544860 conn(0x7fb9140793e0 msgr2=0x7fb9140797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 -- 172.21.15.112:0/453544860 shutdown_connections 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.916+0000 7fb91c4de700 1 -- 172.21.15.112:0/453544860 wait complete. 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.917+0000 7fb91c4de700 1 Processor -- start 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.917+0000 7fb91c4de700 1 -- start start 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.917+0000 7fb91c4de700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb91406bb70 0x7fb91419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.917+0000 7fb91c4de700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91406c530 0x7fb91419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:52.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.917+0000 7fb91c4de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb914109540 0x7fb914199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb91c4de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb914113570 con 0x7fb914109540 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb91aa7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb914109540 0x7fb914199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb919a79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91406c530 0x7fb91419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb91c4de700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9141133f0 con 0x7fb91406c530 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb91c4de700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9141136f0 con 0x7fb91406bb70 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb91aa7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb914109540 0x7fb914199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32806/0 (socket says 172.21.15.112:32806) 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb91aa7b700 1 -- 172.21.15.112:0/531141532 learned_addr learned my addr 172.21.15.112:0/531141532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb91a27a700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb91406bb70 0x7fb91419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb919a79700 1 -- 172.21.15.112:0/531141532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb91406bb70 msgr2=0x7fb91419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb919a79700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb91406bb70 0x7fb91419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:52.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb919a79700 1 -- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb914109540 msgr2=0x7fb914199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb919a79700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb914109540 0x7fb914199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb919a79700 1 -- 172.21.15.112:0/531141532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91419a070 con 0x7fb91406c530 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.918+0000 7fb919a79700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91406c530 0x7fb91419fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9040098e0 tx=0x7fb9040047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.919+0000 7fb90b7fe700 1 -- 172.21.15.112:0/531141532 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904026070 con 0x7fb91406c530 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.919+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91419a300 con 0x7fb91406c530 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.919+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9141023d0 con 0x7fb91406c530 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.919+0000 7fb90b7fe700 1 -- 172.21.15.112:0/531141532 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb904021410 con 0x7fb91406c530 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.919+0000 7fb90b7fe700 1 -- 172.21.15.112:0/531141532 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904017650 con 0x7fb91406c530 2024-05-10T10:41:52.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.920+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb914104c00 con 0x7fb91406c530 2024-05-10T10:41:52.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.922+0000 7fb90b7fe700 1 -- 172.21.15.112:0/531141532 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb904029070 con 0x7fb91406c530 2024-05-10T10:41:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.923+0000 7fb90b7fe700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9000b2880 0x7fb9000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.923+0000 7fb91a27a700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9000b2880 0x7fb9000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.924+0000 7fb90b7fe700 1 -- 172.21.15.112:0/531141532 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb9040e3280 con 0x7fb91406c530 2024-05-10T10:41:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.924+0000 7fb91a27a700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9000b2880 0x7fb9000b4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb91419aa50 tx=0x7fb9100073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:52.928+0000 7fb90b7fe700 1 -- 172.21.15.112:0/531141532 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb904013070 con 0x7fb91406c530 2024-05-10T10:41:53.082 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:52 smithi112 ceph-mon[30100]: pgmap v479: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-10T10:41:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.195+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb91419ae40 con 0x7fb91406c530 2024-05-10T10:41:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.196+0000 7fb90b7fe700 1 -- 172.21.15.112:0/531141532 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fb9040aac40 con 0x7fb91406c530 2024-05-10T10:41:53.198 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:41:53.198 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:40:06.380474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:41:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9000b2880 msgr2=0x7fb9000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9000b2880 0x7fb9000b4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb91419aa50 tx=0x7fb9100073d0 comp rx=0 tx=0).stop 2024-05-10T10:41:53.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91406c530 msgr2=0x7fb91419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91406c530 0x7fb91419fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9040098e0 tx=0x7fb9040047b0 comp rx=0 tx=0).stop 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 shutdown_connections 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb914109540 0x7fb914199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb9000b2880 0x7fb9000b4d40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb91406bb70 0x7fb91419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 --2- 172.21.15.112:0/531141532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91406c530 0x7fb91419fc90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 >> 172.21.15.112:0/531141532 conn(0x7fb9140793e0 msgr2=0x7fb914101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:53.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.199+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 shutdown_connections 2024-05-10T10:41:53.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:53.200+0000 7fb91c4de700 1 -- 172.21.15.112:0/531141532 wait complete. 2024-05-10T10:41:53.202 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-05-10T10:41:53.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:52 smithi148 ceph-mon[33568]: pgmap v479: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-10T10:41:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:52 smithi195 ceph-mon[33773]: pgmap v479: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-10T10:41:53.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:41:54.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:53 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/531141532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:54.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:53 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/531141532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:54.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:53 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/531141532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:41:54.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:54 smithi112 ceph-mon[30100]: pgmap v480: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:41:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:54 smithi148 ceph-mon[33568]: pgmap v480: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:41:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:54 smithi195 ceph-mon[33773]: pgmap v480: 129 pgs: 129 active+clean; 673 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:41:56.524 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 94% (11548/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-10T10:41:56.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.629+0000 7f36c6345700 1 -- 172.21.15.112:0/1719097778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 msgr2=0x7f36c010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.629+0000 7f36c6345700 1 --2- 172.21.15.112:0/1719097778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c010f250 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f36b800b0a0 tx=0x7f36b800b3b0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.629+0000 7f36c6345700 1 -- 172.21.15.112:0/1719097778 shutdown_connections 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.629+0000 7f36c6345700 1 --2- 172.21.15.112:0/1719097778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c010f250 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.629+0000 7f36c6345700 1 --2- 172.21.15.112:0/1719097778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c0108410 0x7f36c01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.629+0000 7f36c6345700 1 --2- 172.21.15.112:0/1719097778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c01023a0 0x7f36c0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.629+0000 7f36c6345700 1 -- 172.21.15.112:0/1719097778 >> 172.21.15.112:0/1719097778 conn(0x7f36c0079350 msgr2=0x7f36c0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.630+0000 7f36c6345700 1 -- 172.21.15.112:0/1719097778 shutdown_connections 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.630+0000 7f36c6345700 1 -- 172.21.15.112:0/1719097778 wait complete. 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.630+0000 7f36c6345700 1 Processor -- start 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.630+0000 7f36c6345700 1 -- start start 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c6345700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c01023a0 0x7f36c019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c6345700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c6345700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c0108410 0x7f36c01a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c6345700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36c01124b0 con 0x7f36c0102d60 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c6345700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0112330 con 0x7f36c01023a0 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c6345700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0112630 con 0x7f36c0108410 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49404/0 (socket says 172.21.15.112:49404) 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 -- 172.21.15.112:0/4025248679 learned_addr learned my addr 172.21.15.112:0/4025248679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bffff700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c01023a0 0x7f36c019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c48e2700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c0108410 0x7f36c01a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 -- 172.21.15.112:0/4025248679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c0108410 msgr2=0x7f36c01a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c0108410 0x7f36c01a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 -- 172.21.15.112:0/4025248679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c01023a0 msgr2=0x7f36c019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c01023a0 0x7f36c019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36bf7fe700 1 -- 172.21.15.112:0/4025248679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c01a5f90 con 0x7f36c0102d60 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.631+0000 7f36c48e2700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c0108410 0x7f36c01a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.632+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c019af10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c920 tx=0x7f36b400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.632+0000 7f36bd7fa700 1 -- 172.21.15.112:0/4025248679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4007b00 con 0x7f36c0102d60 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.632+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c0198820 con 0x7f36c0102d60 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.632+0000 7f36bd7fa700 1 -- 172.21.15.112:0/4025248679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36b4004cc0 con 0x7f36c0102d60 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.632+0000 7f36bd7fa700 1 -- 172.21.15.112:0/4025248679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b40053f0 con 0x7f36c0102d60 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.632+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c0198db0 con 0x7f36c0102d60 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36bd7fa700 1 -- 172.21.15.112:0/4025248679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f36b4004750 con 0x7f36c0102d60 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 Processor -- start 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 -- start start 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58e96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 0x55b1c58ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1c5837860 con 0x55b1c58e54d0 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b1c57d67c0 con 0x55b1c58e5df0 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b1c5833820 con 0x55b1c58e9c80 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58e58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49418/0 (socket says 172.21.15.112:49418) 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 -- 172.21.15.112:0/101254977 learned_addr learned my addr 172.21.15.112:0/101254977 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36bffff700 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58e96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 -- 172.21.15.112:0/101254977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 msgr2=0x55b1c58ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 0x55b1c58ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 -- 172.21.15.112:0/101254977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 msgr2=0x55b1c58e96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58e96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.636+0000 7f36c48e2700 1 -- 172.21.15.112:0/101254977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c57848d0 con 0x55b1c58e54d0 2024-05-10T10:41:56.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.637+0000 7f36c48e2700 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58e58b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009310 tx=0x7f36b8009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41af5e4539efd8eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.637+0000 7f36bd7fa700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36a80b2880 0x7f36a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.637+0000 7f36bffff700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36a80b2880 0x7f36a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.637+0000 7f36a67fc700 1 -- 172.21.15.112:0/101254977 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8003c00 con 0x55b1c58e54d0 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.637+0000 7f36bd7fa700 1 -- 172.21.15.112:0/4025248679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f36b40e2130 con 0x7f36c0102d60 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.637+0000 7f36a67fc700 1 -- 172.21.15.112:0/101254977 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36b8009cd0 con 0x55b1c58e54d0 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.637+0000 7f36a67fc700 1 -- 172.21.15.112:0/101254977 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b801c6c0 con 0x55b1c58e54d0 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36bffff700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36a80b2880 0x7f36a80b4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009b80 tx=0x7f36b0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/101254977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 msgr2=0x55b1c58e58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58e58b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009310 tx=0x7f36b8009340 comp rx=0 tx=0).stop 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/101254977 shutdown_connections 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 0x55b1c58ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58e58b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/101254977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58e96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/101254977 >> 172.21.15.112:0/101254977 conn(0x55b1c5811100 msgr2=0x55b1c58dfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:56.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/101254977 shutdown_connections 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.638+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/101254977 wait complete. 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 -- start start 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 Processor -- start 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58f1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 0x55b1c58eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1c58f1e30 con 0x55b1c58e54d0 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b1c579dc30 con 0x55b1c58e5df0 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b1c58f1fb0 con 0x55b1c58e9c80 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36bffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36d0bd8b80 10 client.0 _ll_register_callbacks cb 0x55b1c57cf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36bffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58f1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49432/0 (socket says 172.21.15.112:49432) 2024-05-10T10:41:56.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36bffff700 1 -- 172.21.15.112:0/4169256117 learned_addr learned my addr 172.21.15.112:0/4169256117 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.639+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36c48e2700 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 0x55b1c58eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36bf7fe700 1 -- 172.21.15.112:0/4169256117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 msgr2=0x55b1c58eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 0x55b1c58eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36bf7fe700 1 -- 172.21.15.112:0/4169256117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 msgr2=0x55b1c58f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58f1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36bf7fe700 1 -- 172.21.15.112:0/4169256117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c57d3690 con 0x55b1c58e5df0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58ee540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b1c58e1710 tx=0x7f36b40a8d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc338d0cc8378ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f369f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f369f7fe700 1 -- 172.21.15.112:0/4169256117 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b401d040 con 0x55b1c58e5df0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f369f7fe700 1 -- 172.21.15.112:0/4169256117 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f36b4004a40 con 0x55b1c58e5df0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b1c58b1650 con 0x55b1c58e5df0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f369f7fe700 1 -- 172.21.15.112:0/4169256117 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36b40a93c0 con 0x55b1c58e5df0 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36d0bd8b80 10 client.25265 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:56.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.640+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1c58fc770 con 0x55b1c58e5df0 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f369f7fe700 1 -- 172.21.15.112:0/4169256117 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b1c58b1650 con 0x55b1c58e5df0 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f369f7fe700 1 -- 172.21.15.112:0/4169256117 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f36b40b1d90 con 0x55b1c58e5df0 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f36d0bd8b80 10 client.25265 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f36d0bd8b80 10 client.25265 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f36d0bd8b80 20 client.25265 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c58fedb0 0x55b1c59011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f36d0bd8b80 4 client.25265 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1c5845240 con 0x55b1c58fedb0 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.641+0000 7f36bffff700 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c58fedb0 0x55b1c59011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.642+0000 7f369f7fe700 10 client.25265 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.642+0000 7f36bffff700 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c58fedb0 0x55b1c59011a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8c77daeaefade48f server_cookie=f731fdb18948f981 in_seq=0 out_seq=0 2024-05-10T10:41:56.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.642+0000 7f369f7fe700 1 -- 172.21.15.112:0/4169256117 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1c5845240 con 0x55b1c58fedb0 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.642+0000 7f369f7fe700 10 client.25265 handle_command_reply: tid=0 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.642+0000 7f36d0bd8b80 1 client.25265 shutdown 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 msgr2=0x55b1c58ee540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58ee540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b1c58e1710 tx=0x7f36b40a8d90 comp rx=0 tx=0).stop 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 shutdown_connections 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c58fedb0 0x55b1c59011a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e9c80 0x55b1c58eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c58f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f369f7fe700 0 client.25265 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/4169256117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58ee540 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 >> 172.21.15.112:0/4169256117 conn(0x55b1c5811100 msgr2=0x55b1c58da720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 shutdown_connections 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/4169256117 wait complete. 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.643+0000 7f36d0bd8b80 20 client.25265 trim_cache size 0 max 16384 2024-05-10T10:41:56.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 Processor -- start 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 -- start start 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e54d0 0x55b1c58eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e9c80 0x55b1c58da5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36c01b1380 con 0x55b1c58e9c80 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36c01b2ec0 con 0x55b1c58e5df0 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36c01b1a50 con 0x55b1c58e54d0 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36c48e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e9c80 0x55b1c58da5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36c48e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e9c80 0x55b1c58da5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49442/0 (socket says 172.21.15.112:49442) 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.658+0000 7f36c48e2700 1 -- 172.21.15.112:0/666065449 learned_addr learned my addr 172.21.15.112:0/666065449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36bffff700 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e54d0 0x55b1c58eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36c48e2700 1 -- 172.21.15.112:0/666065449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e54d0 msgr2=0x55b1c58eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:56.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36c48e2700 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e54d0 0x55b1c58eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36c48e2700 1 -- 172.21.15.112:0/666065449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 msgr2=0x55b1c58faec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36c48e2700 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58faec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36c48e2700 1 -- 172.21.15.112:0/666065449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c58e1520 con 0x55b1c58e9c80 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36bffff700 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e54d0 0x55b1c58eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36c48e2700 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e9c80 0x55b1c58da5a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f36b80093b0 tx=0x7f36b8007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f55fe94302206a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36a5ffb700 1 -- 172.21.15.112:0/666065449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8003c30 con 0x55b1c58e9c80 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36a5ffb700 1 -- 172.21.15.112:0/666065449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36b8003dd0 con 0x55b1c58e9c80 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.659+0000 7f36a5ffb700 1 -- 172.21.15.112:0/666065449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8005550 con 0x55b1c58e9c80 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/666065449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e9c80 msgr2=0x55b1c58da5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e9c80 0x55b1c58da5a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f36b80093b0 tx=0x7f36b8007ee0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/666065449 shutdown_connections 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e9c80 0x55b1c58da5a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e54d0 0x55b1c58eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/666065449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e5df0 0x55b1c58faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/666065449 >> 172.21.15.112:0/666065449 conn(0x55b1c5811100 msgr2=0x55b1c58d9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/666065449 shutdown_connections 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.660+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/666065449 wait complete. 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 -- start start 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 Processor -- start 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c59607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e5df0 0x55b1c5960d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e9c80 0x55b1c58f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36c01b27f0 con 0x55b1c58e54d0 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36c01b2120 con 0x55b1c58e9c80 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36c01af320 con 0x55b1c58e5df0 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36bffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c59607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36d0bd8b80 10 client.0 _ll_register_callbacks cb 0x55b1c57cf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:56.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36bffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c59607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49444/0 (socket says 172.21.15.112:49444) 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.661+0000 7f36bffff700 1 -- 172.21.15.112:0/3090270098 learned_addr learned my addr 172.21.15.112:0/3090270098 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36c48e2700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e9c80 0x55b1c58f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e5df0 0x55b1c5960d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36bffff700 1 -- 172.21.15.112:0/3090270098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e5df0 msgr2=0x55b1c5960d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36bffff700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e5df0 0x55b1c5960d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36bffff700 1 -- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e9c80 msgr2=0x55b1c58f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36bffff700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e9c80 0x55b1c58f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36bffff700 1 -- 172.21.15.112:0/3090270098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c58f5000 con 0x55b1c58e54d0 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36c48e2700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e9c80 0x55b1c58f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36bffff700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c59607c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55b1c5901890 tx=0x7f36b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646ae0eb37fef42d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f369f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f369f7fe700 1 -- 172.21.15.112:0/3090270098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b002e070 con 0x55b1c58e54d0 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f369f7fe700 1 -- 172.21.15.112:0/3090270098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f36b002a2c0 con 0x55b1c58e54d0 2024-05-10T10:41:56.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.662+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b8003dd0 con 0x55b1c58e54d0 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f369f7fe700 1 -- 172.21.15.112:0/3090270098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36b003a050 con 0x55b1c58e54d0 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 10 client.15834 fetch_fsmap learned FSMap version 30 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1c5968fc0 con 0x55b1c58e54d0 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f369f7fe700 1 -- 172.21.15.112:0/3090270098 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36b8003dd0 con 0x55b1c58e54d0 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f369f7fe700 1 -- 172.21.15.112:0/3090270098 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f36b002ace0 con 0x55b1c58e54d0 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 10 client.15834 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 20 client.15834 populate_metadata read hostname 'smithi112' 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c59696b0 0x55b1c596bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 4 client.15834 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1c583f2e0 con 0x55b1c59696b0 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.663+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c59696b0 0x55b1c596bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f369f7fe700 10 client.15834 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:56.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f36bf7fe700 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c59696b0 0x55b1c596bad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abd09424e61c7f20 server_cookie=d87039a9fdf617a4 in_seq=0 out_seq=0 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f369f7fe700 1 -- 172.21.15.112:0/3090270098 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1c583f2e0 con 0x55b1c59696b0 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f369f7fe700 10 client.15834 handle_command_reply: tid=0 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f36d0bd8b80 1 client.15834 shutdown 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 msgr2=0x55b1c59607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c59607c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55b1c5901890 tx=0x7f36b0014000 comp rx=0 tx=0).stop 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 shutdown_connections 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.664+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1c58e9c80 0x55b1c58f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1c58e54d0 0x55b1c59607c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55b1c59696b0 0x55b1c596bad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36d0bd8b80 1 --2- 172.21.15.112:0/3090270098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1c58e5df0 0x55b1c5960d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f369f7fe700 0 client.15834 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 >> 172.21.15.112:0/3090270098 conn(0x55b1c5811100 msgr2=0x55b1c58da720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 shutdown_connections 2024-05-10T10:41:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36d0bd8b80 1 -- 172.21.15.112:0/3090270098 wait complete. 2024-05-10T10:41:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36d0bd8b80 20 client.15834 trim_cache size 0 max 16384 2024-05-10T10:41:56.682 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36a80b2880 msgr2=0x7f36a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36c6345700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36a80b2880 0x7f36a80b4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009b80 tx=0x7f36b0009190 comp rx=0 tx=0).stop 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 msgr2=0x7f36c019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.665+0000 7f36c6345700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c019af10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c920 tx=0x7f36b400cc30 comp rx=0 tx=0).stop 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 shutdown_connections 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f36a80b2880 0x7f36a80b4d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c0108410 0x7f36c01a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c01023a0 0x7f36c019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 --2- 172.21.15.112:0/4025248679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c0102d60 0x7f36c019af10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 >> 172.21.15.112:0/4025248679 conn(0x7f36c0079350 msgr2=0x7f36c01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 shutdown_connections 2024-05-10T10:41:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:41:56.666+0000 7f36c6345700 1 -- 172.21.15.112:0/4025248679 wait complete. 2024-05-10T10:41:57.259 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to 'd39fa08867836ac070c47cb66c782fa559fd480b'. 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-10T10:41:57.260 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-05-10T10:41:57.261 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-10T10:41:57.261 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-10T10:41:57.261 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-10T10:41:57.261 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at d39fa088678 client: fix file cache cap leak which can stall async read call 2024-05-10T10:41:57.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:56 smithi112 ceph-mon[30100]: pgmap v481: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-10T10:41:57.266 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-10T10:41:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:56 smithi148 ceph-mon[33568]: pgmap v481: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-10T10:41:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:56 smithi195 ceph-mon[33773]: pgmap v481: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-10T10:41:57.324 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-10T10:41:57.326 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T10:41:57.326 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-10T10:41:57.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:41:59.157 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-10T10:41:59.239 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-10T10:41:59.283 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T10:41:59.285 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T10:41:59.285 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-10T10:41:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:41:58 smithi148 ceph-mon[33568]: pgmap v482: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:59.303 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:41:58 smithi112 ceph-mon[30100]: pgmap v482: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:59.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:41:58 smithi195 ceph-mon[33773]: pgmap v482: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T10:41:59.325 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T10:41:59.328 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:41:59.329 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-10T10:41:59.345 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-05-10T10:41:59.347 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-05-10T10:41:59.347 DEBUG:teuthology.orchestra.run.smithi112:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39fa08867836ac070c47cb66c782fa559fd480b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-05-10T10:41:59.410 INFO:tasks.workunit.client.0.smithi112.stderr:+ set -e 2024-05-10T10:41:59.410 INFO:tasks.workunit.client.0.smithi112.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-05-10T10:41:59.414 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into 'xfstests-dev'... 2024-05-10T10:42:00.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 -- 172.21.15.112:0/1909290822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8108410 msgr2=0x7f01e81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/1909290822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8108410 0x7f01e81087f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009990 tx=0x7f01d8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 -- 172.21.15.112:0/1909290822 shutdown_connections 2024-05-10T10:42:00.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/1909290822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e8102d60 0x7f01e810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/1909290822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8108410 0x7f01e81087f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/1909290822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e81023a0 0x7f01e8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 -- 172.21.15.112:0/1909290822 >> 172.21.15.112:0/1909290822 conn(0x7f01e8079350 msgr2=0x7f01e8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 -- 172.21.15.112:0/1909290822 shutdown_connections 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 -- 172.21.15.112:0/1909290822 wait complete. 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 Processor -- start 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.175+0000 7f01ed5e1700 1 -- start start 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01ed5e1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e81023a0 0x7f01e819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01ed5e1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8102d60 0x7f01e819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01ed5e1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e8108410 0x7f01e81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01ed5e1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8112410 con 0x7f01e8102d60 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01ed5e1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8112290 con 0x7f01e81023a0 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01ed5e1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8112590 con 0x7f01e8108410 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8102d60 0x7f01e819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8102d60 0x7f01e819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49468/0 (socket says 172.21.15.112:49468) 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 -- 172.21.15.112:0/306779485 learned_addr learned my addr 172.21.15.112:0/306779485 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e77fe700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e8108410 0x7f01e81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e81023a0 0x7f01e819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 -- 172.21.15.112:0/306779485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e8108410 msgr2=0x7f01e81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e8108410 0x7f01e81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 -- 172.21.15.112:0/306779485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e81023a0 msgr2=0x7f01e819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e81023a0 0x7f01e819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 -- 172.21.15.112:0/306779485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e8199fa0 con 0x7f01e8102d60 2024-05-10T10:42:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.176+0000 7f01e67fc700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8102d60 0x7f01e819fb80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c9b0 tx=0x7f01dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.177+0000 7f01cffff700 1 -- 172.21.15.112:0/306779485 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc007b00 con 0x7f01e8102d60 2024-05-10T10:42:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.177+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e819a290 con 0x7f01e8102d60 2024-05-10T10:42:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.177+0000 7f01cffff700 1 -- 172.21.15.112:0/306779485 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01dc00ce50 con 0x7f01e8102d60 2024-05-10T10:42:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.177+0000 7f01cffff700 1 -- 172.21.15.112:0/306779485 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc018660 con 0x7f01e8102d60 2024-05-10T10:42:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.177+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e81a64a0 con 0x7f01e8102d60 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.179+0000 7f01cffff700 1 -- 172.21.15.112:0/306779485 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f01dc00f450 con 0x7f01e8102d60 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.180+0000 7f01cffff700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01d00b2880 0x7f01d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.180+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01d00b2880 0x7f01d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.180+0000 7f01cffff700 1 -- 172.21.15.112:0/306779485 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f01dc0e24d0 con 0x7f01e8102d60 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.180+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01d00b2880 0x7f01d00b4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f01d800b5c0 tx=0x7f01d8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.180+0000 7f01f7e74b80 1 Processor -- start 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.180+0000 7f01f7e74b80 1 -- start start 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f141798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1417d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1417e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f140cb860 con 0x564f141794d0 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f1406a7c0 con 0x564f14179df0 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f140c7820 con 0x564f1417dc80 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f141798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f141798b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49480/0 (socket says 172.21.15.112:49480) 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 -- 172.21.15.112:0/1747031992 learned_addr learned my addr 172.21.15.112:0/1747031992 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e67fc700 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1417d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e77fe700 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1417e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 -- 172.21.15.112:0/1747031992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 msgr2=0x564f1417e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1417e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 -- 172.21.15.112:0/1747031992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 msgr2=0x564f1417d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1417d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 -- 172.21.15.112:0/1747031992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f140188d0 con 0x564f141794d0 2024-05-10T10:42:00.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e77fe700 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1417e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e67fc700 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1417d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f141798b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f01d8015000 tx=0x7f01d801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db5fc8876d932b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.181+0000 7f01cd7fa700 1 -- 172.21.15.112:0/1747031992 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8034070 con 0x564f141794d0 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01cd7fa700 1 -- 172.21.15.112:0/1747031992 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01d802f900 con 0x564f141794d0 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01cd7fa700 1 -- 172.21.15.112:0/1747031992 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d800f900 con 0x564f141794d0 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 -- 172.21.15.112:0/1747031992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 msgr2=0x564f141798b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f141798b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f01d8015000 tx=0x7f01d801f000 comp rx=0 tx=0).stop 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 -- 172.21.15.112:0/1747031992 shutdown_connections 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1417e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f141798b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/1747031992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1417d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 -- 172.21.15.112:0/1747031992 >> 172.21.15.112:0/1747031992 conn(0x564f140a5100 msgr2=0x564f14173c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 -- 172.21.15.112:0/1747031992 shutdown_connections 2024-05-10T10:42:00.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.182+0000 7f01f7e74b80 1 -- 172.21.15.112:0/1747031992 wait complete. 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 -- start start 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 Processor -- start 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f141794d0 0x564f14185070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f14179df0 0x564f14182410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14182950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f14185e30 con 0x564f14179df0 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f14031c30 con 0x564f1417dc80 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f14185fb0 con 0x564f141794d0 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01f7e74b80 10 client.0 _ll_register_callbacks cb 0x564f14063920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f14179df0 0x564f14182410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f14179df0 0x564f14182410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49486/0 (socket says 172.21.15.112:49486) 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 -- 172.21.15.112:0/3482609029 learned_addr learned my addr 172.21.15.112:0/3482609029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f141794d0 0x564f14185070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e77fe700 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14182950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 -- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f141794d0 msgr2=0x564f14185070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f141794d0 0x564f14185070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 -- 172.21.15.112:0/3482609029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 msgr2=0x564f14182950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14182950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.183+0000 7f01e67fc700 1 -- 172.21.15.112:0/3482609029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f14067690 con 0x564f14179df0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01e67fc700 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f14179df0 0x564f14182410 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0a9460 tx=0x7f01dc00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf959ad8ef7e9f6f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01caff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01caff5700 1 -- 172.21.15.112:0/3482609029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0a9760 con 0x564f14179df0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01caff5700 1 -- 172.21.15.112:0/3482609029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f01dc007ca0 con 0x564f14179df0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d802f900 con 0x564f14179df0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01caff5700 1 -- 172.21.15.112:0/3482609029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01dc0adb10 con 0x564f14179df0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 10 client.15846 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f14190660 con 0x564f14179df0 2024-05-10T10:42:00.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01caff5700 1 -- 172.21.15.112:0/3482609029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01d802f900 con 0x564f14179df0 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01caff5700 1 -- 172.21.15.112:0/3482609029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f01dc0bebb0 con 0x564f14179df0 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 10 client.15846 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 20 client.15846 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f14192ca0 0x564f14195090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 4 client.15846 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.184+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f14145650 con 0x564f14192ca0 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.185+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f14192ca0 0x564f14195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.185+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f14192ca0 0x564f14195090 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc298dd9a79c5d37 server_cookie=ff439eb3e3c5d401 in_seq=0 out_seq=0 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.185+0000 7f01caff5700 10 client.15846 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:00.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01caff5700 1 -- 172.21.15.112:0/3482609029 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f14145650 con 0x564f14192ca0 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01caff5700 10 client.15846 handle_command_reply: tid=0 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 client.15846 shutdown 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f14179df0 msgr2=0x564f14182410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f14179df0 0x564f14182410 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0a9460 tx=0x7f01dc00ff70 comp rx=0 tx=0).stop 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 shutdown_connections 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f14192ca0 0x564f14195090 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14182950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f141794d0 0x564f14185070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3482609029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f14179df0 0x564f14182410 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 >> 172.21.15.112:0/3482609029 conn(0x564f140a5100 msgr2=0x564f1416e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01caff5700 0 client.15846 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 shutdown_connections 2024-05-10T10:42:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3482609029 wait complete. 2024-05-10T10:42:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.186+0000 7f01f7e74b80 20 client.15846 trim_cache size 0 max 16384 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 Processor -- start 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 -- start start 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1416e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81af120 con 0x564f141794d0 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81b0c60 con 0x564f14179df0 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81af7f0 con 0x564f1417dc80 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49500/0 (socket says 172.21.15.112:49500) 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 -- 172.21.15.112:0/841887121 learned_addr learned my addr 172.21.15.112:0/841887121 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e77fe700 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e67fc700 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1416e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 -- 172.21.15.112:0/841887121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 msgr2=0x564f1419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 -- 172.21.15.112:0/841887121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 msgr2=0x564f1416e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1416e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 -- 172.21.15.112:0/841887121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f14175520 con 0x564f141794d0 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e67fc700 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1416e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418ef20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f01d8032040 tx=0x7f01d8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ad0c07052dbb05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.201+0000 7f01ccff9700 1 -- 172.21.15.112:0/841887121 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8034040 con 0x564f141794d0 2024-05-10T10:42:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01ccff9700 1 -- 172.21.15.112:0/841887121 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01d8036040 con 0x564f141794d0 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01ccff9700 1 -- 172.21.15.112:0/841887121 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d802f610 con 0x564f141794d0 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 -- 172.21.15.112:0/841887121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 msgr2=0x564f1418ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418ef20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f01d8032040 tx=0x7f01d8005230 comp rx=0 tx=0).stop 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 -- 172.21.15.112:0/841887121 shutdown_connections 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f1417dc80 0x564f1419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418ef20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/841887121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f14179df0 0x564f1416e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 -- 172.21.15.112:0/841887121 >> 172.21.15.112:0/841887121 conn(0x564f140a5100 msgr2=0x564f1416de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 -- 172.21.15.112:0/841887121 shutdown_connections 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 -- 172.21.15.112:0/841887121 wait complete. 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 -- start start 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.202+0000 7f01f7e74b80 1 Processor -- start 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f14179df0 0x564f1418b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01f7e74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14186930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81b0590 con 0x564f141794d0 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81afec0 con 0x564f1417dc80 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01f7e74b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81ad0c0 con 0x564f14179df0 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01f7e74b80 10 client.0 _ll_register_callbacks cb 0x564f14063920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e67fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f14179df0 0x564f1418b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49508/0 (socket says 172.21.15.112:49508) 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 -- 172.21.15.112:0/3300741027 learned_addr learned my addr 172.21.15.112:0/3300741027 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e77fe700 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14186930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 -- 172.21.15.112:0/3300741027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f14179df0 msgr2=0x564f1418b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f14179df0 0x564f1418b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 -- 172.21.15.112:0/3300741027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 msgr2=0x564f14186930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:00.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14186930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 -- 172.21.15.112:0/3300741027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f14187170 con 0x564f141794d0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e77fe700 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14186930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01e6ffd700 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418b3d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f01d802f4e0 tx=0x7f01d8005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5030ca98281581 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01caff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.203+0000 7f01caff5700 1 -- 172.21.15.112:0/3300741027 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8040030 con 0x564f141794d0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01caff5700 1 -- 172.21.15.112:0/3300741027 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f01d802b070 con 0x564f141794d0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d8036040 con 0x564f141794d0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01caff5700 1 -- 172.21.15.112:0/3300741027 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01d8034070 con 0x564f141794d0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 10 client.15852 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f14201440 con 0x564f141794d0 2024-05-10T10:42:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01caff5700 1 -- 172.21.15.112:0/3300741027 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01d8036040 con 0x564f141794d0 2024-05-10T10:42:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01caff5700 1 -- 172.21.15.112:0/3300741027 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f01d8039070 con 0x564f141794d0 2024-05-10T10:42:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 10 client.15852 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:42:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 20 client.15852 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f142016c0 0x564f14203ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:00.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 4 client.15852 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f140d9240 con 0x564f142016c0 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.204+0000 7f01e67fc700 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f142016c0 0x564f14203ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01e67fc700 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f142016c0 0x564f14203ab0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0a1c0ea347c3399 server_cookie=1671afa9a9c11dd1 in_seq=0 out_seq=0 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01caff5700 10 client.15852 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01caff5700 1 -- 172.21.15.112:0/3300741027 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f140d9240 con 0x564f142016c0 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01caff5700 10 client.15852 handle_command_reply: tid=0 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 client.15852 shutdown 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 msgr2=0x564f1418b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418b3d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f01d802f4e0 tx=0x7f01d8005530 comp rx=0 tx=0).stop 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 shutdown_connections 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x564f142016c0 0x564f14203ab0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f1417dc80 0x564f14186930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f141794d0 0x564f1418b3d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 --2- 172.21.15.112:0/3300741027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f14179df0 0x564f1418b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 >> 172.21.15.112:0/3300741027 conn(0x564f140a5100 msgr2=0x564f141782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01caff5700 0 client.15852 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 shutdown_connections 2024-05-10T10:42:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 1 -- 172.21.15.112:0/3300741027 wait complete. 2024-05-10T10:42:00.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.205+0000 7f01f7e74b80 20 client.15852 trim_cache size 0 max 16384 2024-05-10T10:42:00.213 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01d00b2880 msgr2=0x7f01d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01d00b2880 0x7f01d00b4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f01d800b5c0 tx=0x7f01d8005960 comp rx=0 tx=0).stop 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8102d60 msgr2=0x7f01e819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8102d60 0x7f01e819fb80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c9b0 tx=0x7f01dc00ccc0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 shutdown_connections 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f01d00b2880 0x7f01d00b4d40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01e8108410 0x7f01e81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e81023a0 0x7f01e819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 --2- 172.21.15.112:0/306779485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01e8102d60 0x7f01e819fb80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:00.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 >> 172.21.15.112:0/306779485 conn(0x7f01e8079350 msgr2=0x7f01e806bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:00.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 shutdown_connections 2024-05-10T10:42:00.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:00.206+0000 7f01ed5e1700 1 -- 172.21.15.112:0/306779485 wait complete. 2024-05-10T10:42:01.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:00 smithi112 ceph-mon[30100]: pgmap v483: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:42:01.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:42:01.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:00 smithi148 ceph-mon[33568]: pgmap v483: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:42:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:00 smithi195 ceph-mon[33773]: pgmap v483: 129 pgs: 129 active+clean; 674 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T10:42:01.460 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:02.950 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:02 smithi112 ceph-mon[30100]: pgmap v484: 129 pgs: 129 active+clean; 690 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-10T10:42:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:02 smithi148 ceph-mon[33568]: pgmap v484: 129 pgs: 129 active+clean; 690 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-10T10:42:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:02 smithi195 ceph-mon[33773]: pgmap v484: 129 pgs: 129 active+clean; 690 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 -- 172.21.15.112:0/3736335016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4103ee0 msgr2=0x7fd8f4110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/3736335016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4103ee0 0x7fd8f4110370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009f90 tx=0x7fd8ec00b330 comp rx=0 tx=0).stop 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 -- 172.21.15.112:0/3736335016 shutdown_connections 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/3736335016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4103ee0 0x7fd8f4110370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/3736335016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f4109530 0x7fd8f4109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/3736335016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4103520 0x7fd8f41039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 -- 172.21.15.112:0/3736335016 >> 172.21.15.112:0/3736335016 conn(0x7fd8f407a570 msgr2=0x7fd8f407a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 -- 172.21.15.112:0/3736335016 shutdown_connections 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.382+0000 7fd8f97a8700 1 -- 172.21.15.112:0/3736335016 wait complete. 2024-05-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 Processor -- start 2024-05-10T10:42:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 -- start start 2024-05-10T10:42:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f4103520 0x7fd8f419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4103ee0 0x7fd8f419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4109530 0x7fd8f4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4113530 con 0x7fd8f4103ee0 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41133b0 con 0x7fd8f4109530 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.383+0000 7fd8f97a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41136b0 con 0x7fd8f4103520 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4103ee0 0x7fd8f419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f4103520 0x7fd8f419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4103ee0 0x7fd8f419fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49528/0 (socket says 172.21.15.112:49528) 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f4103520 0x7fd8f419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:55306/0 (socket says 172.21.15.112:55306) 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f27fc700 1 -- 172.21.15.112:0/652893820 learned_addr learned my addr 172.21.15.112:0/652893820 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4109530 0x7fd8f4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f37fe700 1 -- 172.21.15.112:0/652893820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f4103520 msgr2=0x7fd8f419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f4103520 0x7fd8f419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f37fe700 1 -- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4103ee0 msgr2=0x7fd8f419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4103ee0 0x7fd8f419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f37fe700 1 -- 172.21.15.112:0/652893820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f4199ff0 con 0x7fd8f4109530 2024-05-10T10:42:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4109530 0x7fd8f4199810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009790 tx=0x7fd8ec00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8dbfff700 1 -- 172.21.15.112:0/652893820 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec021070 con 0x7fd8f4109530 2024-05-10T10:42:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8dbfff700 1 -- 172.21.15.112:0/652893820 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8ec007ac0 con 0x7fd8f4109530 2024-05-10T10:42:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.384+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f419a280 con 0x7fd8f4109530 2024-05-10T10:42:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.385+0000 7fd8dbfff700 1 -- 172.21.15.112:0/652893820 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0052a0 con 0x7fd8f4109530 2024-05-10T10:42:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.385+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41a64a0 con 0x7fd8f4109530 2024-05-10T10:42:03.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.387+0000 7fd8dbfff700 1 -- 172.21.15.112:0/652893820 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd8ec024070 con 0x7fd8f4109530 2024-05-10T10:42:03.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.388+0000 7fd8dbfff700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8dc0b2900 0x7fd8dc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.388+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8dc0b2900 0x7fd8dc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.389+0000 7fd8dbfff700 1 -- 172.21.15.112:0/652893820 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd8ec00f020 con 0x7fd8f4109530 2024-05-10T10:42:03.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.389+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8dc0b2900 0x7fd8dc0b4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009bf0 tx=0x7fd8e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.389+0000 7fd90403bb80 1 Processor -- start 2024-05-10T10:42:03.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.389+0000 7fd90403bb80 1 -- start start 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.389+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.389+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd90403bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7cd739860 con 0x55a7cd7ebc80 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd90403bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7cd6d87c0 con 0x55a7cd7e74d0 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd90403bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7cd735820 con 0x55a7cd7e7df0 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7e78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:36088/0 (socket says 172.21.15.112:36088) 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 -- 172.21.15.112:0/2477845135 learned_addr learned my addr 172.21.15.112:0/2477845135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 -- 172.21.15.112:0/2477845135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 msgr2=0x55a7cd7eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 -- 172.21.15.112:0/2477845135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 msgr2=0x55a7cd7ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7ec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 -- 172.21.15.112:0/2477845135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7cd6868d0 con 0x55a7cd7e74d0 2024-05-10T10:42:03.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.390+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7e78b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4011e00 tx=0x7fd8e4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176ba4125a133e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.391+0000 7fd8d97fa700 1 -- 172.21.15.112:0/2477845135 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4012bc0 con 0x55a7cd7e74d0 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.391+0000 7fd8d97fa700 1 -- 172.21.15.112:0/2477845135 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8e40044c0 con 0x55a7cd7e74d0 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.391+0000 7fd8d97fa700 1 -- 172.21.15.112:0/2477845135 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e401f420 con 0x55a7cd7e74d0 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 -- 172.21.15.112:0/2477845135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 msgr2=0x55a7cd7e78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7e78b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4011e00 tx=0x7fd8e4014000 comp rx=0 tx=0).stop 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 -- 172.21.15.112:0/2477845135 shutdown_connections 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7e78b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 --2- 172.21.15.112:0/2477845135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 -- 172.21.15.112:0/2477845135 >> 172.21.15.112:0/2477845135 conn(0x55a7cd713100 msgr2=0x55a7cd7e1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 -- 172.21.15.112:0/2477845135 shutdown_connections 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.392+0000 7fd90403bb80 1 -- 172.21.15.112:0/2477845135 wait complete. 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 -- start start 2024-05-10T10:42:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 Processor -- start 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7cd7f3e90 con 0x55a7cd7ebc80 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7cd69fc30 con 0x55a7cd7e74d0 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7cd7f4010 con 0x55a7cd7e7df0 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd90403bb80 10 client.0 _ll_register_callbacks cb 0x55a7cd6d1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:55320/0 (socket says 172.21.15.112:55320) 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd8f27fc700 1 -- 172.21.15.112:0/674856702 learned_addr learned my addr 172.21.15.112:0/674856702 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.393+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8f27fc700 1 -- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 msgr2=0x55a7cd7f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8f27fc700 1 -- 172.21.15.112:0/674856702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 msgr2=0x55a7cd7f09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7f09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8f27fc700 1 -- 172.21.15.112:0/674856702 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7cd6d5690 con 0x55a7cd7e7df0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f0460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80128d0 tx=0x7fd8e8012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d8f2e08cff24993 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/674856702 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800f670 con 0x55a7cd7e7df0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/674856702 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd8e800f810 con 0x55a7cd7e7df0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e40044c0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/674856702 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8e800fa50 con 0x55a7cd7e7df0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd90403bb80 10 client.25321 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.394+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7cd7fe750 con 0x55a7cd7e7df0 2024-05-10T10:42:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/674856702 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8e40044c0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/674856702 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd8e8010c30 con 0x55a7cd7e7df0 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd90403bb80 10 client.25321 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd90403bb80 10 client.25321 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd90403bb80 20 client.25321 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd90403bb80 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd800de0 0x55a7cd8031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd90403bb80 4 client.25321 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7cd7b3650 con 0x55a7cd800de0 2024-05-10T10:42:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.395+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd800de0 0x55a7cd8031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.396+0000 7fd8d6ff5700 10 client.25321 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.396+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd800de0 0x55a7cd8031d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27c22ead5c04b116 server_cookie=d6e99de1ab55b644 in_seq=0 out_seq=0 2024-05-10T10:42:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/674856702 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7cd7b3650 con 0x55a7cd800de0 2024-05-10T10:42:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd8d6ff5700 10 client.25321 handle_command_reply: tid=0 2024-05-10T10:42:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 client.25321 shutdown 2024-05-10T10:42:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 msgr2=0x55a7cd7f0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f0460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80128d0 tx=0x7fd8e8012be0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 shutdown_connections 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd800de0 0x55a7cd8031d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7f09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 --2- 172.21.15.112:0/674856702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f0460 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 >> 172.21.15.112:0/674856702 conn(0x55a7cd713100 msgr2=0x55a7cd7dc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd8d6ff5700 0 client.25321 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 shutdown_connections 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 1 -- 172.21.15.112:0/674856702 wait complete. 2024-05-10T10:42:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.397+0000 7fd90403bb80 20 client.25321 trim_cache size 0 max 16384 2024-05-10T10:42:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.416+0000 7fd90403bb80 1 Processor -- start 2024-05-10T10:42:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.416+0000 7fd90403bb80 1 -- start start 2024-05-10T10:42:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7dc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd90403bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41af0e0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd90403bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41b0c20 con 0x55a7cd7ebc80 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd90403bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41af7b0 con 0x55a7cd7e74d0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:55332/0 (socket says 172.21.15.112:55332) 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f2ffd700 1 -- 172.21.15.112:0/4249274008 learned_addr learned my addr 172.21.15.112:0/4249274008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7dc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f27fc700 1 -- 172.21.15.112:0/4249274008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e74d0 msgr2=0x55a7cd7f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f27fc700 1 -- 172.21.15.112:0/4249274008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7ebc80 msgr2=0x55a7cd7dc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7dc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f27fc700 1 -- 172.21.15.112:0/4249274008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7cd7ee9f0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.417+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7fcf20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8000c00 tx=0x7fd8e801c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c1f5b952cf609b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.418+0000 7fd8d8ff9700 1 -- 172.21.15.112:0/4249274008 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e801c7b0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.418+0000 7fd8d8ff9700 1 -- 172.21.15.112:0/4249274008 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8e800fe00 con 0x55a7cd7e7df0 2024-05-10T10:42:03.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.418+0000 7fd8d8ff9700 1 -- 172.21.15.112:0/4249274008 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8003ae0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 -- 172.21.15.112:0/4249274008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7e7df0 msgr2=0x55a7cd7fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7fcf20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8000c00 tx=0x7fd8e801c630 comp rx=0 tx=0).stop 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 -- 172.21.15.112:0/4249274008 shutdown_connections 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7ebc80 0x55a7cd7dc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 --2- 172.21.15.112:0/4249274008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7fcf20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 -- 172.21.15.112:0/4249274008 >> 172.21.15.112:0/4249274008 conn(0x55a7cd713100 msgr2=0x55a7cd7dbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 -- 172.21.15.112:0/4249274008 shutdown_connections 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.419+0000 7fd90403bb80 1 -- 172.21.15.112:0/4249274008 wait complete. 2024-05-10T10:42:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 -- start start 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 Processor -- start 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd86da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41b0550 con 0x55a7cd7ebc80 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41afe80 con 0x55a7cd7e74d0 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41ad080 con 0x55a7cd7e7df0 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd90403bb80 10 client.0 _ll_register_callbacks cb 0x55a7cd6d1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd8f27fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:55348/0 (socket says 172.21.15.112:55348) 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd8f27fc700 1 -- 172.21.15.112:0/3908710257 learned_addr learned my addr 172.21.15.112:0/3908710257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd86da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.420+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f27fc700 1 -- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 msgr2=0x55a7cd7f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f27fc700 1 -- 172.21.15.112:0/3908710257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 msgr2=0x55a7cd86da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd86da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f27fc700 1 -- 172.21.15.112:0/3908710257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7cd86e010 con 0x55a7cd7e7df0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f37fe700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd86da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8f27fc700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f4ae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8e801c530 tx=0x7fd8e8012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae490782bff6b32f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/3908710257 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80052c0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/3908710257 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd8e8005460 con 0x55a7cd7e7df0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.421+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e800fe00 con 0x55a7cd7e7df0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/3908710257 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8e8039050 con 0x55a7cd7e7df0 2024-05-10T10:42:03.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 10 client.25327 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7cd86f2f0 con 0x55a7cd7e7df0 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/3908710257 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8e800fe00 con 0x55a7cd7e7df0 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/3908710257 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd8e8010430 con 0x55a7cd7e7df0 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 10 client.25327 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 10 client.25327 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 20 client.25327 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd86f930 0x55a7cd871d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 4 client.25327 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7cd747240 con 0x55a7cd86f930 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.422+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd86f930 0x55a7cd871d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd8d6ff5700 10 client.25327 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd8f2ffd700 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd86f930 0x55a7cd871d50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c73661ede9d875a1 server_cookie=4a2a77312b486a6 in_seq=0 out_seq=0 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd8d6ff5700 1 -- 172.21.15.112:0/3908710257 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7cd747240 con 0x55a7cd86f930 2024-05-10T10:42:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd8d6ff5700 10 client.25327 handle_command_reply: tid=0 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 client.25327 shutdown 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 msgr2=0x55a7cd7f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f4ae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8e801c530 tx=0x7fd8e8012be0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 shutdown_connections 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55a7cd86f930 0x55a7cd871d50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7cd7ebc80 0x55a7cd86da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd8d6ff5700 0 client.25327 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cd7e74d0 0x55a7cd7f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 --2- 172.21.15.112:0/3908710257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7cd7e7df0 0x55a7cd7f4ae0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 >> 172.21.15.112:0/3908710257 conn(0x55a7cd713100 msgr2=0x55a7cd7dc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 shutdown_connections 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 1 -- 172.21.15.112:0/3908710257 wait complete. 2024-05-10T10:42:03.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.423+0000 7fd90403bb80 20 client.25327 trim_cache size 0 max 16384 2024-05-10T10:42:03.430 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8dc0b2900 msgr2=0x7fd8dc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8dc0b2900 0x7fd8dc0b4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009bf0 tx=0x7fd8e4009260 comp rx=0 tx=0).stop 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4109530 msgr2=0x7fd8f4199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4109530 0x7fd8f4199810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009790 tx=0x7fd8ec00ccb0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 shutdown_connections 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4109530 0x7fd8f4199810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8dc0b2900 0x7fd8dc0b4dc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f4103520 0x7fd8f419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 --2- 172.21.15.112:0/652893820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8f4103ee0 0x7fd8f419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 >> 172.21.15.112:0/652893820 conn(0x7fd8f407a570 msgr2=0x7fd8f41021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 shutdown_connections 2024-05-10T10:42:03.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:03.424+0000 7fd8f97a8700 1 -- 172.21.15.112:0/652893820 wait complete. 2024-05-10T10:42:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.920+0000 7f03e5445640 1 -- 172.21.15.112:0/1368162206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0101b90 msgr2=0x7f03e0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.920+0000 7f03e5445640 1 --2- 172.21.15.112:0/1368162206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0101b90 0x7f03e0102010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f03c800fb60 tx=0x7f03c802e4b0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- 172.21.15.112:0/1368162206 shutdown_connections 2024-05-10T10:42:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 --2- 172.21.15.112:0/1368162206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102550 0x7f03e0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 --2- 172.21.15.112:0/1368162206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0101b90 0x7f03e0102010 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 --2- 172.21.15.112:0/1368162206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0102f40 0x7f03e0103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- 172.21.15.112:0/1368162206 >> 172.21.15.112:0/1368162206 conn(0x7f03e00fc620 msgr2=0x7f03e00fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- 172.21.15.112:0/1368162206 shutdown_connections 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- 172.21.15.112:0/1368162206 wait complete. 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 Processor -- start 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- start start 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0101b90 0x7f03e019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0102550 0x7f03e019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102f40 0x7f03e01a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0114ad0 con 0x7f03e0102f40 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0114950 con 0x7f03e0101b90 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03e5445640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0114c50 con 0x7f03e0102550 2024-05-10T10:42:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102f40 0x7f03e01a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.921+0000 7f03deffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0101b90 0x7f03e019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102f40 0x7f03e01a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49586/0 (socket says 172.21.15.112:49586) 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 -- 172.21.15.112:0/2460195055 learned_addr learned my addr 172.21.15.112:0/2460195055 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03d7fff640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0102550 0x7f03e019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 -- 172.21.15.112:0/2460195055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0102550 msgr2=0x7f03e019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0102550 0x7f03e019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 -- 172.21.15.112:0/2460195055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0101b90 msgr2=0x7f03e019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0101b90 0x7f03e019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 -- 172.21.15.112:0/2460195055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e01a6d00 con 0x7f03e0102f40 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03deffd640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0101b90 0x7f03e019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03df7fe640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102f40 0x7f03e01a6690 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f03d800efc0 tx=0x7f03d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03dcff9640 1 -- 172.21.15.112:0/2460195055 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8003970 con 0x7f03e0102f40 2024-05-10T10:42:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03dcff9640 1 -- 172.21.15.112:0/2460195055 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03d8004020 con 0x7f03e0102f40 2024-05-10T10:42:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e01a6ff0 con 0x7f03e0102f40 2024-05-10T10:42:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03dcff9640 1 -- 172.21.15.112:0/2460195055 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d800b850 con 0x7f03e0102f40 2024-05-10T10:42:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.922+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a7460 con 0x7f03e0102f40 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.925+0000 7f03e8239740 1 Processor -- start 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.925+0000 7f03dcff9640 1 -- 172.21.15.112:0/2460195055 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f03d801d020 con 0x7f03e0102f40 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.925+0000 7f03e8239740 1 -- start start 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03e8239740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03e8239740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03e8239740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3f4910 0x5632cb3f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03e8239740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632cb3535d0 con 0x5632cb3f4910 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03e8239740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5632cb30c890 con 0x5632cb3ef7a0 2024-05-10T10:42:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03e8239740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5632cb34f590 con 0x5632cb3eedb0 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3f4910 0x5632cb3f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03deffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3f4910 0x5632cb3f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49602/0 (socket says 172.21.15.112:49602) 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03d7fff640 1 -- 172.21.15.112:0/1892432480 learned_addr learned my addr 172.21.15.112:0/1892432480 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03deffd640 1 -- 172.21.15.112:0/1892432480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 msgr2=0x5632cb3ef1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03deffd640 1 --2- 172.21.15.112:0/1892432480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3ef1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03deffd640 1 -- 172.21.15.112:0/1892432480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3f4910 msgr2=0x5632cb3f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03deffd640 1 --2- 172.21.15.112:0/1892432480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3f4910 0x5632cb3f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03deffd640 1 -- 172.21.15.112:0/1892432480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632cb065210 con 0x5632cb3ef7a0 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03dcff9640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f03cc0b28a0 0x7f03cc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03deffd640 1 --2- 172.21.15.112:0/1892432480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f42a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03d000bee0 tx=0x7f03d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88bb673d6c2e405 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.926+0000 7f03d7fff640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f03cc0b28a0 0x7f03cc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.927+0000 7f03dcff9640 1 -- 172.21.15.112:0/2460195055 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f03d80e2430 con 0x7f03e0102f40 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.927+0000 7f03d57fa640 1 -- 172.21.15.112:0/1892432480 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0004230 con 0x5632cb3ef7a0 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.927+0000 7f03d57fa640 1 -- 172.21.15.112:0/1892432480 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03d00043d0 con 0x5632cb3ef7a0 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.927+0000 7f03d57fa640 1 -- 172.21.15.112:0/1892432480 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0016820 con 0x5632cb3ef7a0 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.927+0000 7f03e8239740 1 -- 172.21.15.112:0/1892432480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3ef7a0 msgr2=0x5632cb3f42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.927+0000 7f03d7fff640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f03cc0b28a0 0x7f03cc0b4d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f03c802eaa0 tx=0x7f03c8005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.927+0000 7f03e8239740 1 --2- 172.21.15.112:0/1892432480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f42a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03d000bee0 tx=0x7f03d0005f90 comp rx=0 tx=0).stop 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- 172.21.15.112:0/1892432480 shutdown_connections 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 --2- 172.21.15.112:0/1892432480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3f4910 0x5632cb3f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 --2- 172.21.15.112:0/1892432480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f42a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 --2- 172.21.15.112:0/1892432480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3ef1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- 172.21.15.112:0/1892432480 >> 172.21.15.112:0/1892432480 conn(0x5632cb3ea420 msgr2=0x5632cb3ea820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- 172.21.15.112:0/1892432480 shutdown_connections 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- 172.21.15.112:0/1892432480 wait complete. 2024-05-10T10:42:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- start start 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 Processor -- start 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3eedb0 0x5632cb3f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3f4910 0x5632cb400500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632cb3f78c0 con 0x5632cb3ef7a0 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5632cb310370 con 0x5632cb3eedb0 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5632cb3f7a40 con 0x5632cb3f4910 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03e8239740 10 client.0 _ll_register_callbacks cb 0x5632cb064a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49614/0 (socket says 172.21.15.112:49614) 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03d7fff640 1 -- 172.21.15.112:0/1107950838 learned_addr learned my addr 172.21.15.112:0/1107950838 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03df7fe640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3f4910 0x5632cb400500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.928+0000 7f03deffd640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3eedb0 0x5632cb3f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03d7fff640 1 -- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3f4910 msgr2=0x5632cb400500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03d7fff640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3f4910 0x5632cb400500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03d7fff640 1 -- 172.21.15.112:0/1107950838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3eedb0 msgr2=0x5632cb3f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03d7fff640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3eedb0 0x5632cb3f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03d7fff640 1 -- 172.21.15.112:0/1107950838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632cb131620 con 0x5632cb3ef7a0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03deffd640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3eedb0 0x5632cb3f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03d7fff640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f9530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f03c802e9d0 tx=0x7f03c802e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff21d5658ffeb953 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03c67fc640 1 -- 172.21.15.112:0/1107950838 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8052030 con 0x5632cb3ef7a0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03c67fc640 1 -- 172.21.15.112:0/1107950838 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f03c8035b60 con 0x5632cb3ef7a0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d00043d0 con 0x5632cb3ef7a0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03c67fc640 1 -- 172.21.15.112:0/1107950838 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f03c805f050 con 0x5632cb3ef7a0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 10 client.15882 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632cb403550 con 0x5632cb3ef7a0 2024-05-10T10:42:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03c67fc640 1 -- 172.21.15.112:0/1107950838 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03d00043d0 con 0x5632cb3ef7a0 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03c67fc640 1 -- 172.21.15.112:0/1107950838 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f03c800f040 con 0x5632cb3ef7a0 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 10 client.15882 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 20 client.15882 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb404c40 0x5632cb407090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 4 client.15882 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.929+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632cb3611e0 con 0x5632cb404c40 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.930+0000 7f03deffd640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb404c40 0x5632cb407090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.930+0000 7f03c67fc640 10 client.15882 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.930+0000 7f03deffd640 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb404c40 0x5632cb407090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83fe58614ec34d9 server_cookie=8dc36d337b614858 in_seq=0 out_seq=0 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03c67fc640 1 -- 172.21.15.112:0/1107950838 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5632cb3611e0 con 0x5632cb404c40 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03c67fc640 10 client.15882 handle_command_reply: tid=0 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 client.15882 shutdown 2024-05-10T10:42:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 msgr2=0x5632cb3f9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f9530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f03c802e9d0 tx=0x7f03c802e4b0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 shutdown_connections 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3f4910 0x5632cb400500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f9530 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb404c40 0x5632cb407090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 --2- 172.21.15.112:0/1107950838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3eedb0 0x5632cb3f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03c67fc640 0 client.15882 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 >> 172.21.15.112:0/1107950838 conn(0x5632cb3ea420 msgr2=0x5632cb3e9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 shutdown_connections 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 1 -- 172.21.15.112:0/1107950838 wait complete. 2024-05-10T10:42:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.931+0000 7f03e8239740 20 client.15882 trim_cache size 0 max 16384 2024-05-10T10:42:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 Processor -- start 2024-05-10T10:42:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 -- start start 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3f0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b19f0 con 0x5632cb3ef7a0 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b3530 con 0x5632cb3f4910 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03e8239740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b20c0 con 0x5632cb3eedb0 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49616/0 (socket says 172.21.15.112:49616) 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03d7fff640 1 -- 172.21.15.112:0/740393420 learned_addr learned my addr 172.21.15.112:0/740393420 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03df7fe640 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03deffd640 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03d7fff640 1 -- 172.21.15.112:0/740393420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 msgr2=0x5632cb3f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03d7fff640 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03d7fff640 1 -- 172.21.15.112:0/740393420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 msgr2=0x5632cb3f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.946+0000 7f03d7fff640 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.947+0000 7f03d7fff640 1 -- 172.21.15.112:0/740393420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632cb3f0470 con 0x5632cb3ef7a0 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.947+0000 7f03d7fff640 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f3ff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f03c8042f40 tx=0x7f03c8034430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8e574f17d65e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.947+0000 7f03d4ff9640 1 -- 172.21.15.112:0/740393420 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8052050 con 0x5632cb3ef7a0 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.947+0000 7f03d4ff9640 1 -- 172.21.15.112:0/740393420 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03c8053040 con 0x5632cb3ef7a0 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.947+0000 7f03d4ff9640 1 -- 172.21.15.112:0/740393420 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8046040 con 0x5632cb3ef7a0 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.947+0000 7f03e8239740 1 -- 172.21.15.112:0/740393420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 msgr2=0x5632cb3f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.947+0000 7f03e8239740 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f3ff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f03c8042f40 tx=0x7f03c8034430 comp rx=0 tx=0).stop 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- 172.21.15.112:0/740393420 shutdown_connections 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3f3ff0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 --2- 172.21.15.112:0/740393420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- 172.21.15.112:0/740393420 >> 172.21.15.112:0/740393420 conn(0x5632cb3ea420 msgr2=0x5632cb3e5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- 172.21.15.112:0/740393420 shutdown_connections 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- 172.21.15.112:0/740393420 wait complete. 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- start start 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 Processor -- start 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3fd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3fe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b2e60 con 0x5632cb3ef7a0 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b2790 con 0x5632cb3f4910 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01af9d0 con 0x5632cb3eedb0 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03e8239740 10 client.0 _ll_register_callbacks cb 0x7f03c8034b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3fd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49624/0 (socket says 172.21.15.112:49624) 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03d7fff640 1 -- 172.21.15.112:0/1964039664 learned_addr learned my addr 172.21.15.112:0/1964039664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03deffd640 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03df7fe640 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.948+0000 7f03d7fff640 1 -- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 msgr2=0x5632cb3fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03d7fff640 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03d7fff640 1 -- 172.21.15.112:0/1964039664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 msgr2=0x5632cb3fe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03d7fff640 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3fe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03d7fff640 1 -- 172.21.15.112:0/1964039664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632cb3fe830 con 0x5632cb3ef7a0 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03d7fff640 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3fd230 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f03c8034910 tx=0x7f03c802ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd821c00dc2151f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03c67fc640 1 -- 172.21.15.112:0/1964039664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8046030 con 0x5632cb3ef7a0 2024-05-10T10:42:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03c67fc640 1 -- 172.21.15.112:0/1964039664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f03c8004240 con 0x5632cb3ef7a0 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c8053040 con 0x5632cb3ef7a0 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03c67fc640 1 -- 172.21.15.112:0/1964039664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f03c8075050 con 0x5632cb3ef7a0 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 10 client.15888 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632cb3ffb40 con 0x5632cb3ef7a0 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03c67fc640 1 -- 172.21.15.112:0/1964039664 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03c8053040 con 0x5632cb3ef7a0 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03c67fc640 1 -- 172.21.15.112:0/1964039664 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f03c80424d0 con 0x5632cb3ef7a0 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 10 client.15888 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 20 client.15888 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb46f8e0 0x5632cb471d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 4 client.15888 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.949+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632cb35b050 con 0x5632cb46f8e0 2024-05-10T10:42:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.950+0000 7f03deffd640 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb46f8e0 0x5632cb471d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.950+0000 7f03deffd640 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb46f8e0 0x5632cb471d00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3455456043272860 server_cookie=1577865d5e4f0733 in_seq=0 out_seq=0 2024-05-10T10:42:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:03.950+0000 7f03c67fc640 10 client.15888 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03c67fc640 1 -- 172.21.15.112:0/1964039664 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632cb35b050 con 0x5632cb46f8e0 2024-05-10T10:42:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03c67fc640 10 client.15888 handle_command_reply: tid=0 2024-05-10T10:42:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 client.15888 shutdown 2024-05-10T10:42:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 msgr2=0x5632cb3fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3fd230 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f03c8034910 tx=0x7f03c802ec30 comp rx=0 tx=0).stop 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 shutdown_connections 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632cb46f8e0 0x5632cb471d00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03c67fc640 0 client.15888 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632cb3f4910 0x5632cb3fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632cb3ef7a0 0x5632cb3fd230 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 --2- 172.21.15.112:0/1964039664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632cb3eedb0 0x5632cb3fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 >> 172.21.15.112:0/1964039664 conn(0x5632cb3ea420 msgr2=0x5632cb3f0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 shutdown_connections 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 1 -- 172.21.15.112:0/1964039664 wait complete. 2024-05-10T10:42:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.241+0000 7f03e8239740 20 client.15888 trim_cache size 0 max 16384 2024-05-10T10:42:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:42:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:42:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:42:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f03cc0b28a0 msgr2=0x7f03cc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f03cc0b28a0 0x7f03cc0b4d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f03c802eaa0 tx=0x7f03c8005dc0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102f40 msgr2=0x7f03e01a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102f40 0x7f03e01a6690 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f03d800efc0 tx=0x7f03d800c800 comp rx=0 tx=0).stop 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 shutdown_connections 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f03cc0b28a0 0x7f03cc0b4d60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03e0102f40 0x7f03e01a6690 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e0102550 0x7f03e019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 --2- 172.21.15.112:0/2460195055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0101b90 0x7f03e019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 >> 172.21.15.112:0/2460195055 conn(0x7f03e00fc620 msgr2=0x7f03e0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 shutdown_connections 2024-05-10T10:42:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:04.242+0000 7f03e5445640 1 -- 172.21.15.112:0/2460195055 wait complete. 2024-05-10T10:42:04.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:42:04.585 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:04.984 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:04 smithi112 ceph-mon[30100]: pgmap v485: 129 pgs: 129 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 361 KiB/s wr, 12 op/s 2024-05-10T10:42:05.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:04 smithi148 ceph-mon[33568]: pgmap v485: 129 pgs: 129 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 361 KiB/s wr, 12 op/s 2024-05-10T10:42:05.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:04 smithi195 ceph-mon[33773]: pgmap v485: 129 pgs: 129 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 361 KiB/s wr, 12 op/s 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 -- 172.21.15.112:0/2145841132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 msgr2=0x7f64141024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 --2- 172.21.15.112:0/2145841132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f64141024a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6404009990 tx=0x7f6404009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 -- 172.21.15.112:0/2145841132 shutdown_connections 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 --2- 172.21.15.112:0/2145841132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6414106f60 0x7f641410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 --2- 172.21.15.112:0/2145841132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f64141024a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 --2- 172.21.15.112:0/2145841132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6414102a70 0x7f6414106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 -- 172.21.15.112:0/2145841132 >> 172.21.15.112:0/2145841132 conn(0x7f64140776c0 msgr2=0x7f6414077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 -- 172.21.15.112:0/2145841132 shutdown_connections 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 -- 172.21.15.112:0/2145841132 wait complete. 2024-05-10T10:42:06.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.962+0000 7f641893f700 1 Processor -- start 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641893f700 1 -- start start 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641893f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f641419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641893f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6414102a70 0x7f641419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641893f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6414106f60 0x7f64141976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641893f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64141113c0 con 0x7f64141020c0 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641893f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6414111240 con 0x7f6414102a70 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641893f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6414111540 con 0x7f6414106f60 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f641419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f641419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60980/0 (socket says 172.21.15.112:60980) 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 -- 172.21.15.112:0/2889955329 learned_addr learned my addr 172.21.15.112:0/2889955329 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f6412d9d700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6414106f60 0x7f64141976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6414102a70 0x7f641419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 -- 172.21.15.112:0/2889955329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6414106f60 msgr2=0x7f64141976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6414106f60 0x7f64141976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 -- 172.21.15.112:0/2889955329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6414102a70 msgr2=0x7f641419db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6414102a70 0x7f641419db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 -- 172.21.15.112:0/2889955329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6414197f80 con 0x7f64141020c0 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6414102a70 0x7f641419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f641259c700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f641419d5d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6404006010 tx=0x7f64040057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.963+0000 7f64037fe700 1 -- 172.21.15.112:0/2889955329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404027070 con 0x7f64141020c0 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.964+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6414198210 con 0x7f64141020c0 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.964+0000 7f64037fe700 1 -- 172.21.15.112:0/2889955329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f640400fb90 con 0x7f64141020c0 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.964+0000 7f64037fe700 1 -- 172.21.15.112:0/2889955329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040173f0 con 0x7f64141020c0 2024-05-10T10:42:06.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.964+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64141a4550 con 0x7f64141020c0 2024-05-10T10:42:06.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.967+0000 7f64037fe700 1 -- 172.21.15.112:0/2889955329 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f640402a070 con 0x7f64141020c0 2024-05-10T10:42:06.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.967+0000 7f64037fe700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63fc0b2880 0x7f63fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.967+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63fc0b2880 0x7f63fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64037fe700 1 -- 172.21.15.112:0/2889955329 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f64040e34c0 con 0x7f64141020c0 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 Processor -- start 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63fc0b2880 0x7f63fc0b4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6414198e60 tx=0x7f6408005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 -- start start 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f37100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecb2e84860 con 0x55ecb2f324d0 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ecb2e237c0 con 0x55ecb2f32df0 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f64231d2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecb2e80820 con 0x55ecb2f36c80 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f641259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f641259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f328b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60990/0 (socket says 172.21.15.112:60990) 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f641259c700 1 -- 172.21.15.112:0/480709130 learned_addr learned my addr 172.21.15.112:0/480709130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.968+0000 7f6411d9b700 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f6412d9d700 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f37100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f641259c700 1 -- 172.21.15.112:0/480709130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 msgr2=0x55ecb2f37100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f641259c700 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f37100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f641259c700 1 -- 172.21.15.112:0/480709130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 msgr2=0x55ecb2f366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f641259c700 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f641259c700 1 -- 172.21.15.112:0/480709130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb2dd18d0 con 0x55ecb2f324d0 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f6411d9b700 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f641259c700 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f328b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f64040aa770 tx=0x7f6404009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f399aaa30cc08820 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f6400ff9700 1 -- 172.21.15.112:0/480709130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040aa2f0 con 0x55ecb2f324d0 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f6400ff9700 1 -- 172.21.15.112:0/480709130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64040aaee0 con 0x55ecb2f324d0 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.969+0000 7f6400ff9700 1 -- 172.21.15.112:0/480709130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040aaa40 con 0x55ecb2f324d0 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 -- 172.21.15.112:0/480709130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 msgr2=0x55ecb2f328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f328b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f64040aa770 tx=0x7f6404009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 -- 172.21.15.112:0/480709130 shutdown_connections 2024-05-10T10:42:06.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f37100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f328b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 --2- 172.21.15.112:0/480709130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 -- 172.21.15.112:0/480709130 >> 172.21.15.112:0/480709130 conn(0x55ecb2e5e100 msgr2=0x55ecb2f2cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 -- 172.21.15.112:0/480709130 shutdown_connections 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 -- 172.21.15.112:0/480709130 wait complete. 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 -- start start 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.970+0000 7f64231d2b80 1 Processor -- start 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f3b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f3b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecb2e84860 con 0x55ecb2f324d0 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ecb2deac30 con 0x55ecb2f32df0 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecb2f3ec50 con 0x55ecb2f36c80 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 10 client.0 _ll_register_callbacks cb 0x55ecb2e1c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:06.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f3e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32774/0 (socket says 172.21.15.112:32774) 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 -- 172.21.15.112:0/2743877456 learned_addr learned my addr 172.21.15.112:0/2743877456 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f3b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f6412d9d700 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f3b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 -- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 msgr2=0x55ecb2f3b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f3b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 -- 172.21.15.112:0/2743877456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 msgr2=0x55ecb2f3b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f3b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 -- 172.21.15.112:0/2743877456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb2e20690 con 0x55ecb2f324d0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f641259c700 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f3e0e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f64040058d0 tx=0x7f64040a6050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8290e244e1acc6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f63f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2743877456 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040aa2f0 con 0x55ecb2f324d0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2743877456 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f64040b3330 con 0x55ecb2f324d0 2024-05-10T10:42:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.971+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64040aaee0 con 0x55ecb2f324d0 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2743877456 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64040aa7e0 con 0x55ecb2f324d0 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 10 client.15900 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ecb2f49280 con 0x55ecb2f324d0 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2743877456 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64040aaee0 con 0x55ecb2f324d0 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2743877456 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f64040050d0 con 0x55ecb2f324d0 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 10 client.15900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 20 client.15900 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2f4db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 4 client.15900 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb2efe650 con 0x55ecb2f4b750 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.972+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2f4db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.973+0000 7f63f9ffb700 10 client.15900 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:06.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.973+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2f4db40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41946e806974badd server_cookie=500697270141fef5 in_seq=0 out_seq=0 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2743877456 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ecb2efe650 con 0x55ecb2f4b750 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f63f9ffb700 10 client.15900 handle_command_reply: tid=0 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 client.15900 shutdown 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 msgr2=0x55ecb2f3e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f3e0e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f64040058d0 tx=0x7f64040a6050 comp rx=0 tx=0).stop 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 shutdown_connections 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2f4db40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f3b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f63f9ffb700 0 client.15900 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f324d0 0x55ecb2f3e0e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2743877456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f32df0 0x55ecb2f3b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 >> 172.21.15.112:0/2743877456 conn(0x55ecb2e5e100 msgr2=0x55ecb2f27720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 shutdown_connections 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 1 -- 172.21.15.112:0/2743877456 wait complete. 2024-05-10T10:42:06.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.974+0000 7f64231d2b80 20 client.15900 trim_cache size 0 max 16384 2024-05-10T10:42:06.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 Processor -- start 2024-05-10T10:42:06.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 -- start start 2024-05-10T10:42:06.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:06 smithi112 ceph-mon[30100]: pgmap v486: 129 pgs: 129 active+clean; 12 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 969 KiB/s wr, 81 op/s 2024-05-10T10:42:06.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f324d0 0x55ecb2f47b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2f53390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64141acf30 con 0x55ecb2f32df0 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64141aea70 con 0x55ecb2f324d0 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f64231d2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64141ad600 con 0x55ecb2f36c80 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f6411d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2f53390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f6411d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2f53390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32786/0 (socket says 172.21.15.112:32786) 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f6412d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f6411d9b700 1 -- 172.21.15.112:0/4016842988 learned_addr learned my addr 172.21.15.112:0/4016842988 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.990+0000 7f641259c700 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f324d0 0x55ecb2f47b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f6412d9d700 1 -- 172.21.15.112:0/4016842988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f324d0 msgr2=0x55ecb2f47b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f6412d9d700 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f324d0 0x55ecb2f47b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f6412d9d700 1 -- 172.21.15.112:0/4016842988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 msgr2=0x55ecb2f53390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:06.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f6412d9d700 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2f53390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f6412d9d700 1 -- 172.21.15.112:0/4016842988 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb2f399f0 con 0x55ecb2f36c80 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f641259c700 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f324d0 0x55ecb2f47b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f6411d9b700 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2f53390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f6412d9d700 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f538d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f640c00a090 tx=0x7f640c00aa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c712c01428710 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f63fbfff700 1 -- 172.21.15.112:0/4016842988 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c01c030 con 0x55ecb2f36c80 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f63fbfff700 1 -- 172.21.15.112:0/4016842988 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f640c004000 con 0x55ecb2f36c80 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.991+0000 7f63fbfff700 1 -- 172.21.15.112:0/4016842988 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0186a0 con 0x55ecb2f36c80 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 -- 172.21.15.112:0/4016842988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 msgr2=0x55ecb2f538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f538d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f640c00a090 tx=0x7f640c00aa30 comp rx=0 tx=0).stop 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 -- 172.21.15.112:0/4016842988 shutdown_connections 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f36c80 0x55ecb2f538d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f324d0 0x55ecb2f47b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 --2- 172.21.15.112:0/4016842988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2f53390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 -- 172.21.15.112:0/4016842988 >> 172.21.15.112:0/4016842988 conn(0x55ecb2e5e100 msgr2=0x55ecb2f27720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:06.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 -- 172.21.15.112:0/4016842988 shutdown_connections 2024-05-10T10:42:06.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 -- 172.21.15.112:0/4016842988 wait complete. 2024-05-10T10:42:06.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 -- start start 2024-05-10T10:42:06.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.992+0000 7f64231d2b80 1 Processor -- start 2024-05-10T10:42:06.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f324d0 0x55ecb2f441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2fb0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f64231d2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f36c80 0x55ecb2fb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f64231d2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64141ae3a0 con 0x55ecb2f32df0 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f64231d2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64141adcd0 con 0x55ecb2f36c80 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f64231d2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64141aaed0 con 0x55ecb2f324d0 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f64231d2b80 10 client.0 _ll_register_callbacks cb 0x55ecb2e1c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2fb0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2fb0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32788/0 (socket says 172.21.15.112:32788) 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 -- 172.21.15.112:0/2374757301 learned_addr learned my addr 172.21.15.112:0/2374757301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f641259c700 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f324d0 0x55ecb2f441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 -- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f324d0 msgr2=0x55ecb2f441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f324d0 0x55ecb2f441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 -- 172.21.15.112:0/2374757301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f36c80 msgr2=0x55ecb2fb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f36c80 0x55ecb2fb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 -- 172.21.15.112:0/2374757301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb2fb19c0 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f6411d9b700 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2fb0ec0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f640800a540 tx=0x7f6408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11332eca29e4f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.993+0000 7f63f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2374757301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408010b20 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2374757301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6408010cc0 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c004000 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2374757301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6408004830 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 10 client.15906 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ecb2fba350 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2374757301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f640c004000 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2374757301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6408031070 con 0x55ecb2f32df0 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:07.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 10 client.15906 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 20 client.15906 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2fbc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 4 client.15906 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.994+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecb2e92240 con 0x55ecb2f4b750 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f641259c700 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2fbc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f641259c700 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2fbc9c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4824cb43aea79c9c server_cookie=2dec749bc605f9da in_seq=0 out_seq=0 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f63f9ffb700 10 client.15906 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f63f9ffb700 1 -- 172.21.15.112:0/2374757301 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecb2e92240 con 0x55ecb2f4b750 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f63f9ffb700 10 client.15906 handle_command_reply: tid=0 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 client.15906 shutdown 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 msgr2=0x55ecb2fb0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2fb0ec0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f640800a540 tx=0x7f6408004750 comp rx=0 tx=0).stop 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 shutdown_connections 2024-05-10T10:42:07.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ecb2f4b750 0x55ecb2fbc9c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecb2f36c80 0x55ecb2fb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecb2f324d0 0x55ecb2f441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 --2- 172.21.15.112:0/2374757301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb2f32df0 0x55ecb2fb0ec0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 >> 172.21.15.112:0/2374757301 conn(0x55ecb2e5e100 msgr2=0x55ecb2f312e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f63f9ffb700 0 client.15906 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.995+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 shutdown_connections 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f64231d2b80 1 -- 172.21.15.112:0/2374757301 wait complete. 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f64231d2b80 20 client.15906 trim_cache size 0 max 16384 2024-05-10T10:42:07.002 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:07.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63fc0b2880 msgr2=0x7f63fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:07.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63fc0b2880 0x7f63fc0b4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6414198e60 tx=0x7f6408005f50 comp rx=0 tx=0).stop 2024-05-10T10:42:07.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 msgr2=0x7f641419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:07.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f641419d5d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6404006010 tx=0x7f64040057c0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 shutdown_connections 2024-05-10T10:42:07.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63fc0b2880 0x7f63fc0b4d40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6414106f60 0x7f64141976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64141020c0 0x7f641419d5d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 --2- 172.21.15.112:0/2889955329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6414102a70 0x7f641419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:07.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 >> 172.21.15.112:0/2889955329 conn(0x7f64140776c0 msgr2=0x7f6414100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:07.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 shutdown_connections 2024-05-10T10:42:07.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:06.996+0000 7f641893f700 1 -- 172.21.15.112:0/2889955329 wait complete. 2024-05-10T10:42:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:06 smithi148 ceph-mon[33568]: pgmap v486: 129 pgs: 129 active+clean; 12 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 969 KiB/s wr, 81 op/s 2024-05-10T10:42:07.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:06 smithi195 ceph-mon[33773]: pgmap v486: 129 pgs: 129 active+clean; 12 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 969 KiB/s wr, 81 op/s 2024-05-10T10:42:07.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:42:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:09.203 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:08 smithi112 ceph-mon[30100]: pgmap v487: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-05-10T10:42:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:08 smithi148 ceph-mon[33568]: pgmap v487: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-05-10T10:42:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:08 smithi195 ceph-mon[33773]: pgmap v487: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-05-10T10:42:09.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 -- 172.21.15.112:0/3879382193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc1023a0 msgr2=0x7f06cc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 --2- 172.21.15.112:0/3879382193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc1023a0 0x7f06cc102820 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009990 tx=0x7f06c0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 -- 172.21.15.112:0/3879382193 shutdown_connections 2024-05-10T10:42:09.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 --2- 172.21.15.112:0/3879382193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06cc102d60 0x7f06cc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 --2- 172.21.15.112:0/3879382193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc108410 0x7f06cc1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 --2- 172.21.15.112:0/3879382193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc1023a0 0x7f06cc102820 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 -- 172.21.15.112:0/3879382193 >> 172.21.15.112:0/3879382193 conn(0x7f06cc079350 msgr2=0x7f06cc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:09.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 -- 172.21.15.112:0/3879382193 shutdown_connections 2024-05-10T10:42:09.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.827+0000 7f06d1101700 1 -- 172.21.15.112:0/3879382193 wait complete. 2024-05-10T10:42:09.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 Processor -- start 2024-05-10T10:42:09.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 -- start start 2024-05-10T10:42:09.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc1023a0 0x7f06cc104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06cc102d60 0x7f06cc1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc108410 0x7f06cc104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc112400 con 0x7f06cc108410 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc112280 con 0x7f06cc102d60 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06d1101700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc112580 con 0x7f06cc1023a0 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc1023a0 0x7f06cc104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06cb59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc108410 0x7f06cc104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc1023a0 0x7f06cc104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49632/0 (socket says 172.21.15.112:49632) 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06cb59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc108410 0x7f06cc104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32814/0 (socket says 172.21.15.112:32814) 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06ca59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06cc102d60 0x7f06cc1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.828+0000 7f06cad9d700 1 -- 172.21.15.112:0/3800518690 learned_addr learned my addr 172.21.15.112:0/3800518690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06cad9d700 1 -- 172.21.15.112:0/3800518690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06cc102d60 msgr2=0x7f06cc1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06cad9d700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06cc102d60 0x7f06cc1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06cad9d700 1 -- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc108410 msgr2=0x7f06cc104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06cad9d700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc108410 0x7f06cc104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06cad9d700 1 -- 172.21.15.112:0/3800518690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc1a7ff0 con 0x7f06cc1023a0 2024-05-10T10:42:09.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06ca59c700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06cc102d60 0x7f06cc1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06cb59e700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc108410 0x7f06cc104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06cad9d700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc1023a0 0x7f06cc104080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00e9c0 tx=0x7f06bc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06b3fff700 1 -- 172.21.15.112:0/3800518690 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00cbd0 con 0x7f06cc1023a0 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1a82e0 con 0x7f06cc1023a0 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06b3fff700 1 -- 172.21.15.112:0/3800518690 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06bc00ee60 con 0x7f06cc1023a0 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1a87f0 con 0x7f06cc1023a0 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.829+0000 7f06b3fff700 1 -- 172.21.15.112:0/3800518690 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0054b0 con 0x7f06cc1023a0 2024-05-10T10:42:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.832+0000 7f06b3fff700 1 -- 172.21.15.112:0/3800518690 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f06bc021440 con 0x7f06cc1023a0 2024-05-10T10:42:09.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06b3fff700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f06b40b2880 0x7f06b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 Processor -- start 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 -- start start 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06ca59c700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f06b40b2880 0x7f06b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06b3fff700 1 -- 172.21.15.112:0/3800518690 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f06bc0e27f0 con 0x7f06cc1023a0 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e38810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e39190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b080d8a940 con 0x55b080e3e1b0 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b080cefda0 con 0x55b080e38d50 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.833+0000 7f06db994b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b080d86900 con 0x55b080e3ead0 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06ca59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e38810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06ca59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e38810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49648/0 (socket says 172.21.15.112:49648) 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e3e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32826/0 (socket says 172.21.15.112:32826) 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06ca59c700 1 -- 172.21.15.112:0/908852851 learned_addr learned my addr 172.21.15.112:0/908852851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cb59e700 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e39190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 -- 172.21.15.112:0/908852851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 msgr2=0x55b080e38810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e38810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 -- 172.21.15.112:0/908852851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 msgr2=0x55b080e39190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e39190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06ca59c700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f06b40b2880 0x7f06b40b4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f06cc105b10 tx=0x7f06c001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 -- 172.21.15.112:0/908852851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b080cd78d0 con 0x55b080e3e1b0 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06cad9d700 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e3e590 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f06bc005020 tx=0x7f06bc005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c582afaa77ae61df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06b17fa700 1 -- 172.21.15.112:0/908852851 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0b21d0 con 0x55b080e3e1b0 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06b17fa700 1 -- 172.21.15.112:0/908852851 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06bc0aa120 con 0x55b080e3e1b0 2024-05-10T10:42:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.834+0000 7f06b17fa700 1 -- 172.21.15.112:0/908852851 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0a9ee0 con 0x55b080e3e1b0 2024-05-10T10:42:09.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 -- 172.21.15.112:0/908852851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 msgr2=0x55b080e3e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e3e590 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f06bc005020 tx=0x7f06bc005870 comp rx=0 tx=0).stop 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 -- 172.21.15.112:0/908852851 shutdown_connections 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e39190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e3e590 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 --2- 172.21.15.112:0/908852851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e38810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 -- 172.21.15.112:0/908852851 >> 172.21.15.112:0/908852851 conn(0x55b080d64b70 msgr2=0x55b080e332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 -- 172.21.15.112:0/908852851 shutdown_connections 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 -- 172.21.15.112:0/908852851 wait complete. 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.835+0000 7f06db994b80 1 -- start start 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 1 Processor -- start 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e38d50 0x55b080e44710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e3e1b0 0x55b080e41ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e41ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b080e45450 con 0x55b080e38d50 2024-05-10T10:42:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b080d43ed0 con 0x55b080e3e1b0 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b080e455d0 con 0x55b080e3ead0 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e38d50 0x55b080e44710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06db994b80 10 client.0 _ll_register_callbacks cb 0x55b080d1e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e38d50 0x55b080e44710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32830/0 (socket says 172.21.15.112:32830) 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 -- 172.21.15.112:0/87861215 learned_addr learned my addr 172.21.15.112:0/87861215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cb59e700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e41ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06ca59c700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e3e1b0 0x55b080e41ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 -- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 msgr2=0x55b080e41ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e41ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 -- 172.21.15.112:0/87861215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e3e1b0 msgr2=0x55b080e41ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e3e1b0 0x55b080e41ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 -- 172.21.15.112:0/87861215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b080d41760 con 0x55b080e38d50 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06ca59c700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e3e1b0 0x55b080e41ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06cad9d700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e38d50 0x55b080e44710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0a9720 tx=0x7f06bc005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc982b2d82182601 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.836+0000 7f06aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06aeff5700 1 -- 172.21.15.112:0/87861215 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc06e060 con 0x55b080e38d50 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06aeff5700 1 -- 172.21.15.112:0/87861215 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f06bc0a6010 con 0x55b080e38d50 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06bc0aa120 con 0x55b080e38d50 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06aeff5700 1 -- 172.21.15.112:0/87861215 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06bc0c2030 con 0x55b080e38d50 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 10 client.15918 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b080e45450 con 0x55b080e38d50 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06aeff5700 1 -- 172.21.15.112:0/87861215 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06bc0aa120 con 0x55b080e38d50 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06aeff5700 1 -- 172.21.15.112:0/87861215 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f06bc0b23f0 con 0x55b080e38d50 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 20 client.15918 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080e52000 0x55b080e543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 4 client.15918 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b080d98320 con 0x55b080e52000 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.837+0000 7f06ca59c700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080e52000 0x55b080e543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.838+0000 7f06ca59c700 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080e52000 0x55b080e543f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b27bff310da85e server_cookie=3e3a0c3ab10a7d2f in_seq=0 out_seq=0 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.838+0000 7f06aeff5700 10 client.15918 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.839+0000 7f06aeff5700 1 -- 172.21.15.112:0/87861215 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b080d98320 con 0x55b080e52000 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.839+0000 7f06aeff5700 10 client.15918 handle_command_reply: tid=0 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 client.15918 shutdown 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e38d50 msgr2=0x55b080e44710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e38d50 0x55b080e44710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0a9720 tx=0x7f06bc005730 comp rx=0 tx=0).stop 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 shutdown_connections 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080e52000 0x55b080e543f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e41ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e38d50 0x55b080e44710 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 --2- 172.21.15.112:0/87861215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e3e1b0 0x55b080e41ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 >> 172.21.15.112:0/87861215 conn(0x55b080d64b70 msgr2=0x55b080e2db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06aeff5700 0 client.15918 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 shutdown_connections 2024-05-10T10:42:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 1 -- 172.21.15.112:0/87861215 wait complete. 2024-05-10T10:42:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.840+0000 7f06db994b80 20 client.15918 trim_cache size 0 max 16384 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 Processor -- start 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 -- start start 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e4e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e2dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e3d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06b800eac0 con 0x55b080e3e1b0 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06b8010600 con 0x55b080e38d50 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.856+0000 7f06db994b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06b800f190 con 0x55b080e3ead0 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e3d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e3d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49658/0 (socket says 172.21.15.112:49658) 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e4e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 -- 172.21.15.112:0/2340859350 learned_addr learned my addr 172.21.15.112:0/2340859350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06ca59c700 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e2dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 -- 172.21.15.112:0/2340859350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 msgr2=0x55b080e4e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e4e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 -- 172.21.15.112:0/2340859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 msgr2=0x55b080e2dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e2dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 -- 172.21.15.112:0/2340859350 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b080e40020 con 0x55b080e3ead0 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06ca59c700 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e2dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06cb59e700 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e3d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06c4003a30 tx=0x7f06c4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d3b0546a6ff7e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06b0ff9700 1 -- 172.21.15.112:0/2340859350 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c401b070 con 0x55b080e3ead0 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06b0ff9700 1 -- 172.21.15.112:0/2340859350 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06c4007500 con 0x55b080e3ead0 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.857+0000 7f06b0ff9700 1 -- 172.21.15.112:0/2340859350 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c400e710 con 0x55b080e3ead0 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 -- 172.21.15.112:0/2340859350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 msgr2=0x55b080e3d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e3d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06c4003a30 tx=0x7f06c4009920 comp rx=0 tx=0).stop 2024-05-10T10:42:09.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 -- 172.21.15.112:0/2340859350 shutdown_connections 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3ead0 0x55b080e3d4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080e4e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 --2- 172.21.15.112:0/2340859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3e1b0 0x55b080e2dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 -- 172.21.15.112:0/2340859350 >> 172.21.15.112:0/2340859350 conn(0x55b080d64b70 msgr2=0x55b080e3f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 -- 172.21.15.112:0/2340859350 shutdown_connections 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.858+0000 7f06db994b80 1 -- 172.21.15.112:0/2340859350 wait complete. 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 -- start start 2024-05-10T10:42:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 Processor -- start 2024-05-10T10:42:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080eb4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3e1b0 0x55b080eb4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3ead0 0x55b080e47e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06b800ff30 con 0x55b080e3ead0 2024-05-10T10:42:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06b800f860 con 0x55b080e38d50 2024-05-10T10:42:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06b800ca60 con 0x55b080e3e1b0 2024-05-10T10:42:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06db994b80 10 client.0 _ll_register_callbacks cb 0x55b080d1e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3ead0 0x55b080e47e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3ead0 0x55b080e47e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32840/0 (socket says 172.21.15.112:32840) 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06ca59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3e1b0 0x55b080eb4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 -- 172.21.15.112:0/2320933330 learned_addr learned my addr 172.21.15.112:0/2320933330 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cad9d700 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080eb4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 -- 172.21.15.112:0/2320933330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3e1b0 msgr2=0x55b080eb4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3e1b0 0x55b080eb4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 -- 172.21.15.112:0/2320933330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 msgr2=0x55b080eb4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080eb4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.859+0000 7f06cb59e700 1 -- 172.21.15.112:0/2320933330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b080e48580 con 0x55b080e3ead0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06cb59e700 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3ead0 0x55b080e47e40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009c10 tx=0x7f06c4007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ac4c16e2d397b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06aeff5700 1 -- 172.21.15.112:0/2320933330 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c401b040 con 0x55b080e3ead0 2024-05-10T10:42:09.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06aeff5700 1 -- 172.21.15.112:0/2320933330 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f06c4004720 con 0x55b080e3ead0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c4007500 con 0x55b080e3ead0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06aeff5700 1 -- 172.21.15.112:0/2320933330 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06c40091c0 con 0x55b080e3ead0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 10 client.15924 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b080ebc190 con 0x55b080e3ead0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06aeff5700 1 -- 172.21.15.112:0/2320933330 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06c4007500 con 0x55b080e3ead0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06aeff5700 1 -- 172.21.15.112:0/2320933330 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f06c4005040 con 0x55b080e3ead0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 20 client.15924 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080ebc7b0 0x55b080ebebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 4 client.15924 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.860+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b080d923c0 con 0x55b080ebc7b0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.861+0000 7f06cad9d700 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080ebc7b0 0x55b080ebebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.861+0000 7f06cad9d700 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080ebc7b0 0x55b080ebebd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9077d483162afe server_cookie=dc1c66e228eba9 in_seq=0 out_seq=0 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.861+0000 7f06aeff5700 10 client.15924 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06aeff5700 1 -- 172.21.15.112:0/2320933330 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b080d923c0 con 0x55b080ebc7b0 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06aeff5700 10 client.15924 handle_command_reply: tid=0 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 client.15924 shutdown 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3ead0 msgr2=0x55b080e47e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3ead0 0x55b080e47e40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009c10 tx=0x7f06c4007960 comp rx=0 tx=0).stop 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 shutdown_connections 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b080ebc7b0 0x55b080ebebd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b080e3ead0 0x55b080e47e40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b080e38d50 0x55b080eb4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 --2- 172.21.15.112:0/2320933330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b080e3e1b0 0x55b080eb4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 >> 172.21.15.112:0/2320933330 conn(0x55b080d64b70 msgr2=0x55b080e438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06aeff5700 0 client.15924 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:09.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 shutdown_connections 2024-05-10T10:42:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 1 -- 172.21.15.112:0/2320933330 wait complete. 2024-05-10T10:42:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.862+0000 7f06db994b80 20 client.15924 trim_cache size 0 max 16384 2024-05-10T10:42:09.869 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f06b40b2880 msgr2=0x7f06b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f06b40b2880 0x7f06b40b4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f06cc105b10 tx=0x7f06c001a040 comp rx=0 tx=0).stop 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc1023a0 msgr2=0x7f06cc104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc1023a0 0x7f06cc104080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00e9c0 tx=0x7f06bc00ecd0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 shutdown_connections 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06cc108410 0x7f06cc104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f06b40b2880 0x7f06b40b4d40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06cc1023a0 0x7f06cc104080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 --2- 172.21.15.112:0/3800518690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06cc102d60 0x7f06cc1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 >> 172.21.15.112:0/3800518690 conn(0x7f06cc079350 msgr2=0x7f06cc06b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 shutdown_connections 2024-05-10T10:42:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:09.863+0000 7f06d1101700 1 -- 172.21.15.112:0/3800518690 wait complete. 2024-05-10T10:42:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:10 smithi148 ceph-mon[33568]: pgmap v488: 129 pgs: 129 active+clean; 19 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-05-10T10:42:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:10 smithi195 ceph-mon[33773]: pgmap v488: 129 pgs: 129 active+clean; 19 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-05-10T10:42:11.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:10 smithi112 ceph-mon[30100]: pgmap v488: 129 pgs: 129 active+clean; 19 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-05-10T10:42:11.904 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:42:11.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:42:12.208 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:13.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:12 smithi112 ceph-mon[30100]: pgmap v489: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-05-10T10:42:13.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:12 smithi148 ceph-mon[33568]: pgmap v489: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-05-10T10:42:13.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:12 smithi195 ceph-mon[33773]: pgmap v489: 129 pgs: 129 active+clean; 19 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-05-10T10:42:14.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.279+0000 7f2613e2a700 1 -- 172.21.15.112:0/2542534035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c107760 msgr2=0x7f260c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.279+0000 7f2613e2a700 1 --2- 172.21.15.112:0/2542534035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c107760 0x7f260c10f2e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2604009f90 tx=0x7f260400b330 comp rx=0 tx=0).stop 2024-05-10T10:42:14.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 -- 172.21.15.112:0/2542534035 shutdown_connections 2024-05-10T10:42:14.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 --2- 172.21.15.112:0/2542534035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c107760 0x7f260c10f2e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 --2- 172.21.15.112:0/2542534035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c102e00 0x7f260c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 --2- 172.21.15.112:0/2542534035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c1037b0 0x7f260c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 -- 172.21.15.112:0/2542534035 >> 172.21.15.112:0/2542534035 conn(0x7f260c078a10 msgr2=0x7f260c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:14.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 -- 172.21.15.112:0/2542534035 shutdown_connections 2024-05-10T10:42:14.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 -- 172.21.15.112:0/2542534035 wait complete. 2024-05-10T10:42:14.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.280+0000 7f2613e2a700 1 Processor -- start 2024-05-10T10:42:14.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2613e2a700 1 -- start start 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2613e2a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c102e00 0x7f260c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2613e2a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c1037b0 0x7f260c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2613e2a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c107760 0x7f260c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2613e2a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1123e0 con 0x7f260c102e00 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f26123c7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c107760 0x7f260c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f26123c7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c107760 0x7f260c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:46924/0 (socket says 172.21.15.112:46924) 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f26123c7700 1 -- 172.21.15.112:0/899313825 learned_addr learned my addr 172.21.15.112:0/899313825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f26113c5700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c1037b0 0x7f260c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f260c112260 con 0x7f260c107760 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f260c112560 con 0x7f260c1037b0 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.281+0000 7f2611bc6700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c102e00 0x7f260c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f26123c7700 1 -- 172.21.15.112:0/899313825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c1037b0 msgr2=0x7f260c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f26123c7700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c1037b0 0x7f260c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f26123c7700 1 -- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c102e00 msgr2=0x7f260c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f26123c7700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c102e00 0x7f260c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f26123c7700 1 -- 172.21.15.112:0/899313825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c199f70 con 0x7f260c107760 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f26123c7700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c107760 0x7f260c199790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2604009790 tx=0x7f260400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f2602ffd700 1 -- 172.21.15.112:0/899313825 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604021070 con 0x7f260c107760 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c19a200 con 0x7f260c107760 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c1a6470 con 0x7f260c107760 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f2602ffd700 1 -- 172.21.15.112:0/899313825 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f260400ce50 con 0x7f260c107760 2024-05-10T10:42:14.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.282+0000 7f2602ffd700 1 -- 172.21.15.112:0/899313825 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604005040 con 0x7f260c107760 2024-05-10T10:42:14.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.285+0000 7f2602ffd700 1 -- 172.21.15.112:0/899313825 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2604024070 con 0x7f260c107760 2024-05-10T10:42:14.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.286+0000 7f2602ffd700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25f80b2880 0x7f25f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.286+0000 7f2602ffd700 1 -- 172.21.15.112:0/899313825 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f260400f020 con 0x7f260c107760 2024-05-10T10:42:14.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.286+0000 7f2611bc6700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25f80b2880 0x7f25f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.287+0000 7f2611bc6700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25f80b2880 0x7f25f80b4d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2608009cb0 tx=0x7f2608009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:14.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.288+0000 7f261e6bdb80 1 Processor -- start 2024-05-10T10:42:14.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.288+0000 7f261e6bdb80 1 -- start start 2024-05-10T10:42:14.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305be2c0 0x55ac305be6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1370 0x55ac305b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac305038e0 con 0x55ac305b1370 2024-05-10T10:42:14.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ac304a2840 con 0x55ac305b1d30 2024-05-10T10:42:14.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac304ff8a0 con 0x55ac305be2c0 2024-05-10T10:42:14.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1370 0x55ac305b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1370 0x55ac305b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32872/0 (socket says 172.21.15.112:32872) 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 -- 172.21.15.112:0/2833153677 learned_addr learned my addr 172.21.15.112:0/2833153677 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26123c7700 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f2611bc6700 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305be2c0 0x55ac305be6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 -- 172.21.15.112:0/2833153677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305be2c0 msgr2=0x55ac305be6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305be2c0 0x55ac305be6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 -- 172.21.15.112:0/2833153677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 msgr2=0x55ac305c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f26113c5700 1 -- 172.21.15.112:0/2833153677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac30450950 con 0x55ac305b1370 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.289+0000 7f2611bc6700 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305be2c0 0x55ac305be6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.290+0000 7f26113c5700 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1370 0x55ac305b17f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ea0 tx=0x7f25fc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9e3a60099d52e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.290+0000 7f25f7fff700 1 -- 172.21.15.112:0/2833153677 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc004310 con 0x55ac305b1370 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.290+0000 7f25f7fff700 1 -- 172.21.15.112:0/2833153677 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25fc0044b0 con 0x55ac305b1370 2024-05-10T10:42:14.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.290+0000 7f25f7fff700 1 -- 172.21.15.112:0/2833153677 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0197c0 con 0x55ac305b1370 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2833153677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1370 msgr2=0x55ac305b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1370 0x55ac305b17f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ea0 tx=0x7f25fc011720 comp rx=0 tx=0).stop 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2833153677 shutdown_connections 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305be2c0 0x55ac305be6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2833153677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1370 0x55ac305b17f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2833153677 >> 172.21.15.112:0/2833153677 conn(0x55ac304dd180 msgr2=0x55ac305abcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2833153677 shutdown_connections 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2833153677 wait complete. 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.291+0000 7f261e6bdb80 1 -- start start 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.292+0000 7f261e6bdb80 1 Processor -- start 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.292+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.292+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1d30 0x55ac305c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.292+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305be2c0 0x55ac305c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.292+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac305c6770 con 0x55ac305b1d30 2024-05-10T10:42:14.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f26113c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1d30 0x55ac305c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ac30469cb0 con 0x55ac305be2c0 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f26123c7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305be2c0 0x55ac305c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49682/0 (socket says 172.21.15.112:49682) 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 -- 172.21.15.112:0/3863279237 learned_addr learned my addr 172.21.15.112:0/3863279237 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac305c68f0 con 0x55ac305b1370 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f261e6bdb80 10 client.0 _ll_register_callbacks cb 0x55ac3049b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 -- 172.21.15.112:0/3863279237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305be2c0 msgr2=0x55ac305c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305be2c0 0x55ac305c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 -- 172.21.15.112:0/3863279237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1d30 msgr2=0x55ac305c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1d30 0x55ac305c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.293+0000 7f2611bc6700 1 -- 172.21.15.112:0/3863279237 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3049f710 con 0x55ac305b1370 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.294+0000 7f2611bc6700 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c5d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f260801dc10 tx=0x7f26080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669018ef626b3388 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:14.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.294+0000 7f25f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:42:14.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.294+0000 7f25f57fa700 1 -- 172.21.15.112:0/3863279237 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260801d220 con 0x55ac305b1370 2024-05-10T10:42:14.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.294+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc0044b0 con 0x55ac305b1370 2024-05-10T10:42:14.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.294+0000 7f25f57fa700 1 -- 172.21.15.112:0/3863279237 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f260800ea50 con 0x55ac305b1370 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.294+0000 7f25f57fa700 1 -- 172.21.15.112:0/3863279237 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2608030050 con 0x55ac305b1370 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 10 client.25387 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac305d0f30 con 0x55ac305b1370 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f25f57fa700 1 -- 172.21.15.112:0/3863279237 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25fc0044b0 con 0x55ac305b1370 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f25f57fa700 1 -- 172.21.15.112:0/3863279237 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2608024030 con 0x55ac305b1370 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 10 client.25387 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 10 client.25387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:14.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 20 client.25387 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac305d3510 0x55ac305d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 4 client.25387 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac3057d6d0 con 0x55ac305d3510 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.295+0000 7f26113c5700 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac305d3510 0x55ac305d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.296+0000 7f25f57fa700 10 client.25387 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.296+0000 7f26113c5700 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac305d3510 0x55ac305d59d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff7ac451ac1dca2 server_cookie=8bf0987191908065 in_seq=0 out_seq=0 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.297+0000 7f25f57fa700 1 -- 172.21.15.112:0/3863279237 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac3057d6d0 con 0x55ac305d3510 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.297+0000 7f25f57fa700 10 client.25387 handle_command_reply: tid=0 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.297+0000 7f261e6bdb80 1 client.25387 shutdown 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.297+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 msgr2=0x55ac305c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.297+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c5d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f260801dc10 tx=0x7f26080047c0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 shutdown_connections 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac305d3510 0x55ac305d59d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305be2c0 0x55ac305c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c5d30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3863279237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305b1d30 0x55ac305c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f25f57fa700 0 client.25387 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 >> 172.21.15.112:0/3863279237 conn(0x55ac304dd180 msgr2=0x55ac305a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 shutdown_connections 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3863279237 wait complete. 2024-05-10T10:42:14.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.298+0000 7f261e6bdb80 20 client.25387 trim_cache size 0 max 16384 2024-05-10T10:42:14.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.324+0000 7f261e6bdb80 1 Processor -- start 2024-05-10T10:42:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.324+0000 7f261e6bdb80 1 -- start start 2024-05-10T10:42:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305db7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25f000eb00 con 0x55ac305be2c0 2024-05-10T10:42:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0010640 con 0x55ac305b1d30 2024-05-10T10:42:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f25f000f1d0 con 0x55ac305b1370 2024-05-10T10:42:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305db7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305db7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32888/0 (socket says 172.21.15.112:32888) 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f2611bc6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 -- 172.21.15.112:0/2820783938 learned_addr learned my addr 172.21.15.112:0/2820783938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26113c5700 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 -- 172.21.15.112:0/2820783938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 msgr2=0x55ac305c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 -- 172.21.15.112:0/2820783938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 msgr2=0x55ac305db2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305db2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.325+0000 7f26123c7700 1 -- 172.21.15.112:0/2820783938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac305a8890 con 0x55ac305be2c0 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.326+0000 7f26123c7700 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305db7e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2604004fd0 tx=0x7f260400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca360e301cedbb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.326+0000 7f25f77fe700 1 -- 172.21.15.112:0/2820783938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26040cbe60 con 0x55ac305be2c0 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.326+0000 7f25f77fe700 1 -- 172.21.15.112:0/2820783938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26040b1040 con 0x55ac305be2c0 2024-05-10T10:42:14.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.326+0000 7f25f77fe700 1 -- 172.21.15.112:0/2820783938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604007ac0 con 0x55ac305be2c0 2024-05-10T10:42:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2820783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 msgr2=0x55ac305db7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305db7e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2604004fd0 tx=0x7f260400b330 comp rx=0 tx=0).stop 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2820783938 shutdown_connections 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305db7e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1370 0x55ac305c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/2820783938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1d30 0x55ac305db2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2820783938 >> 172.21.15.112:0/2820783938 conn(0x55ac304dd180 msgr2=0x55ac305c4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2820783938 shutdown_connections 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 -- 172.21.15.112:0/2820783938 wait complete. 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.327+0000 7f261e6bdb80 1 -- start start 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 1 Processor -- start 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1370 0x55ac305c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1d30 0x55ac305c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25f000ff70 con 0x55ac305be2c0 2024-05-10T10:42:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25f000f8a0 con 0x55ac305b1370 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f25f000caa0 con 0x55ac305b1d30 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f261e6bdb80 10 client.0 _ll_register_callbacks cb 0x55ac3049b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f26123c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f2611bc6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1370 0x55ac305c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f26123c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32898/0 (socket says 172.21.15.112:32898) 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.328+0000 7f26123c7700 1 -- 172.21.15.112:0/3777448242 learned_addr learned my addr 172.21.15.112:0/3777448242 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f26113c5700 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1d30 0x55ac305c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f2611bc6700 1 -- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1d30 msgr2=0x55ac305c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f2611bc6700 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1d30 0x55ac305c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f2611bc6700 1 -- 172.21.15.112:0/3777448242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 msgr2=0x55ac305c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f2611bc6700 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f2611bc6700 1 -- 172.21.15.112:0/3777448242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3063e140 con 0x55ac305b1370 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f2611bc6700 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1370 0x55ac305c8bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f260801c010 tx=0x7f2608009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c8407444de49364 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f25f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f25f57fa700 1 -- 172.21.15.112:0/3777448242 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260800ee60 con 0x55ac305b1370 2024-05-10T10:42:14.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26040b1040 con 0x55ac305b1370 2024-05-10T10:42:14.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f25f57fa700 1 -- 172.21.15.112:0/3777448242 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f260801b420 con 0x55ac305b1370 2024-05-10T10:42:14.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f25f57fa700 1 -- 172.21.15.112:0/3777448242 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f260801d020 con 0x55ac305b1370 2024-05-10T10:42:14.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f261e6bdb80 10 client.25361 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:14.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.329+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac3063f460 con 0x55ac305b1370 2024-05-10T10:42:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f25f57fa700 1 -- 172.21.15.112:0/3777448242 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26040b1040 con 0x55ac305b1370 2024-05-10T10:42:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f25f57fa700 1 -- 172.21.15.112:0/3777448242 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2608024040 con 0x55ac305b1370 2024-05-10T10:42:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f261e6bdb80 10 client.25361 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f261e6bdb80 10 client.25361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f261e6bdb80 20 client.25361 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac3063fb90 0x55ac30641fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f261e6bdb80 4 client.25361 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.330+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac305112c0 con 0x55ac3063fb90 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.331+0000 7f26113c5700 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac3063fb90 0x55ac30641fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.331+0000 7f26113c5700 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac3063fb90 0x55ac30641fb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd473700c7be8ab2 server_cookie=4ac01875680c85bd in_seq=0 out_seq=0 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.331+0000 7f25f57fa700 10 client.25361 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.332+0000 7f25f57fa700 1 -- 172.21.15.112:0/3777448242 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac305112c0 con 0x55ac3063fb90 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.332+0000 7f25f57fa700 10 client.25361 handle_command_reply: tid=0 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.332+0000 7f261e6bdb80 1 client.25361 shutdown 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1370 msgr2=0x55ac305c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1370 0x55ac305c8bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f260801c010 tx=0x7f2608009f10 comp rx=0 tx=0).stop 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 shutdown_connections 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ac3063fb90 0x55ac30641fb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac305be2c0 0x55ac305c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ac305b1370 0x55ac305c8bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f25f57fa700 0 client.25361 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 --2- 172.21.15.112:0/3777448242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac305b1d30 0x55ac305c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 >> 172.21.15.112:0/3777448242 conn(0x55ac304dd180 msgr2=0x55ac305b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 shutdown_connections 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 1 -- 172.21.15.112:0/3777448242 wait complete. 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "b86a858a-3683-4bc5-b94d-f7a15ce95ed4", 2024-05-10T10:42:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:42:14.337 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:42:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.333+0000 7f261e6bdb80 20 client.25361 trim_cache size 0 max 16384 2024-05-10T10:42:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25f80b2880 msgr2=0x7f25f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25f80b2880 0x7f25f80b4d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2608009cb0 tx=0x7f2608009260 comp rx=0 tx=0).stop 2024-05-10T10:42:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c107760 msgr2=0x7f260c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c107760 0x7f260c199790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2604009790 tx=0x7f260400cc30 comp rx=0 tx=0).stop 2024-05-10T10:42:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 shutdown_connections 2024-05-10T10:42:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f260c107760 0x7f260c199790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f260c102e00 0x7f260c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25f80b2880 0x7f25f80b4d40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 --2- 172.21.15.112:0/899313825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f260c1037b0 0x7f260c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 >> 172.21.15.112:0/899313825 conn(0x7f260c078a10 msgr2=0x7f260c101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 shutdown_connections 2024-05-10T10:42:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:14.334+0000 7f2613e2a700 1 -- 172.21.15.112:0/899313825 wait complete. 2024-05-10T10:42:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:14 smithi112 ceph-mon[30100]: pgmap v490: 129 pgs: 129 active+clean; 20 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-05-10T10:42:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:14 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:42:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:14 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:14 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:42:15.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:14 smithi195 ceph-mon[33773]: pgmap v490: 129 pgs: 129 active+clean; 20 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-05-10T10:42:15.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:14 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:42:15.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:14 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:15.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:14 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:42:15.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:42:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:14 smithi148 ceph-mon[33568]: pgmap v490: 129 pgs: 129 active+clean; 20 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-05-10T10:42:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:14 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:42:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:14 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:14 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:42:15.504 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:16.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:15 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:16.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:15 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:16.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:15 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:17.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:16 smithi148 ceph-mon[33568]: pgmap v491: 129 pgs: 129 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-10T10:42:17.302 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:16 smithi112 ceph-mon[30100]: pgmap v491: 129 pgs: 129 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-10T10:42:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:16 smithi195 ceph-mon[33773]: pgmap v491: 129 pgs: 129 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-10T10:42:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 -- 172.21.15.112:0/1312372961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 msgr2=0x7fd35810ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/1312372961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd35810ae50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd350009ca0 tx=0x7fd35002a8c0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 -- 172.21.15.112:0/1312372961 shutdown_connections 2024-05-10T10:42:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/1312372961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd35810ae50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/1312372961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd358103c00 0x7fd358104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/1312372961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd358102a10 0x7fd358102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 -- 172.21.15.112:0/1312372961 >> 172.21.15.112:0/1312372961 conn(0x7fd3580fdfa0 msgr2=0x7fd3581003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 -- 172.21.15.112:0/1312372961 shutdown_connections 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.348+0000 7fd35d0e1640 1 -- 172.21.15.112:0/1312372961 wait complete. 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 Processor -- start 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 -- start start 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd358102a10 0x7fd358198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd358103c00 0x7fd358198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd3581a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810e440 con 0x7fd3581045c0 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810e2c0 con 0x7fd358102a10 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.349+0000 7fd35d0e1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810e5c0 con 0x7fd358103c00 2024-05-10T10:42:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd3581a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd3581a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37692/0 (socket says 172.21.15.112:37692) 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 -- 172.21.15.112:0/983505140 learned_addr learned my addr 172.21.15.112:0/983505140 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35659c640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd358103c00 0x7fd358198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd356d9d640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd358102a10 0x7fd358198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 -- 172.21.15.112:0/983505140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd358103c00 msgr2=0x7fd358198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd358103c00 0x7fd358198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 -- 172.21.15.112:0/983505140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd358102a10 msgr2=0x7fd358198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd358102a10 0x7fd358198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 -- 172.21.15.112:0/983505140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3581a06a0 con 0x7fd3581045c0 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd356d9d640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd358102a10 0x7fd358198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35659c640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd358103c00 0x7fd358198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35759e640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd3581a0030 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd35002aeb0 tx=0x7fd350003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd33ffff640 1 -- 172.21.15.112:0/983505140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350003e10 con 0x7fd3581045c0 2024-05-10T10:42:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3581a0930 con 0x7fd3581045c0 2024-05-10T10:42:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd33ffff640 1 -- 172.21.15.112:0/983505140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3500044d0 con 0x7fd3581045c0 2024-05-10T10:42:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd33ffff640 1 -- 172.21.15.112:0/983505140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350030c90 con 0x7fd3581045c0 2024-05-10T10:42:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.350+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a0e80 con 0x7fd3581045c0 2024-05-10T10:42:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd33ffff640 1 -- 172.21.15.112:0/983505140 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd350004680 con 0x7fd3581045c0 2024-05-10T10:42:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 Processor -- start 2024-05-10T10:42:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 -- start start 2024-05-10T10:42:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beaddb0 0x55ee9beae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beb32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beb3910 0x55ee9beb5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee9be125d0 con 0x55ee9beae7a0 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ee9bdcb890 con 0x55ee9beb3910 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ee9be0e590 con 0x55ee9beaddb0 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beb32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beb32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37698/0 (socket says 172.21.15.112:37698) 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 -- 172.21.15.112:0/3381107203 learned_addr learned my addr 172.21.15.112:0/3381107203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35659c640 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beaddb0 0x55ee9beae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35659c640 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beb3910 0x55ee9beb5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 -- 172.21.15.112:0/3381107203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beaddb0 msgr2=0x55ee9beae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beaddb0 0x55ee9beae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 -- 172.21.15.112:0/3381107203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beb3910 msgr2=0x55ee9beb5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beb3910 0x55ee9beb5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd356d9d640 1 -- 172.21.15.112:0/3381107203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee9bb24210 con 0x55ee9beae7a0 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.353+0000 7fd35659c640 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beb3910 0x55ee9beb5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.354+0000 7fd33ffff640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd3400b28a0 0x7fd3400b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.354+0000 7fd356d9d640 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beb32a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd34800bd30 tx=0x7fd34800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a16f9132c52fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.354+0000 7fd35659c640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd3400b28a0 0x7fd3400b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.354+0000 7fd33d7fa640 1 -- 172.21.15.112:0/3381107203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348015070 con 0x55ee9beae7a0 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.354+0000 7fd33ffff640 1 -- 172.21.15.112:0/983505140 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd350103bc0 con 0x7fd3581045c0 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.354+0000 7fd33d7fa640 1 -- 172.21.15.112:0/3381107203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd34800be20 con 0x55ee9beae7a0 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.354+0000 7fd33d7fa640 1 -- 172.21.15.112:0/3381107203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348010630 con 0x55ee9beae7a0 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3381107203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 msgr2=0x55ee9beb32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35659c640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd3400b28a0 0x7fd3400b4d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55ee9beaf000 tx=0x7fd34c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beb32a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd34800bd30 tx=0x7fd34800f730 comp rx=0 tx=0).stop 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3381107203 shutdown_connections 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beb3910 0x55ee9beb5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beb32a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3381107203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beaddb0 0x55ee9beae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3381107203 >> 172.21.15.112:0/3381107203 conn(0x55ee9bea9420 msgr2=0x55ee9bea9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3381107203 shutdown_connections 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3381107203 wait complete. 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- start start 2024-05-10T10:42:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 Processor -- start 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beb8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9beb8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bebf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee9beb68c0 con 0x55ee9beb3910 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ee9bdcf370 con 0x55ee9beaddb0 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ee9beb6a40 con 0x55ee9beae7a0 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.355+0000 7fd35fdfe740 10 client.0 _ll_register_callbacks cb 0x55ee9bb23a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bebf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bebf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37702/0 (socket says 172.21.15.112:37702) 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 -- 172.21.15.112:0/2645585290 learned_addr learned my addr 172.21.15.112:0/2645585290 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd356d9d640 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beb8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35659c640 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9beb8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 -- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 msgr2=0x55ee9beb8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9beb8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 -- 172.21.15.112:0/2645585290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 msgr2=0x55ee9beb8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beb8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 -- 172.21.15.112:0/2645585290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee9bbf0620 con 0x55ee9beb3910 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35759e640 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bebf580 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd3500c6ed0 tx=0x7fd3500ca960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73ce3bc80270951 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd33aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd33aff5640 1 -- 172.21.15.112:0/2645585290 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3500cf200 con 0x55ee9beb3910 2024-05-10T10:42:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd33aff5640 1 -- 172.21.15.112:0/2645585290 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd3500c70e0 con 0x55ee9beb3910 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd34800be20 con 0x55ee9beb3910 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd33aff5640 1 -- 172.21.15.112:0/2645585290 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3500d3cb0 con 0x55ee9beb3910 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35fdfe740 10 client.15954 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee9beb68c0 con 0x55ee9beb3910 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.356+0000 7fd33aff5640 1 -- 172.21.15.112:0/2645585290 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd34800be20 con 0x55ee9beb3910 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd33aff5640 1 -- 172.21.15.112:0/2645585290 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd350004b40 con 0x55ee9beb3910 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd35fdfe740 10 client.15954 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd35fdfe740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd35fdfe740 20 client.15954 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bec3c20 0x55ee9bec6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd35fdfe740 4 client.15954 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee9be201e0 con 0x55ee9bec3c20 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd356d9d640 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bec3c20 0x55ee9bec6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd356d9d640 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bec3c20 0x55ee9bec6060 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400ef13529c120a0 server_cookie=b32cdd33c02c5203 in_seq=0 out_seq=0 2024-05-10T10:42:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.357+0000 7fd33aff5640 10 client.15954 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd33aff5640 1 -- 172.21.15.112:0/2645585290 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee9be201e0 con 0x55ee9bec3c20 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd33aff5640 10 client.15954 handle_command_reply: tid=0 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 client.15954 shutdown 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 msgr2=0x55ee9bebf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bebf580 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd3500c6ed0 tx=0x7fd3500ca960 comp rx=0 tx=0).stop 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 shutdown_connections 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bebf580 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9beb8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bec3c20 0x55ee9bec6060 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/2645585290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beb8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd33aff5640 0 client.15954 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 >> 172.21.15.112:0/2645585290 conn(0x55ee9bea9420 msgr2=0x55ee9bea8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 shutdown_connections 2024-05-10T10:42:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 1 -- 172.21.15.112:0/2645585290 wait complete. 2024-05-10T10:42:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.358+0000 7fd35fdfe740 20 client.15954 trim_cache size 0 max 16384 2024-05-10T10:42:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.373+0000 7fd35fdfe740 1 Processor -- start 2024-05-10T10:42:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.373+0000 7fd35fdfe740 1 -- start start 2024-05-10T10:42:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beb3910 0x55ee9beacdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581ab350 con 0x55ee9beae7a0 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581ace90 con 0x55ee9beaddb0 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581aba20 con 0x55ee9beb3910 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37710/0 (socket says 172.21.15.112:37710) 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 -- 172.21.15.112:0/3006579855 learned_addr learned my addr 172.21.15.112:0/3006579855 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35759e640 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beb3910 0x55ee9beacdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 -- 172.21.15.112:0/3006579855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beb3910 msgr2=0x55ee9beacdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beb3910 0x55ee9beacdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 -- 172.21.15.112:0/3006579855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 msgr2=0x55ee9beac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd356d9d640 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beac330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 -- 172.21.15.112:0/3006579855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee9beaf030 con 0x55ee9beae7a0 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd35659c640 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beac870 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00f9d0 tx=0x7fd34c013050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3cae23ecaa33c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd33cff9640 1 -- 172.21.15.112:0/3006579855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c004220 con 0x55ee9beae7a0 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd33cff9640 1 -- 172.21.15.112:0/3006579855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd34c0043c0 con 0x55ee9beae7a0 2024-05-10T10:42:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.374+0000 7fd33cff9640 1 -- 172.21.15.112:0/3006579855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c01d9f0 con 0x55ee9beae7a0 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3006579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 msgr2=0x55ee9beac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beac870 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00f9d0 tx=0x7fd34c013050 comp rx=0 tx=0).stop 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3006579855 shutdown_connections 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beb3910 0x55ee9beacdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beae7a0 0x55ee9beac870 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/3006579855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9beac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3006579855 >> 172.21.15.112:0/3006579855 conn(0x55ee9bea9420 msgr2=0x55ee9beaf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3006579855 shutdown_connections 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 -- 172.21.15.112:0/3006579855 wait complete. 2024-05-10T10:42:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 -- start start 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.375+0000 7fd35fdfe740 1 Processor -- start 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9bf2a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9bf2a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bf31880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581ac7c0 con 0x55ee9beb3910 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581ac0f0 con 0x55ee9beaddb0 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581a9330 con 0x55ee9beae7a0 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 10 client.0 _ll_register_callbacks cb 0x7fd350005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bf31880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bf31880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37716/0 (socket says 172.21.15.112:37716) 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 -- 172.21.15.112:0/1521044313 learned_addr learned my addr 172.21.15.112:0/1521044313 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35659c640 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9bf2a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd356d9d640 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9bf2a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 -- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 msgr2=0x55ee9bf2a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9bf2a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 -- 172.21.15.112:0/1521044313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 msgr2=0x55ee9bf2a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9bf2a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 -- 172.21.15.112:0/1521044313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee9bf31e70 con 0x55ee9beb3910 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35759e640 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bf31880 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd350004f80 tx=0x7fd3500cad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b2d7784bf5fbb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd33aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd33aff5640 1 -- 172.21.15.112:0/1521044313 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3500ea070 con 0x55ee9beb3910 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd33aff5640 1 -- 172.21.15.112:0/1521044313 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd3500ca380 con 0x55ee9beb3910 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd34c0043c0 con 0x55ee9beb3910 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd33aff5640 1 -- 172.21.15.112:0/1521044313 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3500ad050 con 0x55ee9beb3910 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.376+0000 7fd35fdfe740 10 client.15960 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee9bf331f0 con 0x55ee9beb3910 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd33aff5640 1 -- 172.21.15.112:0/1521044313 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd34c0043c0 con 0x55ee9beb3910 2024-05-10T10:42:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd33aff5640 1 -- 172.21.15.112:0/1521044313 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd3500cb0b0 con 0x55ee9beb3910 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd35fdfe740 10 client.15960 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd35fdfe740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd35fdfe740 20 client.15960 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bf33a60 0x55ee9bf35e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd35fdfe740 4 client.15960 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee9be1a050 con 0x55ee9bf33a60 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd356d9d640 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bf33a60 0x55ee9bf35e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd33aff5640 10 client.15960 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:17.377+0000 7fd356d9d640 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bf33a60 0x55ee9bf35e80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be65fbe7396517e9 server_cookie=b013e67614b1c1e9 in_seq=0 out_seq=0 2024-05-10T10:42:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd33aff5640 1 -- 172.21.15.112:0/1521044313 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee9be1a050 con 0x55ee9bf33a60 2024-05-10T10:42:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd33aff5640 10 client.15960 handle_command_reply: tid=0 2024-05-10T10:42:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 client.15960 shutdown 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 msgr2=0x55ee9bf31880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bf31880 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd350004f80 tx=0x7fd3500cad00 comp rx=0 tx=0).stop 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 shutdown_connections 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee9beb3910 0x55ee9bf31880 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee9beae7a0 0x55ee9bf2a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee9bf33a60 0x55ee9bf35e80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 --2- 172.21.15.112:0/1521044313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee9beaddb0 0x55ee9bf2a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 >> 172.21.15.112:0/1521044313 conn(0x55ee9bea9420 msgr2=0x55ee9bea7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd33aff5640 0 client.15960 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 shutdown_connections 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 1 -- 172.21.15.112:0/1521044313 wait complete. 2024-05-10T10:42:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.641+0000 7fd35fdfe740 20 client.15960 trim_cache size 0 max 16384 2024-05-10T10:42:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:42:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:42:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:42:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:42:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd3400b28a0 msgr2=0x7fd3400b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd3400b28a0 0x7fd3400b4d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55ee9beaf000 tx=0x7fd34c005f90 comp rx=0 tx=0).stop 2024-05-10T10:42:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 msgr2=0x7fd3581a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd3581a0030 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd35002aeb0 tx=0x7fd350003c20 comp rx=0 tx=0).stop 2024-05-10T10:42:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 shutdown_connections 2024-05-10T10:42:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd3400b28a0 0x7fd3400b4d60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3581045c0 0x7fd3581a0030 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd358103c00 0x7fd358198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 --2- 172.21.15.112:0/983505140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd358102a10 0x7fd358198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 >> 172.21.15.112:0/983505140 conn(0x7fd3580fdfa0 msgr2=0x7fd358106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 shutdown_connections 2024-05-10T10:42:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:18.642+0000 7fd35d0e1640 1 -- 172.21.15.112:0/983505140 wait complete. 2024-05-10T10:42:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:18 smithi148 ceph-mon[33568]: pgmap v492: 129 pgs: 129 active+clean; 28 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.6 MiB/s wr, 323 op/s 2024-05-10T10:42:19.300 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:18 smithi112 ceph-mon[30100]: pgmap v492: 129 pgs: 129 active+clean; 28 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.6 MiB/s wr, 323 op/s 2024-05-10T10:42:19.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:18 smithi195 ceph-mon[33773]: pgmap v492: 129 pgs: 129 active+clean; 28 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.6 MiB/s wr, 323 op/s 2024-05-10T10:42:20.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:19 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:42:20.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:19 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:42:20.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:19 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 -- 172.21.15.112:0/1649885811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 msgr2=0x7f79b4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/1649885811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 0x7f79b4101f10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009990 tx=0x7f79a4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 -- 172.21.15.112:0/1649885811 shutdown_connections 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/1649885811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4102450 0x7f79b4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/1649885811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b4109520 0x7f79b4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/1649885811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 0x7f79b4101f10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 -- 172.21.15.112:0/1649885811 >> 172.21.15.112:0/1649885811 conn(0x7f79b4078a10 msgr2=0x7f79b40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 -- 172.21.15.112:0/1649885811 shutdown_connections 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.110+0000 7f79bc5e9700 1 -- 172.21.15.112:0/1649885811 wait complete. 2024-05-10T10:42:21.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 Processor -- start 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 -- start start 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 0x7f79b419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b4102450 0x7f79b419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4109520 0x7f79b41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41134e0 con 0x7f79b4101a90 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4113360 con 0x7f79b4109520 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bc5e9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4113660 con 0x7f79b4102450 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79ba385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 0x7f79b419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4109520 0x7f79b41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79b9b84700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b4102450 0x7f79b419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4109520 0x7f79b41998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34010/0 (socket says 172.21.15.112:34010) 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 -- 172.21.15.112:0/2635379452 learned_addr learned my addr 172.21.15.112:0/2635379452 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 -- 172.21.15.112:0/2635379452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b4102450 msgr2=0x7f79b419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b4102450 0x7f79b419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 -- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 msgr2=0x7f79b419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 0x7f79b419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.111+0000 7f79bab86700 1 -- 172.21.15.112:0/2635379452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b419a000 con 0x7f79b4109520 2024-05-10T10:42:21.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.112+0000 7f79b9b84700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b4102450 0x7f79b419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:21.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.112+0000 7f79bab86700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4109520 0x7f79b41998a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00ef30 tx=0x7f79ac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:21.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:20 smithi112 ceph-mon[30100]: pgmap v493: 129 pgs: 129 active+clean; 26 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.4 MiB/s wr, 416 op/s 2024-05-10T10:42:21.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.112+0000 7f79ab7fe700 1 -- 172.21.15.112:0/2635379452 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0092e0 con 0x7f79b4109520 2024-05-10T10:42:21.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.112+0000 7f79ab7fe700 1 -- 172.21.15.112:0/2635379452 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79ac004bb0 con 0x7f79b4109520 2024-05-10T10:42:21.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.112+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b419a290 con 0x7f79b4109520 2024-05-10T10:42:21.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.112+0000 7f79ab7fe700 1 -- 172.21.15.112:0/2635379452 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac004020 con 0x7f79b4109520 2024-05-10T10:42:21.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.112+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41a65b0 con 0x7f79b4109520 2024-05-10T10:42:21.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.115+0000 7f79ab7fe700 1 -- 172.21.15.112:0/2635379452 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f79ac01b030 con 0x7f79b4109520 2024-05-10T10:42:21.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.116+0000 7f79ab7fe700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f79a00b29d0 0x7f79a00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.116+0000 7f79ab7fe700 1 -- 172.21.15.112:0/2635379452 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f79ac0e1f80 con 0x7f79b4109520 2024-05-10T10:42:21.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79ba385700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f79a00b29d0 0x7f79a00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 Processor -- start 2024-05-10T10:42:21.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 -- start start 2024-05-10T10:42:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375925c0 0x555a375929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a37592ee0 0x555a3758cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a3758d160 0x555a3758d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555a374da940 con 0x555a37592ee0 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555a3743fda0 con 0x555a3758d160 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555a374d6900 con 0x555a375925c0 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79ba385700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f79a00b29d0 0x7f79a00b4e90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79b419a9e0 tx=0x7f79b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79b9b84700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a37592ee0 0x555a3758cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79bab86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a3758d160 0x555a3758d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79ba385700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375925c0 0x555a375929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79bab86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a3758d160 0x555a3758d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34014/0 (socket says 172.21.15.112:34014) 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79ba385700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375925c0 0x555a375929a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46510/0 (socket says 172.21.15.112:46510) 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79bab86700 1 -- 172.21.15.112:0/2168179210 learned_addr learned my addr 172.21.15.112:0/2168179210 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.117+0000 7f79b9b84700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a37592ee0 0x555a3758cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37756/0 (socket says 172.21.15.112:37756) 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79bab86700 1 -- 172.21.15.112:0/2168179210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375925c0 msgr2=0x555a375929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79bab86700 1 --2- 172.21.15.112:0/2168179210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375925c0 0x555a375929a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79bab86700 1 -- 172.21.15.112:0/2168179210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a37592ee0 msgr2=0x555a3758cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79bab86700 1 --2- 172.21.15.112:0/2168179210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a37592ee0 0x555a3758cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79bab86700 1 -- 172.21.15.112:0/2168179210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a374278d0 con 0x555a3758d160 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79bab86700 1 --2- 172.21.15.112:0/2168179210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a3758d160 0x555a3758d5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0a9230 tx=0x7f79ac0a8db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b1427e9c454f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79a8ff9700 1 -- 172.21.15.112:0/2168179210 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0a92a0 con 0x555a3758d160 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79a8ff9700 1 -- 172.21.15.112:0/2168179210 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79ac0ad650 con 0x555a3758d160 2024-05-10T10:42:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.118+0000 7f79a8ff9700 1 -- 172.21.15.112:0/2168179210 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0b1e40 con 0x555a3758d160 2024-05-10T10:42:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2168179210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a3758d160 msgr2=0x555a3758d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2168179210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a3758d160 0x555a3758d5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0a9230 tx=0x7f79ac0a8db0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2168179210 shutdown_connections 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2168179210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a3758d160 0x555a3758d5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2168179210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375925c0 0x555a375929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2168179210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a37592ee0 0x555a3758cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2168179210 >> 172.21.15.112:0/2168179210 conn(0x555a374b4b70 msgr2=0x555a375832a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2168179210 shutdown_connections 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2168179210 wait complete. 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 -- start start 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.119+0000 7f79c6e7cb80 1 Processor -- start 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a3758d160 0x555a37585fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375925c0 0x555a37586500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a37586a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555a375859d0 con 0x555a3758d160 2024-05-10T10:42:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555a37493ed0 con 0x555a375925c0 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555a37585b50 con 0x555a37592ee0 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79c6e7cb80 10 client.0 _ll_register_callbacks cb 0x555a3746e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a3758d160 0x555a37585fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a3758d160 0x555a37585fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37762/0 (socket says 172.21.15.112:37762) 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79b9b84700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375925c0 0x555a37586500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 -- 172.21.15.112:0/2179675891 learned_addr learned my addr 172.21.15.112:0/2179675891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79bab86700 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a37586a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 -- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 msgr2=0x555a37586a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a37586a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 -- 172.21.15.112:0/2179675891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375925c0 msgr2=0x555a37586500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375925c0 0x555a37586500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 -- 172.21.15.112:0/2179675891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a37491760 con 0x555a3758d160 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79b9b84700 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375925c0 0x555a37586500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.120+0000 7f79ba385700 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a3758d160 0x555a37585fc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f79b0011f50 tx=0x7f79b0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58456ecb0c0357c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f799dffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f799dffb700 1 -- 172.21.15.112:0/2179675891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0032060 con 0x555a3758d160 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f799dffb700 1 -- 172.21.15.112:0/2179675891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f79b0020cc0 con 0x555a3758d160 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79ac0ad650 con 0x555a3758d160 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f799dffb700 1 -- 172.21.15.112:0/2179675891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79b0039050 con 0x555a3758d160 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f79c6e7cb80 10 client.15972 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a3759fef0 con 0x555a3758d160 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f799dffb700 1 -- 172.21.15.112:0/2179675891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79b0004050 con 0x555a3758d160 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f799dffb700 1 -- 172.21.15.112:0/2179675891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f79b0003840 con 0x555a3758d160 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f79c6e7cb80 10 client.15972 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f79c6e7cb80 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.121+0000 7f79c6e7cb80 20 client.15972 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.122+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a375a2530 0x555a375a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.122+0000 7f79c6e7cb80 4 client.15972 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.122+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f79ac0ad650 con 0x555a375a2530 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.122+0000 7f79b9b84700 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a375a2530 0x555a375a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.122+0000 7f79b9b84700 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a375a2530 0x555a375a4920 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e811a5be1d01a21 server_cookie=a5f9ec7a5e4380b4 in_seq=0 out_seq=0 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.122+0000 7f799dffb700 10 client.15972 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f799dffb700 1 -- 172.21.15.112:0/2179675891 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f79ac0ad650 con 0x555a375a2530 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f799dffb700 10 client.15972 handle_command_reply: tid=0 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 client.15972 shutdown 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a3758d160 msgr2=0x555a37585fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a3758d160 0x555a37585fc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f79b0011f50 tx=0x7f79b0013040 comp rx=0 tx=0).stop 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 shutdown_connections 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a375a2530 0x555a375a4920 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f799dffb700 0 client.15972 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a37586a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a3758d160 0x555a37585fc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/2179675891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375925c0 0x555a37586500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.124+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 >> 172.21.15.112:0/2179675891 conn(0x555a374b4b70 msgr2=0x555a3757db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:21.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.125+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 shutdown_connections 2024-05-10T10:42:21.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.125+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/2179675891 wait complete. 2024-05-10T10:42:21.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.125+0000 7f79c6e7cb80 20 client.15972 trim_cache size 0 max 16384 2024-05-10T10:42:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 Processor -- start 2024-05-10T10:42:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 -- start start 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a375981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a375aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375a2530 0x555a375aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41acfd0 con 0x555a375925c0 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41aeb10 con 0x555a375a2530 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41ad6a0 con 0x555a37592ee0 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79ba385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a375981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79bab86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375a2530 0x555a375aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79ba385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a375981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37770/0 (socket says 172.21.15.112:37770) 2024-05-10T10:42:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79ba385700 1 -- 172.21.15.112:0/690043655 learned_addr learned my addr 172.21.15.112:0/690043655 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.151+0000 7f79b9b84700 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a375aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f79ba385700 1 -- 172.21.15.112:0/690043655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 msgr2=0x555a375aa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f79ba385700 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a375aa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f79ba385700 1 -- 172.21.15.112:0/690043655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375a2530 msgr2=0x555a375aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f79ba385700 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375a2530 0x555a375aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f79ba385700 1 -- 172.21.15.112:0/690043655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a37592bf0 con 0x555a375925c0 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f79bab86700 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375a2530 0x555a375aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f79ba385700 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a375981d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f79b0000c00 tx=0x7f79b00220d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46df87505edc5c6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f799ffff700 1 -- 172.21.15.112:0/690043655 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0020cc0 con 0x555a375925c0 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f799ffff700 1 -- 172.21.15.112:0/690043655 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79b0042070 con 0x555a375925c0 2024-05-10T10:42:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.152+0000 7f799ffff700 1 -- 172.21.15.112:0/690043655 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0005320 con 0x555a375925c0 2024-05-10T10:42:21.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/690043655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 msgr2=0x555a375981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a375981d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f79b0000c00 tx=0x7f79b00220d0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/690043655 shutdown_connections 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a375a2530 0x555a375aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a375981d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/690043655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a37592ee0 0x555a375aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/690043655 >> 172.21.15.112:0/690043655 conn(0x555a374b4b70 msgr2=0x555a37589e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/690043655 shutdown_connections 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.153+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/690043655 wait complete. 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 -- start start 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 Processor -- start 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a3759c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a37592ee0 0x555a3759cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375a2530 0x555a3759d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41ae440 con 0x555a375925c0 2024-05-10T10:42:21.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41add70 con 0x555a37592ee0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41aaf70 con 0x555a375a2530 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79c6e7cb80 10 client.0 _ll_register_callbacks cb 0x555a3758ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79ba385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a3759c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79ba385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a3759c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37784/0 (socket says 172.21.15.112:37784) 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79ba385700 1 -- 172.21.15.112:0/1045581319 learned_addr learned my addr 172.21.15.112:0/1045581319 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79b9b84700 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a37592ee0 0x555a3759cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.154+0000 7f79bab86700 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375a2530 0x555a3759d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79ba385700 1 -- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375a2530 msgr2=0x555a3759d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79ba385700 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375a2530 0x555a3759d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79ba385700 1 -- 172.21.15.112:0/1045581319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a37592ee0 msgr2=0x555a3759cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79ba385700 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a37592ee0 0x555a3759cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79ba385700 1 -- 172.21.15.112:0/1045581319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a3760be10 con 0x555a375925c0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79ba385700 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a3759c580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009410 tx=0x7f79b0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4e3d428df979a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f799dffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:21.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f799dffb700 1 -- 172.21.15.112:0/1045581319 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b000eb70 con 0x555a375925c0 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f799dffb700 1 -- 172.21.15.112:0/1045581319 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f79b000ed10 con 0x555a375925c0 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b0042070 con 0x555a375925c0 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f799dffb700 1 -- 172.21.15.112:0/1045581319 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79b0054050 con 0x555a375925c0 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79c6e7cb80 10 client.15978 fetch_fsmap learned FSMap version 30 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.155+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a3760d100 con 0x555a375925c0 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f799dffb700 1 -- 172.21.15.112:0/1045581319 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79b0042070 con 0x555a375925c0 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f799dffb700 1 -- 172.21.15.112:0/1045581319 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f79b0022450 con 0x555a375925c0 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f79c6e7cb80 10 client.15978 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f79c6e7cb80 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f79c6e7cb80 20 client.15978 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a3760d720 0x555a3760fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:21.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f79c6e7cb80 4 client.15978 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a374e8320 con 0x555a3760d720 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.156+0000 7f79b9b84700 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a3760d720 0x555a3760fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f799dffb700 10 client.15978 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79b9b84700 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a3760d720 0x555a3760fb40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa192bd00cdddd9 server_cookie=2739bb9de1b27f8f in_seq=0 out_seq=0 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f799dffb700 1 -- 172.21.15.112:0/1045581319 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a374e8320 con 0x555a3760d720 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f799dffb700 10 client.15978 handle_command_reply: tid=0 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 client.15978 shutdown 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 msgr2=0x555a3759c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a3759c580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009410 tx=0x7f79b0005a00 comp rx=0 tx=0).stop 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 shutdown_connections 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555a3760d720 0x555a3760fb40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a375a2530 0x555a3759d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a375925c0 0x555a3759c580 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 --2- 172.21.15.112:0/1045581319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a37592ee0 0x555a3759cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f799dffb700 0 client.15978 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 >> 172.21.15.112:0/1045581319 conn(0x555a374b4b70 msgr2=0x555a37585da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 shutdown_connections 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.157+0000 7f79c6e7cb80 1 -- 172.21.15.112:0/1045581319 wait complete. 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79c6e7cb80 20 client.15978 trim_cache size 0 max 16384 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:42:21.164 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:42:21.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f79a00b29d0 msgr2=0x7f79a00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f79a00b29d0 0x7f79a00b4e90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79b419a9e0 tx=0x7f79b0009260 comp rx=0 tx=0).stop 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4109520 msgr2=0x7f79b41998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4109520 0x7f79b41998a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00ef30 tx=0x7f79ac00c3a0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 shutdown_connections 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b4109520 0x7f79b41998a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101a90 0x7f79b419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f79a00b29d0 0x7f79a00b4e90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.158+0000 7f79bc5e9700 1 --2- 172.21.15.112:0/2635379452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b4102450 0x7f79b419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.159+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 >> 172.21.15.112:0/2635379452 conn(0x7f79b4078a10 msgr2=0x7f79b4100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.159+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 shutdown_connections 2024-05-10T10:42:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:21.159+0000 7f79bc5e9700 1 -- 172.21.15.112:0/2635379452 wait complete. 2024-05-10T10:42:21.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:20 smithi148 ceph-mon[33568]: pgmap v493: 129 pgs: 129 active+clean; 26 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.4 MiB/s wr, 416 op/s 2024-05-10T10:42:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:20 smithi195 ceph-mon[33773]: pgmap v493: 129 pgs: 129 active+clean; 26 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.4 MiB/s wr, 416 op/s 2024-05-10T10:42:22.299 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:42:22.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:42:22.635 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:22.846 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 7% (321/4287) Updating files: 8% (343/4287) Updating files: 9% (386/4287) Updating files: 10% (429/4287) Updating files: 11% (472/4287) Updating files: 12% (515/4287) Updating files: 12% (539/4287) Updating files: 13% (558/4287) Updating files: 14% (601/4287) Updating files: 15% (644/4287) Updating files: 16% (686/4287) Updating files: 17% (729/4287) Updating files: 18% (772/4287) Updating files: 19% (815/4287) Updating files: 20% (858/4287) Updating files: 21% (901/4287) Updating files: 22% (944/4287) Updating files: 22% (950/4287) Updating files: 23% (987/4287) Updating files: 24% (1029/4287) Updating files: 25% (1072/4287) Updating files: 26% (1115/4287) Updating files: 27% (1158/4287) Updating files: 28% (1201/4287) Updating files: 29% (1244/4287) Updating files: 30% (1287/4287) Updating files: 30% (1288/4287) Updating files: 31% (1329/4287) Updating files: 32% (1372/4287) Updating files: 33% (1415/4287) Updating files: 34% (1458/4287) Updating files: 35% (1501/4287) Updating files: 35% (1520/4287) Updating files: 36% (1544/4287) Updating files: 37% (1587/4287) Updating files: 38% (1630/4287) Updating files: 39% (1672/4287) Updating files: 40% (1715/4287) Updating files: 41% (1758/4287) Updating files: 42% (1801/4287) Updating files: 43% (1844/4287) Updating files: 43% (1846/4287) Updating files: 44% (1887/4287) Updating files: 45% (1930/4287) Updating files: 46% (1973/4287) Updating files: 47% (2015/4287) Updating files: 48% (2058/4287) Updating files: 49% (2101/4287) Updating files: 50% (2144/4287) Updating files: 51% (2187/4287) Updating files: 52% (2230/4287) Updating files: 52% (2233/4287) Updating files: 53% (2273/4287) Updating files: 54% (2315/4287) Updating files: 55% (2358/4287) Updating files: 56% (2401/4287) Updating files: 57% (2444/4287) Updating files: 57% (2456/4287) Updating files: 58% (2487/4287) Updating files: 59% (2530/4287) Updating files: 60% (2573/4287) Updating files: 61% (2616/4287) Updating files: 62% (2658/4287) Updating files: 63% (2701/4287) Updating files: 64% (2744/4287) Updating files: 65% (2787/4287) Updating files: 65% (2788/4287) Updating files: 66% (2830/4287) Updating files: 67% (2873/4287) Updating files: 68% (2916/4287) Updating files: 69% (2959/4287) Updating files: 70% (3001/4287) Updating files: 70% (3022/4287) Updating files: 71% (3044/4287) Updating files: 71% (3082/4287) Updating files: 72% (3087/4287) Updating files: 73% (3130/4287) Updating files: 74% (3173/4287) Updating files: 75% (3216/4287) Updating files: 76% (3259/4287) Updating files: 77% (3301/4287) Updating files: 78% (3344/4287) Updating files: 79% (3387/4287) Updating files: 79% (3414/4287) Updating files: 80% (3430/4287) Updating files: 81% (3473/4287) Updating files: 82% (3516/4287) Updating files: 83% (3559/4287) Updating files: 84% (3602/4287) Updating files: 85% (3644/4287) Updating files: 86% (3687/4287) Updating files: 86% (3703/4287) Updating files: 87% (3730/4287) Updating files: 88% (3773/4287) Updating files: 89% (3816/4287) Updating files: 90% (3859/4287) Updating files: 91% (3902/4287) Updating files: 91% (3942/4287) Updating files: 92% (3945/4287) Updating files: 93% (3987/4287) Updating files: 94% (4030/4287) Updating files: 95% (4073/4287) Updating files: 96% (4116/4287) Updating files: 97% (4159/4287) Updating files: 98% (4202/4287) Updating files: 99% (4245/4287) Updating files: 99% (4275/4287) Updating files: 100% (4287/4287) Updating files: 100% (4287/4287), done. 2024-05-10T10:42:22.856 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd xfstests-dev 2024-05-10T10:42:22.857 INFO:tasks.workunit.client.0.smithi112.stderr:+ make -j4 2024-05-10T10:42:22.906 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize -cfi 2024-05-10T10:42:23.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:22 smithi112 ceph-mon[30100]: pgmap v494: 129 pgs: 129 active+clean; 23 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.3 MiB/s wr, 812 op/s 2024-05-10T10:42:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:22 smithi148 ceph-mon[33568]: pgmap v494: 129 pgs: 129 active+clean; 23 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.3 MiB/s wr, 812 op/s 2024-05-10T10:42:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:22 smithi195 ceph-mon[33773]: pgmap v494: 129 pgs: 129 active+clean; 23 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.3 MiB/s wr, 812 op/s 2024-05-10T10:42:24.070 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: putting auxiliary files in '.'. 2024-05-10T10:42:24.070 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './config.guess' 2024-05-10T10:42:24.076 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './config.sub' 2024-05-10T10:42:24.083 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './install-sh' 2024-05-10T10:42:24.091 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './ltmain.sh' 2024-05-10T10:42:24.124 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-05-10T10:42:24.124 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-05-10T10:42:24.133 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-05-10T10:42:24.143 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-05-10T10:42:24.151 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-05-10T10:42:24.160 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-05-10T10:42:24.168 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-05-10T10:42:24.169 INFO:tasks.workunit.client.0.smithi112.stdout:cp include/install-sh . 2024-05-10T10:42:24.193 INFO:tasks.workunit.client.0.smithi112.stdout:aclocal -I m4 2024-05-10T10:42:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 -- 172.21.15.112:0/3781535606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 msgr2=0x7f51d01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 --2- 172.21.15.112:0/3781535606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01087e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009990 tx=0x7f51c4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 -- 172.21.15.112:0/3781535606 shutdown_connections 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 --2- 172.21.15.112:0/3781535606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d0102d50 0x7f51d010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 --2- 172.21.15.112:0/3781535606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01087e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 --2- 172.21.15.112:0/3781535606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d0102390 0x7f51d0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 -- 172.21.15.112:0/3781535606 >> 172.21.15.112:0/3781535606 conn(0x7f51d0079310 msgr2=0x7f51d0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 -- 172.21.15.112:0/3781535606 shutdown_connections 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.811+0000 7f51d7039700 1 -- 172.21.15.112:0/3781535606 wait complete. 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 Processor -- start 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 -- start start 2024-05-10T10:42:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d0102390 0x7f51d019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d0102d50 0x7f51d019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0112400 con 0x7f51d0108400 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0112280 con 0x7f51d0102d50 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d7039700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0112580 con 0x7f51d0102390 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d55d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d55d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37794/0 (socket says 172.21.15.112:37794) 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d55d6700 1 -- 172.21.15.112:0/1496375756 learned_addr learned my addr 172.21.15.112:0/1496375756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51cffff700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d0102d50 0x7f51d019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.812+0000 7f51d4dd5700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d0102390 0x7f51d019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d55d6700 1 -- 172.21.15.112:0/1496375756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d0102390 msgr2=0x7f51d019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d55d6700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d0102390 0x7f51d019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d55d6700 1 -- 172.21.15.112:0/1496375756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d0102d50 msgr2=0x7f51d019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d55d6700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d0102d50 0x7f51d019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d55d6700 1 -- 172.21.15.112:0/1496375756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d0198fa0 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d4dd5700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d0102390 0x7f51d019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d55d6700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01987c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f51c8007ee0 tx=0x7f51c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51cdffb700 1 -- 172.21.15.112:0/1496375756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c800dbd0 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51cdffb700 1 -- 172.21.15.112:0/1496375756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51c800dd70 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d0199290 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51cdffb700 1 -- 172.21.15.112:0/1496375756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c80058d0 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.813+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d01a54a0 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.814+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d0103ab0 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.815+0000 7f51cdffb700 1 -- 172.21.15.112:0/1496375756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f51c801a020 con 0x7f51d0108400 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.816+0000 7f51cdffb700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51c00b2900 0x7f51c00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.816+0000 7f51d4dd5700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51c00b2900 0x7f51c00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.816+0000 7f51cdffb700 1 -- 172.21.15.112:0/1496375756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f51c80e1c70 con 0x7f51d0108400 2024-05-10T10:42:24.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.816+0000 7f51d4dd5700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51c00b2900 0x7f51c00b4dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f51c4006010 tx=0x7f51c401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:24.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:24.825+0000 7f51cdffb700 1 -- 172.21.15.112:0/1496375756 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f51c8010070 con 0x7f51d0108400 2024-05-10T10:42:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:24 smithi112 ceph-mon[30100]: pgmap v495: 129 pgs: 129 active+clean; 28 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 979 op/s 2024-05-10T10:42:25.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.098+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51d002e510 con 0x7f51d0108400 2024-05-10T10:42:25.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.099+0000 7f51cdffb700 1 -- 172.21.15.112:0/1496375756 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f51c80cb070 con 0x7f51d0108400 2024-05-10T10:42:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:42:25.101 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:40:06.380474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51c00b2900 msgr2=0x7f51c00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51c00b2900 0x7f51c00b4dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f51c4006010 tx=0x7f51c401a040 comp rx=0 tx=0).stop 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 msgr2=0x7f51d01987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01987c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f51c8007ee0 tx=0x7f51c800d3b0 comp rx=0 tx=0).stop 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 shutdown_connections 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d0108400 0x7f51d01987c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f51c00b2900 0x7f51c00b4dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d0102390 0x7f51d019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:25.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 --2- 172.21.15.112:0/1496375756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d0102d50 0x7f51d019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 >> 172.21.15.112:0/1496375756 conn(0x7f51d0079310 msgr2=0x7f51d0101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.101+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 shutdown_connections 2024-05-10T10:42:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:25.102+0000 7f51d7039700 1 -- 172.21.15.112:0/1496375756 wait complete. 2024-05-10T10:42:25.104 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-05-10T10:42:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:24 smithi148 ceph-mon[33568]: pgmap v495: 129 pgs: 129 active+clean; 28 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 979 op/s 2024-05-10T10:42:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:24 smithi195 ceph-mon[33773]: pgmap v495: 129 pgs: 129 active+clean; 28 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 979 op/s 2024-05-10T10:42:25.732 INFO:tasks.workunit.client.0.smithi112.stdout:autoheader 2024-05-10T10:42:26.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:25 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1496375756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:42:26.305 INFO:tasks.workunit.client.0.smithi112.stdout:autoconf 2024-05-10T10:42:26.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:25 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1496375756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:42:26.325 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:25 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1496375756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:42:26.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:42:26.544 INFO:tasks.workunit.client.0.smithi112.stdout:./configure \ 2024-05-10T10:42:26.605 INFO:tasks.workunit.client.0.smithi112.stdout: --libexecdir=/usr/lib \ 2024-05-10T10:42:26.605 INFO:tasks.workunit.client.0.smithi112.stdout: --exec_prefix=/var/lib 2024-05-10T10:42:26.653 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:26.746 INFO:tasks.workunit.client.0.smithi112.stdout:checking build system type... x86_64-pc-linux-gnu 2024-05-10T10:42:26.746 INFO:tasks.workunit.client.0.smithi112.stdout:checking host system type... x86_64-pc-linux-gnu 2024-05-10T10:42:26.748 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to print strings... printf 2024-05-10T10:42:26.748 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... gcc 2024-05-10T10:42:26.973 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the C compiler works... yes 2024-05-10T10:42:26.973 INFO:tasks.workunit.client.0.smithi112.stdout:checking for C compiler default output file name... a.out 2024-05-10T10:42:27.044 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of executables... 2024-05-10T10:42:27.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:26 smithi112 ceph-mon[30100]: pgmap v496: 129 pgs: 129 active+clean; 29 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.00k op/s 2024-05-10T10:42:27.097 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are cross compiling... no 2024-05-10T10:42:27.149 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of object files... o 2024-05-10T10:42:27.182 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... yes 2024-05-10T10:42:27.214 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... yes 2024-05-10T10:42:27.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:26 smithi148 ceph-mon[33568]: pgmap v496: 129 pgs: 129 active+clean; 29 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.00k op/s 2024-05-10T10:42:27.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:26 smithi195 ceph-mon[33773]: pgmap v496: 129 pgs: 129 active+clean; 29 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.00k op/s 2024-05-10T10:42:27.573 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-10T10:42:27.579 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-05-10T10:42:27.584 INFO:tasks.workunit.client.0.smithi112.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-10T10:42:27.585 INFO:tasks.workunit.client.0.smithi112.stdout:checking for egrep... /usr/bin/grep -E 2024-05-10T10:42:27.587 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fgrep... /usr/bin/grep -F 2024-05-10T10:42:27.591 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ld used by gcc... /usr/bin/ld 2024-05-10T10:42:27.593 INFO:tasks.workunit.client.0.smithi112.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-05-10T10:42:27.631 INFO:tasks.workunit.client.0.smithi112.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-05-10T10:42:29.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:28 smithi148 ceph-mon[33568]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 932 op/s 2024-05-10T10:42:29.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:28 smithi195 ceph-mon[33773]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 932 op/s 2024-05-10T10:42:29.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:28 smithi112 ceph-mon[30100]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 932 op/s 2024-05-10T10:42:30.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 -- 172.21.15.112:0/3523900046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 msgr2=0x7efe6c10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 --2- 172.21.15.112:0/3523900046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c10f2d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efe6400b0d0 tx=0x7efe6400b3e0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 -- 172.21.15.112:0/3523900046 shutdown_connections 2024-05-10T10:42:30.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 --2- 172.21.15.112:0/3523900046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c10f2d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 --2- 172.21.15.112:0/3523900046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe6c1029e0 0x7efe6c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 --2- 172.21.15.112:0/3523900046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe6c103390 0x7efe6c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 -- 172.21.15.112:0/3523900046 >> 172.21.15.112:0/3523900046 conn(0x7efe6c079350 msgr2=0x7efe6c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 -- 172.21.15.112:0/3523900046 shutdown_connections 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.653+0000 7efe716da700 1 -- 172.21.15.112:0/3523900046 wait complete. 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.654+0000 7efe716da700 1 Processor -- start 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.654+0000 7efe716da700 1 -- start start 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.654+0000 7efe716da700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe6c1029e0 0x7efe6c10fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.654+0000 7efe716da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe6c103390 0x7efe6c1101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.654+0000 7efe716da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.654+0000 7efe716da700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1135a0 con 0x7efe6c107730 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe716da700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c113420 con 0x7efe6c103390 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe716da700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c113720 con 0x7efe6c1029e0 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52464/0 (socket says 172.21.15.112:52464) 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 -- 172.21.15.112:0/755238132 learned_addr learned my addr 172.21.15.112:0/755238132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6affd700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe6c1029e0 0x7efe6c10fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe6c103390 0x7efe6c1101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 -- 172.21.15.112:0/755238132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe6c1029e0 msgr2=0x7efe6c10fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe6c1029e0 0x7efe6c10fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 -- 172.21.15.112:0/755238132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe6c103390 msgr2=0x7efe6c1101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe6c103390 0x7efe6c1101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 -- 172.21.15.112:0/755238132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1a7fe0 con 0x7efe6c107730 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c110730 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efe64007dc0 tx=0x7efe64007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe63fff700 1 -- 172.21.15.112:0/755238132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6401f070 con 0x7efe6c107730 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe63fff700 1 -- 172.21.15.112:0/755238132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe64005500 con 0x7efe6c107730 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1a8270 con 0x7efe6c107730 2024-05-10T10:42:30.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe63fff700 1 -- 172.21.15.112:0/755238132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6401b7d0 con 0x7efe6c107730 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.655+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1a86c0 con 0x7efe6c107730 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.658+0000 7efe63fff700 1 -- 172.21.15.112:0/755238132 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7efe64022070 con 0x7efe6c107730 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.658+0000 7efe63fff700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe580b2880 0x7efe580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.658+0000 7efe6affd700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe580b2880 0x7efe580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.658+0000 7efe63fff700 1 -- 172.21.15.112:0/755238132 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe6400f020 con 0x7efe6c107730 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 Processor -- start 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe6affd700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe580b2880 0x7efe580b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005950 tx=0x7efe5c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 -- start start 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba3644d0 0x55c2ba3648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba369100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba2b6860 con 0x55c2ba3644d0 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba2557c0 con 0x55c2ba368c80 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba2b2820 con 0x55c2ba364df0 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe6affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba3644d0 0x55c2ba3648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe6affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba3644d0 0x55c2ba3648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52474/0 (socket says 172.21.15.112:52474) 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe6affd700 1 -- 172.21.15.112:0/169993983 learned_addr learned my addr 172.21.15.112:0/169993983 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.659+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba369100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe6affd700 1 -- 172.21.15.112:0/169993983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 msgr2=0x55c2ba3686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe6affd700 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3686f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe6affd700 1 -- 172.21.15.112:0/169993983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 msgr2=0x55c2ba369100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe6affd700 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba369100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe6affd700 1 -- 172.21.15.112:0/169993983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ba2038d0 con 0x55c2ba3644d0 2024-05-10T10:42:30.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe6affd700 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba3644d0 0x55c2ba3648b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efe5c010fa0 tx=0x7efe5c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a10a16e0b6a502f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe617fa700 1 -- 172.21.15.112:0/169993983 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00be60 con 0x55c2ba3644d0 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe617fa700 1 -- 172.21.15.112:0/169993983 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5c011db0 con 0x55c2ba3644d0 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.660+0000 7efe617fa700 1 -- 172.21.15.112:0/169993983 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c01c650 con 0x55c2ba3644d0 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 -- 172.21.15.112:0/169993983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba3644d0 msgr2=0x55c2ba3648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba3644d0 0x55c2ba3648b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efe5c010fa0 tx=0x7efe5c013000 comp rx=0 tx=0).stop 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 -- 172.21.15.112:0/169993983 shutdown_connections 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba369100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba3644d0 0x55c2ba3648b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/169993983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 -- 172.21.15.112:0/169993983 >> 172.21.15.112:0/169993983 conn(0x55c2ba290100 msgr2=0x55c2ba35ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 -- 172.21.15.112:0/169993983 shutdown_connections 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 -- 172.21.15.112:0/169993983 wait complete. 2024-05-10T10:42:30.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 -- start start 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 Processor -- start 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.661+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba370040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba36d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba36d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba370e90 con 0x55c2ba364df0 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba21cc30 con 0x55c2ba368c80 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2ba371010 con 0x55c2ba3644d0 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe7bf6db80 10 client.0 _ll_register_callbacks cb 0x55c2ba24e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba36d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba36d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52486/0 (socket says 172.21.15.112:52486) 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6a7fc700 1 -- 172.21.15.112:0/1064044012 learned_addr learned my addr 172.21.15.112:0/1064044012 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:30.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6affd700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba370040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba36d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6affd700 1 -- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 msgr2=0x55c2ba36d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6affd700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba36d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6affd700 1 -- 172.21.15.112:0/1064044012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 msgr2=0x55c2ba36d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6affd700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba36d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6affd700 1 -- 172.21.15.112:0/1064044012 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ba252690 con 0x55c2ba3644d0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba36d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe6affd700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba370040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005d90 tx=0x7efe5c01e650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac23ff1c2e2d33f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe527fc700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe527fc700 1 -- 172.21.15.112:0/1064044012 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c035060 con 0x55c2ba3644d0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe5c011db0 con 0x55c2ba3644d0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.662+0000 7efe527fc700 1 -- 172.21.15.112:0/1064044012 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe5c011bb0 con 0x55c2ba3644d0 2024-05-10T10:42:30.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe527fc700 1 -- 172.21.15.112:0/1064044012 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efe5c03c050 con 0x55c2ba3644d0 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 10 client.25441 fetch_fsmap learned FSMap version 31 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c2ba37b6c0 con 0x55c2ba3644d0 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe527fc700 1 -- 172.21.15.112:0/1064044012 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5c011db0 con 0x55c2ba3644d0 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe527fc700 1 -- 172.21.15.112:0/1064044012 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7efe5c0263f0 con 0x55c2ba3644d0 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 10 client.25441 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 10 client.25441 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 20 client.25441 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba37dbf0 0x55c2ba37ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 4 client.25441 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2ba330650 con 0x55c2ba37dbf0 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.663+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba37dbf0 0x55c2ba37ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe527fc700 10 client.25441 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:42:30.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba37dbf0 0x55c2ba37ffe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6eac52d92c36b77e server_cookie=5c0813c33c742603 in_seq=0 out_seq=0 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe527fc700 1 -- 172.21.15.112:0/1064044012 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2ba330650 con 0x55c2ba37dbf0 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe527fc700 10 client.25441 handle_command_reply: tid=0 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 client.25441 shutdown 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 msgr2=0x55c2ba370040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba370040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005d90 tx=0x7efe5c01e650 comp rx=0 tx=0).stop 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 shutdown_connections 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba36d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba37dbf0 0x55c2ba37ffe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba370040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1064044012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba36d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe527fc700 0 client.25441 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.664+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 >> 172.21.15.112:0/1064044012 conn(0x55c2ba290100 msgr2=0x55c2ba359720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.665+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 shutdown_connections 2024-05-10T10:42:30.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.665+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1064044012 wait complete. 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.665+0000 7efe7bf6db80 20 client.25441 trim_cache size 0 max 16384 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.679+0000 7efe7bf6db80 1 Processor -- start 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.679+0000 7efe7bf6db80 1 -- start start 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba379f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba3639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba385580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b1320 con 0x55c2ba364df0 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b2e90 con 0x55c2ba368c80 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b19f0 con 0x55c2ba3644d0 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba3639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba3639f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52494/0 (socket says 172.21.15.112:52494) 2024-05-10T10:42:30.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 -- 172.21.15.112:0/489021864 learned_addr learned my addr 172.21.15.112:0/489021864 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6affd700 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba379f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba385580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 -- 172.21.15.112:0/489021864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 msgr2=0x55c2ba379f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba379f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 -- 172.21.15.112:0/489021864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 msgr2=0x55c2ba385580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba385580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 -- 172.21.15.112:0/489021864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ba36b990 con 0x55c2ba364df0 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba385580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6affd700 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba379f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.680+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba3639f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55c2ba371140 tx=0x7efe54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9045da7d73013e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe60ff9700 1 -- 172.21.15.112:0/489021864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54021070 con 0x55c2ba364df0 2024-05-10T10:42:30.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe60ff9700 1 -- 172.21.15.112:0/489021864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe54016be0 con 0x55c2ba364df0 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe60ff9700 1 -- 172.21.15.112:0/489021864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe540156f0 con 0x55c2ba364df0 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 -- 172.21.15.112:0/489021864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 msgr2=0x55c2ba3639f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba3639f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55c2ba371140 tx=0x7efe54008040 comp rx=0 tx=0).stop 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 -- 172.21.15.112:0/489021864 shutdown_connections 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba368c80 0x55c2ba385580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba3644d0 0x55c2ba379f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/489021864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba364df0 0x55c2ba3639f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 -- 172.21.15.112:0/489021864 >> 172.21.15.112:0/489021864 conn(0x55c2ba290100 msgr2=0x55c2ba3632e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.681+0000 7efe7bf6db80 1 -- 172.21.15.112:0/489021864 shutdown_connections 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 -- 172.21.15.112:0/489021864 wait complete. 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 -- start start 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 Processor -- start 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba3644d0 0x55c2ba372080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba368c80 0x55c2ba372b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b2790 con 0x55c2ba368c80 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b20c0 con 0x55c2ba3644d0 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1af2c0 con 0x55c2ba364df0 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.682+0000 7efe7bf6db80 10 client.0 _ll_register_callbacks cb 0x55c2ba24e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba368c80 0x55c2ba372b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba368c80 0x55c2ba372b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52510/0 (socket says 172.21.15.112:52510) 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 -- 172.21.15.112:0/1185588151 learned_addr learned my addr 172.21.15.112:0/1185588151 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6affd700 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba3644d0 0x55c2ba372080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6a7fc700 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 -- 172.21.15.112:0/1185588151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 msgr2=0x55c2ba3725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 -- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba3644d0 msgr2=0x55c2ba372080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba3644d0 0x55c2ba372080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 -- 172.21.15.112:0/1185588151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ba3e7390 con 0x55c2ba368c80 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe6b7fe700 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba368c80 0x55c2ba372b00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efe640e1430 tx=0x7efe640b3380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40c565f3bc649ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe527fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe527fc700 1 -- 172.21.15.112:0/1185588151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640c3030 con 0x55c2ba368c80 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe527fc700 1 -- 172.21.15.112:0/1185588151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe640a7260 con 0x55c2ba368c80 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe54016be0 con 0x55c2ba368c80 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe527fc700 1 -- 172.21.15.112:0/1185588151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efe640c52c0 con 0x55c2ba368c80 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe7bf6db80 10 client.16002 fetch_fsmap learned FSMap version 31 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c2ba3e85e0 con 0x55c2ba368c80 2024-05-10T10:42:30.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.683+0000 7efe527fc700 1 -- 172.21.15.112:0/1185588151 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe54016be0 con 0x55c2ba368c80 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe527fc700 1 -- 172.21.15.112:0/1185588151 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7efe64011070 con 0x55c2ba368c80 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe7bf6db80 10 client.16002 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe7bf6db80 10 client.16002 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe7bf6db80 20 client.16002 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba3e8cc0 0x55c2ba3eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe7bf6db80 4 client.16002 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2ba2c4240 con 0x55c2ba3e8cc0 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe6affd700 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba3e8cc0 0x55c2ba3eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe527fc700 10 client.16002 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.684+0000 7efe6affd700 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba3e8cc0 0x55c2ba3eb0b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c96b5f4caad110b5 server_cookie=a7da130c1ccb5ed5 in_seq=0 out_seq=0 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe527fc700 1 -- 172.21.15.112:0/1185588151 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2ba2c4240 con 0x55c2ba3e8cc0 2024-05-10T10:42:30.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe527fc700 10 client.16002 handle_command_reply: tid=0 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 client.16002 shutdown 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba368c80 msgr2=0x55c2ba372b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba368c80 0x55c2ba372b00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efe640e1430 tx=0x7efe640b3380 comp rx=0 tx=0).stop 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 shutdown_connections 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2ba368c80 0x55c2ba372b00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2ba3644d0 0x55c2ba372080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55c2ba3e8cc0 0x55c2ba3eb0b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 --2- 172.21.15.112:0/1185588151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2ba364df0 0x55c2ba3725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 >> 172.21.15.112:0/1185588151 conn(0x55c2ba290100 msgr2=0x55c2ba363b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:30.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe527fc700 0 client.16002 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:42:30.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 shutdown_connections 2024-05-10T10:42:30.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 1 -- 172.21.15.112:0/1185588151 wait complete. 2024-05-10T10:42:30.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe7bf6db80 20 client.16002 trim_cache size 0 max 16384 2024-05-10T10:42:30.704 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:30.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe580b2880 msgr2=0x7efe580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe716da700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe580b2880 0x7efe580b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005950 tx=0x7efe5c00b410 comp rx=0 tx=0).stop 2024-05-10T10:42:30.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 msgr2=0x7efe6c110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:30.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.685+0000 7efe716da700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c110730 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efe64007dc0 tx=0x7efe64007df0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 shutdown_connections 2024-05-10T10:42:30.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe6c107730 0x7efe6c110730 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe580b2880 0x7efe580b4d40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe6c1029e0 0x7efe6c10fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 --2- 172.21.15.112:0/755238132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe6c103390 0x7efe6c1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:30.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 >> 172.21.15.112:0/755238132 conn(0x7efe6c079350 msgr2=0x7efe6c1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:30.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 shutdown_connections 2024-05-10T10:42:30.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:30.686+0000 7efe716da700 1 -- 172.21.15.112:0/755238132 wait complete. 2024-05-10T10:42:31.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:30 smithi112 ceph-mon[30100]: pgmap v498: 129 pgs: 129 active+clean; 33 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.3 MiB/s wr, 897 op/s 2024-05-10T10:42:31.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:30 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:30 smithi148 ceph-mon[33568]: pgmap v498: 129 pgs: 129 active+clean; 33 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.3 MiB/s wr, 897 op/s 2024-05-10T10:42:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:30 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:31.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:30 smithi195 ceph-mon[33773]: pgmap v498: 129 pgs: 129 active+clean; 33 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.3 MiB/s wr, 897 op/s 2024-05-10T10:42:31.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:30 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:31.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:42:31.736 INFO:tasks.workunit.client.0.smithi112.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-05-10T10:42:31.736 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether ln -s works... yes 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 -- 172.21.15.112:0/1531768619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe381034c0 msgr2=0x7efe38109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 --2- 172.21.15.112:0/1531768619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe381034c0 0x7efe38109d50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efe30009ca0 tx=0x7efe3002a8c0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 -- 172.21.15.112:0/1531768619 shutdown_connections 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 --2- 172.21.15.112:0/1531768619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe381034c0 0x7efe38109d50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 --2- 172.21.15.112:0/1531768619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe38102b00 0x7efe38102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 --2- 172.21.15.112:0/1531768619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe38101910 0x7efe38101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 -- 172.21.15.112:0/1531768619 >> 172.21.15.112:0/1531768619 conn(0x7efe380fce80 msgr2=0x7efe380ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 -- 172.21.15.112:0/1531768619 shutdown_connections 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.741+0000 7efe3eacc640 1 -- 172.21.15.112:0/1531768619 wait complete. 2024-05-10T10:42:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 Processor -- start 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 -- start start 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe38101910 0x7efe38198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe38102b00 0x7efe38198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe381034c0 0x7efe381a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe3810e470 con 0x7efe38102b00 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe3810e2f0 con 0x7efe38101910 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3eacc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe3810e5f0 con 0x7efe381034c0 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe38102b00 0x7efe38198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe38102b00 0x7efe38198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52528/0 (socket says 172.21.15.112:52528) 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 -- 172.21.15.112:0/3661322466 learned_addr learned my addr 172.21.15.112:0/3661322466 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3c868640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe38101910 0x7efe38198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3d069640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe381034c0 0x7efe381a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 -- 172.21.15.112:0/3661322466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe381034c0 msgr2=0x7efe381a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe381034c0 0x7efe381a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 -- 172.21.15.112:0/3661322466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe38101910 msgr2=0x7efe38198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe38101910 0x7efe38198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe37fff640 1 -- 172.21.15.112:0/3661322466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe381a06c0 con 0x7efe38102b00 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.742+0000 7efe3d069640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe381034c0 0x7efe381a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.743+0000 7efe37fff640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe38102b00 0x7efe38198fe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efe2400d950 tx=0x7efe2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.743+0000 7efe35ffb640 1 -- 172.21.15.112:0/3661322466 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24017070 con 0x7efe38102b00 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.743+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe381a09b0 con 0x7efe38102b00 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.743+0000 7efe35ffb640 1 -- 172.21.15.112:0/3661322466 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe24009c30 con 0x7efe38102b00 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.743+0000 7efe35ffb640 1 -- 172.21.15.112:0/3661322466 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2400f620 con 0x7efe38102b00 2024-05-10T10:42:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.743+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe381a0f00 con 0x7efe38102b00 2024-05-10T10:42:31.748 INFO:tasks.workunit.client.0.smithi112.stdout:checking the maximum length of command line arguments... 1572864 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 Processor -- start 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 -- start start 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe35ffb640 1 -- 172.21.15.112:0/3661322466 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7efe24010460 con 0x7efe38102b00 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5bbb0 0x5651d6e606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5651d6dbb5d0 con 0x5651d6e60d20 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5651d6d74890 con 0x5651d6e5b1c0 2024-05-10T10:42:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe417df740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5651d6db7590 con 0x5651d6e5bbb0 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e631b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52544/0 (socket says 172.21.15.112:52544) 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe37fff640 1 -- 172.21.15.112:0/1443836448 learned_addr learned my addr 172.21.15.112:0/1443836448 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe3c868640 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5bbb0 0x5651d6e606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.746+0000 7efe3d069640 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe37fff640 1 -- 172.21.15.112:0/1443836448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5bbb0 msgr2=0x5651d6e606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe37fff640 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5bbb0 0x5651d6e606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe37fff640 1 -- 172.21.15.112:0/1443836448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 msgr2=0x5651d6e5b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe37fff640 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe37fff640 1 -- 172.21.15.112:0/1443836448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d6acd210 con 0x5651d6e60d20 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe3c868640 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5bbb0 0x5651d6e606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe37fff640 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e631b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efe240e5fd0 tx=0x7efe240ea910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16eeb95cca142ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe35ffb640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe280b28a0 0x7efe280b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe3d069640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe280b28a0 0x7efe280b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe35ffb640 1 -- 172.21.15.112:0/3661322466 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe240e2260 con 0x7efe38102b00 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe22ffd640 1 -- 172.21.15.112:0/1443836448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240f5070 con 0x5651d6e60d20 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.747+0000 7efe22ffd640 1 -- 172.21.15.112:0/1443836448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe240a9000 con 0x5651d6e60d20 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe22ffd640 1 -- 172.21.15.112:0/1443836448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240a8e40 con 0x5651d6e60d20 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe3d069640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe280b28a0 0x7efe280b4d60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efe30004af0 tx=0x7efe30005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 -- 172.21.15.112:0/1443836448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 msgr2=0x5651d6e631b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e631b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efe240e5fd0 tx=0x7efe240ea910 comp rx=0 tx=0).stop 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 -- 172.21.15.112:0/1443836448 shutdown_connections 2024-05-10T10:42:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e631b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5bbb0 0x5651d6e606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 --2- 172.21.15.112:0/1443836448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 -- 172.21.15.112:0/1443836448 >> 172.21.15.112:0/1443836448 conn(0x5651d6e52420 msgr2=0x5651d6e52820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.748+0000 7efe417df740 1 -- 172.21.15.112:0/1443836448 shutdown_connections 2024-05-10T10:42:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 -- 172.21.15.112:0/1443836448 wait complete. 2024-05-10T10:42:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 -- start start 2024-05-10T10:42:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 Processor -- start 2024-05-10T10:42:31.754 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-05-10T10:42:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e65440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e65980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e60d20 0x5651d6e6c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5651d6e63cd0 con 0x5651d6e5bbb0 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5651d6d78370 con 0x5651d6e60d20 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5651d6e63e50 con 0x5651d6e5b1c0 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe417df740 10 client.0 _ll_register_callbacks cb 0x5651d6acca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.749+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e65980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3d069640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e60d20 0x5651d6e6c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e65980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52546/0 (socket says 172.21.15.112:52546) 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe37fff640 1 -- 172.21.15.112:0/3458360600 learned_addr learned my addr 172.21.15.112:0/3458360600 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3c868640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e65440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3d069640 1 -- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 msgr2=0x5651d6e65440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3d069640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e65440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3d069640 1 -- 172.21.15.112:0/3458360600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 msgr2=0x5651d6e65980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3d069640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e65980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3d069640 1 -- 172.21.15.112:0/3458360600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d6b99620 con 0x5651d6e60d20 2024-05-10T10:42:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe37fff640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e65980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3c868640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e65440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe3d069640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e60d20 0x5651d6e6c9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe3002ade0 tx=0x7efe3002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fefc55dcd4e4c092 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe17fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe17fff640 1 -- 172.21.15.112:0/3458360600 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30004310 con 0x5651d6e60d20 2024-05-10T10:42:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe240a9000 con 0x5651d6e60d20 2024-05-10T10:42:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.750+0000 7efe17fff640 1 -- 172.21.15.112:0/3458360600 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe300044b0 con 0x5651d6e60d20 2024-05-10T10:42:31.758 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-05-10T10:42:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe17fff640 1 -- 172.21.15.112:0/3458360600 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efe300046b0 con 0x5651d6e60d20 2024-05-10T10:42:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 10 client.25424 fetch_fsmap learned FSMap version 31 2024-05-10T10:42:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5651d6e6f9a0 con 0x5651d6e60d20 2024-05-10T10:42:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe17fff640 1 -- 172.21.15.112:0/3458360600 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe240a9000 con 0x5651d6e60d20 2024-05-10T10:42:31.760 INFO:tasks.workunit.client.0.smithi112.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-05-10T10:42:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe17fff640 1 -- 172.21.15.112:0/3458360600 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7efe30050070 con 0x5651d6e60d20 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 10 client.25424 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 10 client.25424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 20 client.25424 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6e71080 0x5651d6e734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 4 client.25424 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.751+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651d6dc91e0 con 0x5651d6e71080 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.752+0000 7efe3c868640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6e71080 0x5651d6e734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.752+0000 7efe17fff640 10 client.25424 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.752+0000 7efe3c868640 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6e71080 0x5651d6e734e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22465220e45b0559 server_cookie=35ef9d9234e63e79 in_seq=0 out_seq=0 2024-05-10T10:42:31.762 INFO:tasks.workunit.client.0.smithi112.stdout:checking for objdump... objdump 2024-05-10T10:42:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe17fff640 1 -- 172.21.15.112:0/3458360600 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651d6dc91e0 con 0x5651d6e71080 2024-05-10T10:42:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe17fff640 10 client.25424 handle_command_reply: tid=0 2024-05-10T10:42:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 client.25424 shutdown 2024-05-10T10:42:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e60d20 msgr2=0x5651d6e6c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e60d20 0x5651d6e6c9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe3002ade0 tx=0x7efe3002a8c0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 shutdown_connections 2024-05-10T10:42:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6e71080 0x5651d6e734e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.763 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to recognize dependent libraries... pass_all 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e60d20 0x5651d6e6c9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e65980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe17fff640 0 client.25424 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 --2- 172.21.15.112:0/3458360600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e65440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 >> 172.21.15.112:0/3458360600 conn(0x5651d6e52420 msgr2=0x5651d6e51580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 shutdown_connections 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 1 -- 172.21.15.112:0/3458360600 wait complete. 2024-05-10T10:42:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.753+0000 7efe417df740 20 client.25424 trim_cache size 0 max 16384 2024-05-10T10:42:31.765 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dlltool... no 2024-05-10T10:42:31.765 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-05-10T10:42:31.765 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ar... ar 2024-05-10T10:42:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 Processor -- start 2024-05-10T10:42:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 -- start start 2024-05-10T10:42:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5bbb0 0x5651d6e5d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e6c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe381a92b0 con 0x5651d6e60d20 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe381aae20 con 0x5651d6e5bbb0 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe417df740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe381a9980 con 0x5651d6e5b1c0 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe3d069640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e6c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5bbb0 0x5651d6e5d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe3d069640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e6c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52556/0 (socket says 172.21.15.112:52556) 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe3d069640 1 -- 172.21.15.112:0/3523610908 learned_addr learned my addr 172.21.15.112:0/3523610908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.771+0000 7efe3c868640 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe37fff640 1 -- 172.21.15.112:0/3523610908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 msgr2=0x5651d6e5cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe37fff640 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe37fff640 1 -- 172.21.15.112:0/3523610908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 msgr2=0x5651d6e6c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe37fff640 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e6c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe37fff640 1 -- 172.21.15.112:0/3523610908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d6e73b60 con 0x5651d6e5bbb0 2024-05-10T10:42:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe37fff640 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5bbb0 0x5651d6e5d310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe240e2610 tx=0x7efe240a53b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ae8ac2ad776093 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe227fc640 1 -- 172.21.15.112:0/3523610908 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24020070 con 0x5651d6e5bbb0 2024-05-10T10:42:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe227fc640 1 -- 172.21.15.112:0/3523610908 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe240e5b90 con 0x5651d6e5bbb0 2024-05-10T10:42:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.772+0000 7efe227fc640 1 -- 172.21.15.112:0/3523610908 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240a8c60 con 0x5651d6e5bbb0 2024-05-10T10:42:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 -- 172.21.15.112:0/3523610908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5bbb0 msgr2=0x5651d6e5d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5bbb0 0x5651d6e5d310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe240e2610 tx=0x7efe240a53b0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 -- 172.21.15.112:0/3523610908 shutdown_connections 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e60d20 0x5651d6e6c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5bbb0 0x5651d6e5d310 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 --2- 172.21.15.112:0/3523610908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e5b1c0 0x5651d6e5cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 -- 172.21.15.112:0/3523610908 >> 172.21.15.112:0/3523610908 conn(0x5651d6e52420 msgr2=0x5651d6e4d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 -- 172.21.15.112:0/3523610908 shutdown_connections 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.773+0000 7efe417df740 1 -- 172.21.15.112:0/3523610908 wait complete. 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 -- start start 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 Processor -- start 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e69fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e6a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e60d20 0x5651d6e6b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe381aa720 con 0x5651d6e5bbb0 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe381aa050 con 0x5651d6e5b1c0 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe381a7290 con 0x5651d6e60d20 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe417df740 10 client.0 _ll_register_callbacks cb 0x7efe30009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e6a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e6a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52558/0 (socket says 172.21.15.112:52558) 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe37fff640 1 -- 172.21.15.112:0/2448209380 learned_addr learned my addr 172.21.15.112:0/2448209380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe3c868640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e69fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.774+0000 7efe3d069640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e60d20 0x5651d6e6b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe3c868640 1 -- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e60d20 msgr2=0x5651d6e6b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe3c868640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e60d20 0x5651d6e6b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe3c868640 1 -- 172.21.15.112:0/2448209380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 msgr2=0x5651d6e6a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe3c868640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e6a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe3c868640 1 -- 172.21.15.112:0/2448209380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d6e52250 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe37fff640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e6a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe3c868640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e69fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5651d6e73d60 tx=0x7efe2c0129c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e1ed60a7c9502a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe17fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe17fff640 1 -- 172.21.15.112:0/2448209380 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0035f0 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe17fff640 1 -- 172.21.15.112:0/2448209380 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe2c003790 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe240e5b90 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe17fff640 1 -- 172.21.15.112:0/2448209380 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efe2c031050 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe417df740 10 client.25430 fetch_fsmap learned FSMap version 31 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.775+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5651d6e6cc20 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe17fff640 1 -- 172.21.15.112:0/2448209380 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe240e5b90 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe17fff640 1 -- 172.21.15.112:0/2448209380 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7efe2c0216a0 con 0x5651d6e5b1c0 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe417df740 10 client.25430 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T10:42:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe417df740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe417df740 20 client.25430 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe417df740 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6edbfc0 0x5651d6ede3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe417df740 4 client.25430 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:42:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651d6dc3050 con 0x5651d6edbfc0 2024-05-10T10:42:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.776+0000 7efe37fff640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6edbfc0 0x5651d6ede3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.777+0000 7efe17fff640 10 client.25430 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:31.777+0000 7efe37fff640 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6edbfc0 0x5651d6ede3e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980a70bbcab731d8 server_cookie=132d479e36bd24a4 in_seq=0 out_seq=0 2024-05-10T10:42:32.017 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:32.219 INFO:tasks.workunit.client.0.smithi112.stdout:checking for archiver @FILE support... @ 2024-05-10T10:42:32.220 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strip... strip 2024-05-10T10:42:32.221 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ranlib... ranlib 2024-05-10T10:42:32.222 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gawk... gawk 2024-05-10T10:42:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe17fff640 1 -- 172.21.15.112:0/2448209380 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651d6dc3050 con 0x5651d6edbfc0 2024-05-10T10:42:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe17fff640 10 client.25430 handle_command_reply: tid=0 2024-05-10T10:42:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 client.25430 shutdown 2024-05-10T10:42:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 msgr2=0x5651d6e69fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e69fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5651d6e73d60 tx=0x7efe2c0129c0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 shutdown_connections 2024-05-10T10:42:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651d6e60d20 0x5651d6e6b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5651d6e5bbb0 0x5651d6e6a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5651d6edbfc0 0x5651d6ede3e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 --2- 172.21.15.112:0/2448209380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651d6e5b1c0 0x5651d6e69fa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe17fff640 0 client.25430 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 >> 172.21.15.112:0/2448209380 conn(0x5651d6e52420 msgr2=0x5651d6e65440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.274+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 shutdown_connections 2024-05-10T10:42:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe417df740 1 -- 172.21.15.112:0/2448209380 wait complete. 2024-05-10T10:42:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe417df740 20 client.25430 trim_cache size 0 max 16384 2024-05-10T10:42:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:42:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:42:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:42:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:42:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe280b28a0 msgr2=0x7efe280b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe280b28a0 0x7efe280b4d60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efe30004af0 tx=0x7efe30005d80 comp rx=0 tx=0).stop 2024-05-10T10:42:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe38102b00 msgr2=0x7efe38198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe38102b00 0x7efe38198fe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efe2400d950 tx=0x7efe2400de10 comp rx=0 tx=0).stop 2024-05-10T10:42:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 shutdown_connections 2024-05-10T10:42:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe280b28a0 0x7efe280b4d60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe381034c0 0x7efe381a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe38102b00 0x7efe38198fe0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 --2- 172.21.15.112:0/3661322466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe38101910 0x7efe38198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 >> 172.21.15.112:0/3661322466 conn(0x7efe380fce80 msgr2=0x7efe38105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 shutdown_connections 2024-05-10T10:42:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:32.275+0000 7efe3eacc640 1 -- 172.21.15.112:0/3661322466 wait complete. 2024-05-10T10:42:32.408 INFO:tasks.workunit.client.0.smithi112.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-05-10T10:42:32.409 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sysroot... no 2024-05-10T10:42:32.430 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a working dd... /usr/bin/dd 2024-05-10T10:42:32.448 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-05-10T10:42:32.505 INFO:tasks.workunit.client.0.smithi112.stdout:checking for mt... no 2024-05-10T10:42:32.526 INFO:tasks.workunit.client.0.smithi112.stdout:checking if : is a manifest tool... no 2024-05-10T10:42:32.708 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to run the C preprocessor... gcc -E 2024-05-10T10:42:33.004 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ANSI C header files... yes 2024-05-10T10:42:33.057 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... yes 2024-05-10T10:42:33.118 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... yes 2024-05-10T10:42:33.184 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdlib.h... yes 2024-05-10T10:42:33.248 INFO:tasks.workunit.client.0.smithi112.stdout:checking for string.h... yes 2024-05-10T10:42:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:32 smithi148 ceph-mon[33568]: pgmap v499: 129 pgs: 129 active+clean; 34 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 766 op/s 2024-05-10T10:42:33.317 INFO:tasks.workunit.client.0.smithi112.stdout:checking for memory.h... yes 2024-05-10T10:42:33.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:32 smithi195 ceph-mon[33773]: pgmap v499: 129 pgs: 129 active+clean; 34 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 766 op/s 2024-05-10T10:42:33.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:32 smithi112 ceph-mon[30100]: pgmap v499: 129 pgs: 129 active+clean; 34 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 766 op/s 2024-05-10T10:42:33.390 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... yes 2024-05-10T10:42:33.455 INFO:tasks.workunit.client.0.smithi112.stdout:checking for inttypes.h... yes 2024-05-10T10:42:33.523 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdint.h... yes 2024-05-10T10:42:33.589 INFO:tasks.workunit.client.0.smithi112.stdout:checking for unistd.h... yes 2024-05-10T10:42:33.656 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dlfcn.h... yes 2024-05-10T10:42:33.668 INFO:tasks.workunit.client.0.smithi112.stdout:checking for objdir... .libs 2024-05-10T10:42:33.848 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-05-10T10:42:33.848 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-05-10T10:42:33.907 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-05-10T10:42:34.028 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc static flag -static works... no 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 -- 172.21.15.112:0/3406892114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91541023a0 msgr2=0x7f9154102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 --2- 172.21.15.112:0/3406892114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91541023a0 0x7f9154102820 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9144009990 tx=0x7f9144009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 -- 172.21.15.112:0/3406892114 shutdown_connections 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 --2- 172.21.15.112:0/3406892114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9154102d60 0x7f915410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 --2- 172.21.15.112:0/3406892114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9154108410 0x7f91541087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 --2- 172.21.15.112:0/3406892114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91541023a0 0x7f9154102820 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 -- 172.21.15.112:0/3406892114 >> 172.21.15.112:0/3406892114 conn(0x7f9154079350 msgr2=0x7f9154079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 -- 172.21.15.112:0/3406892114 shutdown_connections 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.033+0000 7f915b3bf700 1 -- 172.21.15.112:0/3406892114 wait complete. 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 Processor -- start 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 -- start start 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91541023a0 0x7f91541a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9154102d60 0x7f91541a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9154108410 0x7f915419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9154112450 con 0x7f9154108410 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91541122d0 con 0x7f9154102d60 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915b3bf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91541125d0 con 0x7f91541023a0 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91541023a0 0x7f91541a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915995c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9154108410 0x7f915419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91541023a0 0x7f91541a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:37366/0 (socket says 172.21.15.112:37366) 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 -- 172.21.15.112:0/4137416416 learned_addr learned my addr 172.21.15.112:0/4137416416 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915895a700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9154102d60 0x7f91541a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 -- 172.21.15.112:0/4137416416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9154102d60 msgr2=0x7f91541a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9154102d60 0x7f91541a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 -- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9154108410 msgr2=0x7f915419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9154108410 0x7f915419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.034+0000 7f915915b700 1 -- 172.21.15.112:0/4137416416 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915419c210 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f915895a700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9154102d60 0x7f91541a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f915995c700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9154108410 0x7f915419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f915915b700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91541023a0 0x7f91541a18f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f915000e9c0 tx=0x7f915000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f914a7fc700 1 -- 172.21.15.112:0/4137416416 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915000cbd0 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915419c500 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f914a7fc700 1 -- 172.21.15.112:0/4137416416 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f915000cd70 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f914a7fc700 1 -- 172.21.15.112:0/4137416416 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150005540 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.035+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a87c0 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.037+0000 7f914a7fc700 1 -- 172.21.15.112:0/4137416416 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9150004750 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.038+0000 7f914a7fc700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91400b2880 0x7f91400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.038+0000 7f914a7fc700 1 -- 172.21.15.112:0/4137416416 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91500e27f0 con 0x7f91541023a0 2024-05-10T10:42:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.039+0000 7f915895a700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91400b2880 0x7f91400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.039+0000 7f915895a700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91400b2880 0x7f91400b4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f914400b2e0 tx=0x7f914401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.039+0000 7f9165c52b80 1 Processor -- start 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.039+0000 7f9165c52b80 1 -- start start 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f9165c52b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558287c27940 con 0x558287cde560 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f9165c52b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558287b8cda0 con 0x558287ce42e0 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f9165c52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558287c23900 con 0x558287ce39c0 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915895a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915915b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915895a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cde020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:38146/0 (socket says 172.21.15.112:38146) 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915895a700 1 -- 172.21.15.112:0/3530810630 learned_addr learned my addr 172.21.15.112:0/3530810630 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915995c700 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915915b700 1 -- 172.21.15.112:0/3530810630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 msgr2=0x558287cde020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915915b700 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cde020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915915b700 1 -- 172.21.15.112:0/3530810630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 msgr2=0x558287cde9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915915b700 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915915b700 1 -- 172.21.15.112:0/3530810630 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558287b748d0 con 0x558287ce39c0 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.040+0000 7f915995c700 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.041+0000 7f915915b700 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce3da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91500dff60 tx=0x7f91500a94f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=464f0b1b45895585 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.041+0000 7f913f7fe700 1 -- 172.21.15.112:0/3530810630 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500c3070 con 0x558287ce39c0 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.041+0000 7f913f7fe700 1 -- 172.21.15.112:0/3530810630 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91500b2230 con 0x558287ce39c0 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.041+0000 7f913f7fe700 1 -- 172.21.15.112:0/3530810630 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500a9f40 con 0x558287ce39c0 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 -- 172.21.15.112:0/3530810630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 msgr2=0x558287ce3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce3da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91500dff60 tx=0x7f91500a94f0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 -- 172.21.15.112:0/3530810630 shutdown_connections 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce3da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3530810630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cde020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 -- 172.21.15.112:0/3530810630 >> 172.21.15.112:0/3530810630 conn(0x558287c01b70 msgr2=0x558287cd02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 -- 172.21.15.112:0/3530810630 shutdown_connections 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 -- 172.21.15.112:0/3530810630 wait complete. 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 -- start start 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.042+0000 7f9165c52b80 1 Processor -- start 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287cde560 0x558287ce9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287ce73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce42e0 0x558287ce7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f9165c52b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558287ceac00 con 0x558287ce39c0 2024-05-10T10:42:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f9165c52b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558287be0ed0 con 0x558287cde560 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915895a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287ce73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f9165c52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558287cead80 con 0x558287ce42e0 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915895a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287ce73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52586/0 (socket says 172.21.15.112:52586) 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915895a700 1 -- 172.21.15.112:0/2913530624 learned_addr learned my addr 172.21.15.112:0/2913530624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915915b700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287cde560 0x558287ce9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915995c700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce42e0 0x558287ce7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f9165c52b80 10 client.0 _ll_register_callbacks cb 0x558287bbb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915915b700 1 -- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce42e0 msgr2=0x558287ce7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915915b700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce42e0 0x558287ce7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915915b700 1 -- 172.21.15.112:0/2913530624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 msgr2=0x558287ce73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915915b700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287ce73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915915b700 1 -- 172.21.15.112:0/2913530624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558287bde760 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.043+0000 7f915895a700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287ce73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f915915b700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287cde560 0x558287ce9fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91500a9c40 tx=0x7f91500b2660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687cf621a613fd9b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f913cff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f913cff9700 1 -- 172.21.15.112:0/2913530624 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915006d070 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f913cff9700 1 -- 172.21.15.112:0/2913530624 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91500a6070 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91500b2230 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f913cff9700 1 -- 172.21.15.112:0/2913530624 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f91500c3030 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 10 client.25439 fetch_fsmap learned FSMap version 31 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558287cf54f0 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f913cff9700 1 -- 172.21.15.112:0/2913530624 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91500b2230 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f913cff9700 1 -- 172.21.15.112:0/2913530624 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f915006f2c0 con 0x558287cde560 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 10 client.25439 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T10:42:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 10 client.25439 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 20 client.25439 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287cf79c0 0x558287cf9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 4 client.25439 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.044+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558287c35320 con 0x558287cf79c0 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.045+0000 7f915895a700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287cf79c0 0x558287cf9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.045+0000 7f915895a700 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287cf79c0 0x558287cf9db0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4adb6a126ab5743b server_cookie=bc3b6e18d2876c7f in_seq=0 out_seq=0 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.045+0000 7f913cff9700 10 client.25439 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.045+0000 7f913cff9700 1 -- 172.21.15.112:0/2913530624 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558287c35320 con 0x558287cf79c0 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.045+0000 7f913cff9700 10 client.25439 handle_command_reply: tid=0 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 client.25439 shutdown 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287cde560 msgr2=0x558287ce9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287cde560 0x558287ce9fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91500a9c40 tx=0x7f91500b2660 comp rx=0 tx=0).stop 2024-05-10T10:42:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 shutdown_connections 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287cf79c0 0x558287cf9db0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce42e0 0x558287ce7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f913cff9700 0 client.25439 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287cde560 0x558287ce9fe0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 --2- 172.21.15.112:0/2913530624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287ce73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 >> 172.21.15.112:0/2913530624 conn(0x558287c01b70 msgr2=0x558287ccab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 shutdown_connections 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 1 -- 172.21.15.112:0/2913530624 wait complete. 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.046+0000 7f9165c52b80 20 client.25439 trim_cache size 0 max 16384 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.061+0000 7f9165c52b80 1 Processor -- start 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.061+0000 7f9165c52b80 1 -- start start 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287ce4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f9165c52b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91541acff0 con 0x558287cde560 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f9165c52b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91541aeb60 con 0x558287ce42e0 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f9165c52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91541ad6c0 con 0x558287ce39c0 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52596/0 (socket says 172.21.15.112:52596) 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 -- 172.21.15.112:0/727963197 learned_addr learned my addr 172.21.15.112:0/727963197 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915895a700 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915995c700 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287ce4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 -- 172.21.15.112:0/727963197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 msgr2=0x558287ce6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 -- 172.21.15.112:0/727963197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 msgr2=0x558287ce4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287ce4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 -- 172.21.15.112:0/727963197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558287ce5940 con 0x558287cde560 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915995c700 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287ce4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f915915b700 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde020 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f91500a5ab0 tx=0x7f91500dffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc668cc22fd5dff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f913effd700 1 -- 172.21.15.112:0/727963197 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915006d070 con 0x558287cde560 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f913effd700 1 -- 172.21.15.112:0/727963197 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91500b2230 con 0x558287cde560 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.062+0000 7f913effd700 1 -- 172.21.15.112:0/727963197 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500aa180 con 0x558287cde560 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 -- 172.21.15.112:0/727963197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 msgr2=0x558287cde020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde020 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f91500a5ab0 tx=0x7f91500dffd0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 -- 172.21.15.112:0/727963197 shutdown_connections 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287ce4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287cde560 0x558287cde020 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 --2- 172.21.15.112:0/727963197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287ce39c0 0x558287ce6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 -- 172.21.15.112:0/727963197 >> 172.21.15.112:0/727963197 conn(0x558287c01b70 msgr2=0x558287ce24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 -- 172.21.15.112:0/727963197 shutdown_connections 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 -- 172.21.15.112:0/727963197 wait complete. 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.063+0000 7f9165c52b80 1 -- start start 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 1 Processor -- start 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287cde560 0x558287cfcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287cfd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cfa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91541ae460 con 0x558287ce39c0 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91541add90 con 0x558287ce42e0 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91541aafc0 con 0x558287cde560 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f9165c52b80 10 client.0 _ll_register_callbacks cb 0x558287bbb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287cfd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287cfd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52608/0 (socket says 172.21.15.112:52608) 2024-05-10T10:42:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 -- 172.21.15.112:0/3133733183 learned_addr learned my addr 172.21.15.112:0/3133733183 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 -- 172.21.15.112:0/3133733183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287cde560 msgr2=0x558287cfcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287cde560 0x558287cfcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 -- 172.21.15.112:0/3133733183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 msgr2=0x558287cfa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cfa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.064+0000 7f915895a700 1 -- 172.21.15.112:0/3133733183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558287cfad70 con 0x558287ce39c0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f915895a700 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287cfd480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9144009990 tx=0x7f9144009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9f5673634c78f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f913cff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f913cff9700 1 -- 172.21.15.112:0/3133733183 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91440363d0 con 0x558287ce39c0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91500b2230 con 0x558287ce39c0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f913cff9700 1 -- 172.21.15.112:0/3133733183 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9144036570 con 0x558287ce39c0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f913cff9700 1 -- 172.21.15.112:0/3133733183 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9144047050 con 0x558287ce39c0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 10 client.16038 fetch_fsmap learned FSMap version 31 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558287d61ae0 con 0x558287ce39c0 2024-05-10T10:42:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f913cff9700 1 -- 172.21.15.112:0/3133733183 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91500b2230 con 0x558287ce39c0 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f913cff9700 1 -- 172.21.15.112:0/3133733183 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f9144023af0 con 0x558287ce39c0 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 10 client.16038 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 20 client.16038 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287d62200 0x558287d64660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 4 client.16038 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.065+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558287c2f3c0 con 0x558287d62200 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f915915b700 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287d62200 0x558287d64660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.192 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f913cff9700 10 client.16038 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f915915b700 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287d62200 0x558287d64660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef81cd9da7c0cfd6 server_cookie=a151804d56392feb in_seq=0 out_seq=0 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f913cff9700 1 -- 172.21.15.112:0/3133733183 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558287c2f3c0 con 0x558287d62200 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f913cff9700 10 client.16038 handle_command_reply: tid=0 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 client.16038 shutdown 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 msgr2=0x558287cfd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287cfd480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9144009990 tx=0x7f9144009310 comp rx=0 tx=0).stop 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 shutdown_connections 2024-05-10T10:42:34.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x558287d62200 0x558287d64660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558287ce42e0 0x558287cfa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558287cde560 0x558287cfcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 --2- 172.21.15.112:0/3133733183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558287ce39c0 0x558287cfd480 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 >> 172.21.15.112:0/3133733183 conn(0x558287c01b70 msgr2=0x558287ce2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f913cff9700 0 client.16038 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 shutdown_connections 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 1 -- 172.21.15.112:0/3133733183 wait complete. 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.066+0000 7f9165c52b80 20 client.16038 trim_cache size 0 max 16384 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91400b2880 msgr2=0x7f91400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91400b2880 0x7f91400b4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f914400b2e0 tx=0x7f914401a040 comp rx=0 tx=0).stop 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91541023a0 msgr2=0x7f91541a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91541023a0 0x7f91541a18f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f915000e9c0 tx=0x7f915000ecd0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 shutdown_connections 2024-05-10T10:42:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9154108410 0x7f915419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91400b2880 0x7f91400b4d40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91541023a0 0x7f91541a18f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 --2- 172.21.15.112:0/4137416416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9154102d60 0x7f91541a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 >> 172.21.15.112:0/4137416416 conn(0x7f9154079350 msgr2=0x7f915406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 shutdown_connections 2024-05-10T10:42:34.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:34.067+0000 7f915b3bf700 1 -- 172.21.15.112:0/4137416416 wait complete. 2024-05-10T10:42:34.196 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -c -o file.o... yes 2024-05-10T10:42:34.196 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-05-10T10:42:34.196 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-05-10T10:42:34.306 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether -lc should be explicitly linked in... no 2024-05-10T10:42:34.437 INFO:tasks.workunit.client.0.smithi112.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-05-10T10:42:34.437 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to hardcode library paths into programs... immediate 2024-05-10T10:42:34.438 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether stripping libraries is possible... yes 2024-05-10T10:42:34.439 INFO:tasks.workunit.client.0.smithi112.stdout:checking if libtool supports shared libraries... yes 2024-05-10T10:42:34.439 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether to build shared libraries... yes 2024-05-10T10:42:34.439 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether to build static libraries... yes 2024-05-10T10:42:34.447 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... (cached) gcc 2024-05-10T10:42:34.502 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-10T10:42:34.502 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-10T10:42:34.503 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-10T10:42:34.503 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gmake... /usr/bin/gmake 2024-05-10T10:42:34.504 INFO:tasks.workunit.client.0.smithi112.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-05-10T10:42:34.504 INFO:tasks.workunit.client.0.smithi112.stdout:checking for tar... /bin/tar 2024-05-10T10:42:34.505 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gzip... /bin/gzip 2024-05-10T10:42:34.525 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc -MM is supported... yes 2024-05-10T10:42:34.526 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sort... /bin/sort 2024-05-10T10:42:34.526 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether ln -s works... yes 2024-05-10T10:42:34.527 INFO:tasks.workunit.client.0.smithi112.stdout:checking for rpm... /bin/rpm 2024-05-10T10:42:34.532 INFO:tasks.workunit.client.0.smithi112.stdout:checking for rpmbuild... no 2024-05-10T10:42:34.617 INFO:tasks.workunit.client.0.smithi112.stdout:checking assert.h usability... yes 2024-05-10T10:42:34.640 INFO:tasks.workunit.client.0.smithi112.stdout:checking assert.h presence... yes 2024-05-10T10:42:34.640 INFO:tasks.workunit.client.0.smithi112.stdout:checking for assert.h... yes 2024-05-10T10:42:34.724 INFO:tasks.workunit.client.0.smithi112.stdout:checking bstring.h usability... no 2024-05-10T10:42:34.768 INFO:tasks.workunit.client.0.smithi112.stdout:checking bstring.h presence... no 2024-05-10T10:42:34.768 INFO:tasks.workunit.client.0.smithi112.stdout:checking for bstring.h... no 2024-05-10T10:42:34.837 INFO:tasks.workunit.client.0.smithi112.stdout:checking libgen.h usability... yes 2024-05-10T10:42:34.862 INFO:tasks.workunit.client.0.smithi112.stdout:checking libgen.h presence... yes 2024-05-10T10:42:34.862 INFO:tasks.workunit.client.0.smithi112.stdout:checking for libgen.h... yes 2024-05-10T10:42:34.967 INFO:tasks.workunit.client.0.smithi112.stdout:checking dirent.h usability... yes 2024-05-10T10:42:35.034 INFO:tasks.workunit.client.0.smithi112.stdout:checking dirent.h presence... yes 2024-05-10T10:42:35.034 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dirent.h... yes 2024-05-10T10:42:35.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:34 smithi112 ceph-mon[30100]: pgmap v500: 129 pgs: 129 active+clean; 26 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.1 MiB/s wr, 389 op/s 2024-05-10T10:42:35.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:42:35.129 INFO:tasks.workunit.client.0.smithi112.stdout:checking errno.h usability... yes 2024-05-10T10:42:35.154 INFO:tasks.workunit.client.0.smithi112.stdout:checking errno.h presence... yes 2024-05-10T10:42:35.154 INFO:tasks.workunit.client.0.smithi112.stdout:checking for errno.h... yes 2024-05-10T10:42:35.230 INFO:tasks.workunit.client.0.smithi112.stdout:checking malloc.h usability... yes 2024-05-10T10:42:35.260 INFO:tasks.workunit.client.0.smithi112.stdout:checking malloc.h presence... yes 2024-05-10T10:42:35.260 INFO:tasks.workunit.client.0.smithi112.stdout:checking for malloc.h... yes 2024-05-10T10:42:35.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:34 smithi148 ceph-mon[33568]: pgmap v500: 129 pgs: 129 active+clean; 26 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.1 MiB/s wr, 389 op/s 2024-05-10T10:42:35.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:34 smithi195 ceph-mon[33773]: pgmap v500: 129 pgs: 129 active+clean; 26 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.1 MiB/s wr, 389 op/s 2024-05-10T10:42:35.346 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid.h usability... no 2024-05-10T10:42:35.347 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:35.394 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid.h presence... no 2024-05-10T10:42:35.394 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid.h... no 2024-05-10T10:42:35.489 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid/uuid.h usability... yes 2024-05-10T10:42:35.512 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid/uuid.h presence... yes 2024-05-10T10:42:35.512 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid/uuid.h... yes 2024-05-10T10:42:35.604 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/uuid.h usability... no 2024-05-10T10:42:35.647 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/uuid.h presence... no 2024-05-10T10:42:35.648 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/uuid.h... no 2024-05-10T10:42:35.712 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/file.h usability... yes 2024-05-10T10:42:35.735 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/file.h presence... yes 2024-05-10T10:42:35.735 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/file.h... yes 2024-05-10T10:42:35.820 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fcntl.h usability... yes 2024-05-10T10:42:35.843 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fcntl.h presence... yes 2024-05-10T10:42:35.844 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fcntl.h... yes 2024-05-10T10:42:35.936 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/syssgi.h usability... no 2024-05-10T10:42:35.979 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/syssgi.h presence... no 2024-05-10T10:42:35.979 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/syssgi.h... no 2024-05-10T10:42:36.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:35 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:36.188 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/param.h usability... yes 2024-05-10T10:42:36.215 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/param.h presence... yes 2024-05-10T10:42:36.215 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/param.h... yes 2024-05-10T10:42:36.240 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... (cached) yes 2024-05-10T10:42:36.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:35 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:36.307 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/statvfs.h usability... yes 2024-05-10T10:42:36.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:35 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:36.330 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/statvfs.h presence... yes 2024-05-10T10:42:36.330 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/statvfs.h... yes 2024-05-10T10:42:36.406 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h usability... yes 2024-05-10T10:42:36.432 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h presence... yes 2024-05-10T10:42:36.432 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/time.h... yes 2024-05-10T10:42:36.509 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/ioctl.h usability... yes 2024-05-10T10:42:36.532 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/ioctl.h presence... yes 2024-05-10T10:42:36.532 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/ioctl.h... yes 2024-05-10T10:42:36.632 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/wait.h usability... yes 2024-05-10T10:42:36.657 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/wait.h presence... yes 2024-05-10T10:42:36.657 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/wait.h... yes 2024-05-10T10:42:36.687 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... (cached) yes 2024-05-10T10:42:36.706 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... (cached) yes 2024-05-10T10:42:36.773 INFO:tasks.workunit.client.0.smithi112.stdout:checking err.h usability... yes 2024-05-10T10:42:36.798 INFO:tasks.workunit.client.0.smithi112.stdout:checking err.h presence... yes 2024-05-10T10:42:36.799 INFO:tasks.workunit.client.0.smithi112.stdout:checking for err.h... yes 2024-05-10T10:42:36.874 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/falloc.h usability... yes 2024-05-10T10:42:36.896 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/falloc.h presence... yes 2024-05-10T10:42:36.897 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/falloc.h... yes 2024-05-10T10:42:36.980 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-05-10T10:42:37.023 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-05-10T10:42:37.023 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fs/xfs_fsops.h... no 2024-05-10T10:42:37.096 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_itable.h usability... no 2024-05-10T10:42:37.137 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_itable.h presence... no 2024-05-10T10:42:37.137 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fs/xfs_itable.h... no 2024-05-10T10:42:37.197 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:36 smithi112 ceph-mon[30100]: pgmap v501: 129 pgs: 129 active+clean; 28 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 915 KiB/s wr, 354 op/s 2024-05-10T10:42:37.222 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/platform_defs.h usability... no 2024-05-10T10:42:37.268 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/platform_defs.h presence... no 2024-05-10T10:42:37.268 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/platform_defs.h... no 2024-05-10T10:42:37.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:36 smithi148 ceph-mon[33568]: pgmap v501: 129 pgs: 129 active+clean; 28 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 915 KiB/s wr, 354 op/s 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 -- 172.21.15.112:0/1715217950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 msgr2=0x7fc74c107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 --2- 172.21.15.112:0/1715217950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 0x7fc74c107fc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0099c0 tx=0x7fc73c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 -- 172.21.15.112:0/1715217950 shutdown_connections 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 --2- 172.21.15.112:0/1715217950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc74c102540 0x7fc74c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 --2- 172.21.15.112:0/1715217950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 0x7fc74c107fc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 --2- 172.21.15.112:0/1715217950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 0x7fc74c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 -- 172.21.15.112:0/1715217950 >> 172.21.15.112:0/1715217950 conn(0x7fc74c078a10 msgr2=0x7fc74c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 -- 172.21.15.112:0/1715217950 shutdown_connections 2024-05-10T10:42:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.313+0000 7fc7526d0700 1 -- 172.21.15.112:0/1715217950 wait complete. 2024-05-10T10:42:37.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:36 smithi195 ceph-mon[33773]: pgmap v501: 129 pgs: 129 active+clean; 28 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 915 KiB/s wr, 354 op/s 2024-05-10T10:42:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.314+0000 7fc7526d0700 1 Processor -- start 2024-05-10T10:42:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.314+0000 7fc7526d0700 1 -- start start 2024-05-10T10:42:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc7526d0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 0x7fc74c1037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc7526d0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc74c102540 0x7fc74c103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc7526d0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 0x7fc74c104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc7526d0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c112430 con 0x7fc74c107be0 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc7526d0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1122b0 con 0x7fc74c101b80 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc7526d0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1125b0 con 0x7fc74c102540 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 0x7fc74c1037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 0x7fc74c1037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:38804/0 (socket says 172.21.15.112:38804) 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74bfff700 1 -- 172.21.15.112:0/2503922734 learned_addr learned my addr 172.21.15.112:0/2503922734 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc74c102540 0x7fc74c103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc750c6d700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 0x7fc74c104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74b7fe700 1 -- 172.21.15.112:0/2503922734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 msgr2=0x7fc74c1037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 0x7fc74c1037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74b7fe700 1 -- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 msgr2=0x7fc74c104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 0x7fc74c104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74b7fe700 1 -- 172.21.15.112:0/2503922734 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c1a7ee0 con 0x7fc74c102540 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc74bfff700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 0x7fc74c1037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.315+0000 7fc750c6d700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 0x7fc74c104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.316+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc74c102540 0x7fc74c103ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc74000d750 tx=0x7fc74000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.316+0000 7fc7497fa700 1 -- 172.21.15.112:0/2503922734 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740014030 con 0x7fc74c102540 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.316+0000 7fc7497fa700 1 -- 172.21.15.112:0/2503922734 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7400044b0 con 0x7fc74c102540 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.316+0000 7fc7497fa700 1 -- 172.21.15.112:0/2503922734 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740005260 con 0x7fc74c102540 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.316+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c1a81d0 con 0x7fc74c102540 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.316+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c1a86c0 con 0x7fc74c102540 2024-05-10T10:42:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.318+0000 7fc7497fa700 1 -- 172.21.15.112:0/2503922734 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc7400049e0 con 0x7fc74c102540 2024-05-10T10:42:37.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.319+0000 7fc7497fa700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc7340b2880 0x7fc7340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.319+0000 7fc74bfff700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc7340b2880 0x7fc7340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.319+0000 7fc7497fa700 1 -- 172.21.15.112:0/2503922734 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc7400e2170 con 0x7fc74c102540 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc74bfff700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc7340b2880 0x7fc7340b4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009990 tx=0x7fc73c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 Processor -- start 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 -- start start 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab61b0 0x5574b9ab6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab0d50 0x5574b9ab1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574b9a02940 con 0x5574b9ab0d50 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5574b9967da0 con 0x5574b9ab6ad0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574b99fe900 con 0x5574b9ab61b0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab61b0 0x5574b9ab6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc750c6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab0d50 0x5574b9ab1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab61b0 0x5574b9ab6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:39286/0 (socket says 172.21.15.112:39286) 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc74bfff700 1 -- 172.21.15.112:0/1315489763 learned_addr learned my addr 172.21.15.112:0/1315489763 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.320+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc74bfff700 1 -- 172.21.15.112:0/1315489763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 msgr2=0x5574b9ab0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc74bfff700 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc74bfff700 1 -- 172.21.15.112:0/1315489763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab0d50 msgr2=0x5574b9ab1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc74bfff700 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab0d50 0x5574b9ab1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc74bfff700 1 -- 172.21.15.112:0/1315489763 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b994f8d0 con 0x5574b9ab61b0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc74bfff700 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab61b0 0x5574b9ab6590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000c00 tx=0x7fc73c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fc923121d1803b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc7327fc700 1 -- 172.21.15.112:0/1315489763 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0037c0 con 0x5574b9ab61b0 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc7327fc700 1 -- 172.21.15.112:0/1315489763 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc73c003df0 con 0x5574b9ab61b0 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.321+0000 7fc7327fc700 1 -- 172.21.15.112:0/1315489763 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c024b90 con 0x5574b9ab61b0 2024-05-10T10:42:37.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 -- 172.21.15.112:0/1315489763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab61b0 msgr2=0x5574b9ab6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab61b0 0x5574b9ab6590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000c00 tx=0x7fc73c020000 comp rx=0 tx=0).stop 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 -- 172.21.15.112:0/1315489763 shutdown_connections 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab0d50 0x5574b9ab1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab61b0 0x5574b9ab6590 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/1315489763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 -- 172.21.15.112:0/1315489763 >> 172.21.15.112:0/1315489763 conn(0x5574b99dcb70 msgr2=0x5574b9aab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 -- 172.21.15.112:0/1315489763 shutdown_connections 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 -- 172.21.15.112:0/1315489763 wait complete. 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.322+0000 7fc75cf63b80 1 -- start start 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 1 Processor -- start 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9abc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9aba180 0x5574b9aba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574b9abd390 con 0x5574b9aba180 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5574b99bbed0 con 0x5574b9ab6ad0 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574b9abd510 con 0x5574b9ab0d50 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc75cf63b80 10 client.0 _ll_register_callbacks cb 0x5574b99965b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc750c6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9aba180 0x5574b9aba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc750c6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9aba180 0x5574b9aba5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33462/0 (socket says 172.21.15.112:33462) 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc750c6d700 1 -- 172.21.15.112:0/3564457106 learned_addr learned my addr 172.21.15.112:0/3564457106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.323+0000 7fc74bfff700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9abc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc74b7fe700 1 -- 172.21.15.112:0/3564457106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 msgr2=0x5574b9abc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9abc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc74b7fe700 1 -- 172.21.15.112:0/3564457106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9aba180 msgr2=0x5574b9aba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9aba180 0x5574b9aba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc74b7fe700 1 -- 172.21.15.112:0/3564457106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b99b9760 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc750c6d700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9aba180 0x5574b9aba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7400df930 tx=0x7fc7400a9bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90cdd0712d64a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc72b7fe700 1 -- 172.21.15.112:0/3564457106 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7400c3060 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc72b7fe700 1 -- 172.21.15.112:0/3564457106 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc7400a9870 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc73c003df0 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc72b7fe700 1 -- 172.21.15.112:0/3564457106 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc74001d040 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc75cf63b80 10 client.25451 fetch_fsmap learned FSMap version 32 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.324+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574b9ac8280 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc72b7fe700 1 -- 172.21.15.112:0/3564457106 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc73c003df0 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc72b7fe700 1 -- 172.21.15.112:0/3564457106 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc74006e070 con 0x5574b9ab6ad0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc75cf63b80 10 client.25451 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc75cf63b80 10 client.25451 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc75cf63b80 20 client.25451 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9aca810 0x5574b9accc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc75cf63b80 4 client.25451 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:37.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574b9a10320 con 0x5574b9aca810 2024-05-10T10:42:37.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.325+0000 7fc74bfff700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9aca810 0x5574b9accc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.326+0000 7fc72b7fe700 10 client.25451 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:37.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.326+0000 7fc74bfff700 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9aca810 0x5574b9accc00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=276fdd61a4a260a server_cookie=fd72af4472da8d69 in_seq=0 out_seq=0 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.326+0000 7fc72b7fe700 1 -- 172.21.15.112:0/3564457106 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574b9a10320 con 0x5574b9aca810 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.326+0000 7fc72b7fe700 10 client.25451 handle_command_reply: tid=0 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.326+0000 7fc75cf63b80 1 client.25451 shutdown 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 msgr2=0x5574b9ab9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7400df930 tx=0x7fc7400a9bc0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 shutdown_connections 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9aca810 0x5574b9accc00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9aba180 0x5574b9aba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9abc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/3564457106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 >> 172.21.15.112:0/3564457106 conn(0x5574b99dcb70 msgr2=0x5574b9aa5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc72b7fe700 0 client.25451 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:37.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 shutdown_connections 2024-05-10T10:42:37.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 1 -- 172.21.15.112:0/3564457106 wait complete. 2024-05-10T10:42:37.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.327+0000 7fc75cf63b80 20 client.25451 trim_cache size 0 max 16384 2024-05-10T10:42:37.337 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfs/ioctl.h usability... no 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 Processor -- start 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 -- start start 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9ab0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9aba180 0x5574b9aa5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1b12f0 con 0x5574b9ab6ad0 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1b2e90 con 0x5574b9aba180 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1b19c0 con 0x5574b9ab0d50 2024-05-10T10:42:37.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33476/0 (socket says 172.21.15.112:33476) 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 -- 172.21.15.112:0/2018249267 learned_addr learned my addr 172.21.15.112:0/2018249267 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74bfff700 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9ab0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 -- 172.21.15.112:0/2018249267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 msgr2=0x5574b9ab0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9ab0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 -- 172.21.15.112:0/2018249267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9aba180 msgr2=0x5574b9aa5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9aba180 0x5574b9aa5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.345+0000 7fc74b7fe700 1 -- 172.21.15.112:0/2018249267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b9ababe0 con 0x5574b9ab6ad0 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.346+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc7400a6670 tx=0x7fc7400a9300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491c7d663668c313 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.346+0000 7fc731ffb700 1 -- 172.21.15.112:0/2018249267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7400ad7b0 con 0x5574b9ab6ad0 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.346+0000 7fc731ffb700 1 -- 172.21.15.112:0/2018249267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7400b2260 con 0x5574b9ab6ad0 2024-05-10T10:42:37.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.346+0000 7fc731ffb700 1 -- 172.21.15.112:0/2018249267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7400b0b30 con 0x5574b9ab6ad0 2024-05-10T10:42:37.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.346+0000 7fc75cf63b80 1 -- 172.21.15.112:0/2018249267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 msgr2=0x5574b9ab9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.346+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc7400a6670 tx=0x7fc7400a9300 comp rx=0 tx=0).stop 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 -- 172.21.15.112:0/2018249267 shutdown_connections 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9aba180 0x5574b9aa5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9ab0d50 0x5574b9ab0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/2018249267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ab9c00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 -- 172.21.15.112:0/2018249267 >> 172.21.15.112:0/2018249267 conn(0x5574b99dcb70 msgr2=0x5574b9ab94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 -- 172.21.15.112:0/2018249267 shutdown_connections 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 -- 172.21.15.112:0/2018249267 wait complete. 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 -- start start 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.347+0000 7fc75cf63b80 1 Processor -- start 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab0d50 0x5574b9ad2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ac27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc75cf63b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9aba180 0x5574b9ac2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1b2760 con 0x5574b9ab6ad0 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1b2090 con 0x5574b9ab0d50 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc75cf63b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1af290 con 0x5574b9aba180 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc75cf63b80 10 client.0 _ll_register_callbacks cb 0x5574b99965b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ac27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ac27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33486/0 (socket says 172.21.15.112:33486) 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74b7fe700 1 -- 172.21.15.112:0/274432767 learned_addr learned my addr 172.21.15.112:0/274432767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74b7fe700 1 -- 172.21.15.112:0/274432767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9aba180 msgr2=0x5574b9ac2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc750c6d700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9aba180 0x5574b9ac2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9aba180 0x5574b9ac2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74bfff700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab0d50 0x5574b9ad2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74b7fe700 1 -- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab0d50 msgr2=0x5574b9ad2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.348+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab0d50 0x5574b9ad2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc74b7fe700 1 -- 172.21.15.112:0/274432767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b9ad1490 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc750c6d700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9aba180 0x5574b9ac2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc74b7fe700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ac27a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc74000d720 tx=0x7fc7400a8e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17eaefb422948bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc72b7fe700 1 -- 172.21.15.112:0/274432767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7400c3070 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc72b7fe700 1 -- 172.21.15.112:0/274432767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc7400ca380 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7400b2260 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc72b7fe700 1 -- 172.21.15.112:0/274432767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc740004070 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc75cf63b80 10 client.16056 fetch_fsmap learned FSMap version 32 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574b9ad1490 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.349+0000 7fc72b7fe700 1 -- 172.21.15.112:0/274432767 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7400b2260 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc72b7fe700 1 -- 172.21.15.112:0/274432767 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc74006e040 con 0x5574b9ab6ad0 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc75cf63b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T10:42:37.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc75cf63b80 10 client.16056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc75cf63b80 20 client.16056 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9b34520 0x5574b9b36940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc75cf63b80 4 client.16056 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574b9a0a3c0 con 0x5574b9b34520 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc74bfff700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9b34520 0x5574b9b36940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc72b7fe700 10 client.16056 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.350+0000 7fc74bfff700 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9b34520 0x5574b9b36940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aab021dff8422706 server_cookie=b1640163f3a1560e in_seq=0 out_seq=0 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc72b7fe700 1 -- 172.21.15.112:0/274432767 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574b9a0a3c0 con 0x5574b9b34520 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc72b7fe700 10 client.16056 handle_command_reply: tid=0 2024-05-10T10:42:37.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 client.16056 shutdown 2024-05-10T10:42:37.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 msgr2=0x5574b9ac27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ac27a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc74000d720 tx=0x7fc7400a8e80 comp rx=0 tx=0).stop 2024-05-10T10:42:37.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 shutdown_connections 2024-05-10T10:42:37.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b9aba180 0x5574b9ac2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574b9ab0d50 0x5574b9ad2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5574b9b34520 0x5574b9b36940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 --2- 172.21.15.112:0/274432767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574b9ab6ad0 0x5574b9ac27a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc72b7fe700 0 client.16056 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:42:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 >> 172.21.15.112:0/274432767 conn(0x5574b99dcb70 msgr2=0x5574b9ab3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 shutdown_connections 2024-05-10T10:42:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 1 -- 172.21.15.112:0/274432767 wait complete. 2024-05-10T10:42:37.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.351+0000 7fc75cf63b80 20 client.16056 trim_cache size 0 max 16384 2024-05-10T10:42:37.357 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc7340b2880 msgr2=0x7fc7340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc7340b2880 0x7fc7340b4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009990 tx=0x7fc73c005960 comp rx=0 tx=0).stop 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc74c102540 msgr2=0x7fc74c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc74c102540 0x7fc74c103ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc74000d750 tx=0x7fc74000da60 comp rx=0 tx=0).stop 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 shutdown_connections 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc74c107be0 0x7fc74c104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c101b80 0x7fc74c1037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc7340b2880 0x7fc7340b4d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 --2- 172.21.15.112:0/2503922734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc74c102540 0x7fc74c103ce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 >> 172.21.15.112:0/2503922734 conn(0x7fc74c078a10 msgr2=0x7fc74c100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 shutdown_connections 2024-05-10T10:42:37.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:37.352+0000 7fc7526d0700 1 -- 172.21.15.112:0/2503922734 wait complete. 2024-05-10T10:42:37.381 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfs/ioctl.h presence... no 2024-05-10T10:42:37.382 INFO:tasks.workunit.client.0.smithi112.stdout:checking for btrfs/ioctl.h... no 2024-05-10T10:42:37.446 INFO:tasks.workunit.client.0.smithi112.stdout:checking cifs/ioctl.h usability... no 2024-05-10T10:42:37.487 INFO:tasks.workunit.client.0.smithi112.stdout:checking cifs/ioctl.h presence... no 2024-05-10T10:42:37.488 INFO:tasks.workunit.client.0.smithi112.stdout:checking for cifs/ioctl.h... no 2024-05-10T10:42:37.584 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/mman.h usability... yes 2024-05-10T10:42:37.607 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/mman.h presence... yes 2024-05-10T10:42:37.607 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/mman.h... yes 2024-05-10T10:42:37.691 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/ext4.h usability... no 2024-05-10T10:42:37.732 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/ext4.h presence... no 2024-05-10T10:42:37.732 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/ext4.h... no 2024-05-10T10:42:37.780 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xfs_log_format.h... no 2024-05-10T10:42:37.863 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid_compare... no 2024-05-10T10:42:37.926 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid_compare in -luuid... yes 2024-05-10T10:42:38.145 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xfs.h... yes 2024-05-10T10:42:38.221 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/libxfs.h... no 2024-05-10T10:42:38.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:42:38.266 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether xlog_assign_lsn is declared... no 2024-05-10T10:42:38.317 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xqm.h... yes 2024-05-10T10:42:38.400 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfsctl from xfs/xfs.h... ok 2024-05-10T10:42:38.458 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/handle.h usability... yes 2024-05-10T10:42:38.481 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/handle.h presence... yes 2024-05-10T10:42:38.481 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/handle.h... yes 2024-05-10T10:42:38.515 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:38.579 INFO:tasks.workunit.client.0.smithi112.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-05-10T10:42:38.650 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/acl.h usability... yes 2024-05-10T10:42:38.675 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/acl.h presence... yes 2024-05-10T10:42:38.675 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/acl.h... yes 2024-05-10T10:42:38.752 INFO:tasks.workunit.client.0.smithi112.stdout:checking acl/libacl.h usability... yes 2024-05-10T10:42:38.778 INFO:tasks.workunit.client.0.smithi112.stdout:checking acl/libacl.h presence... yes 2024-05-10T10:42:38.778 INFO:tasks.workunit.client.0.smithi112.stdout:checking for acl/libacl.h... yes 2024-05-10T10:42:38.848 INFO:tasks.workunit.client.0.smithi112.stdout:checking for acl_init in -lacl... yes 2024-05-10T10:42:38.915 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm-ndbm.h usability... no 2024-05-10T10:42:38.959 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm-ndbm.h presence... no 2024-05-10T10:42:38.959 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gdbm-ndbm.h... no 2024-05-10T10:42:39.014 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm/ndbm.h usability... no 2024-05-10T10:42:39.054 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm/ndbm.h presence... no 2024-05-10T10:42:39.054 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gdbm/ndbm.h... no 2024-05-10T10:42:39.125 INFO:tasks.workunit.client.0.smithi112.stdout:checking ndbm.h usability... no 2024-05-10T10:42:39.173 INFO:tasks.workunit.client.0.smithi112.stdout:checking ndbm.h presence... no 2024-05-10T10:42:39.174 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ndbm.h... no 2024-05-10T10:42:39.241 INFO:tasks.workunit.client.0.smithi112.stdout:checking libaio.h usability... yes 2024-05-10T10:42:39.267 INFO:tasks.workunit.client.0.smithi112.stdout:checking libaio.h presence... yes 2024-05-10T10:42:39.267 INFO:tasks.workunit.client.0.smithi112.stdout:checking for libaio.h... yes 2024-05-10T10:42:39.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:38 smithi148 ceph-mon[33568]: pgmap v502: 129 pgs: 129 active+clean; 28 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 811 KiB/s wr, 414 op/s 2024-05-10T10:42:39.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:38 smithi195 ceph-mon[33773]: pgmap v502: 129 pgs: 129 active+clean; 28 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 811 KiB/s wr, 414 op/s 2024-05-10T10:42:39.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:38 smithi112 ceph-mon[30100]: pgmap v502: 129 pgs: 129 active+clean; 28 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 811 KiB/s wr, 414 op/s 2024-05-10T10:42:39.347 INFO:tasks.workunit.client.0.smithi112.stdout:checking liburing.h usability... no 2024-05-10T10:42:39.388 INFO:tasks.workunit.client.0.smithi112.stdout:checking liburing.h presence... no 2024-05-10T10:42:39.388 INFO:tasks.workunit.client.0.smithi112.stdout:checking for liburing.h... no 2024-05-10T10:42:39.440 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/capability.h usability... no 2024-05-10T10:42:39.484 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/capability.h presence... no 2024-05-10T10:42:39.484 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/capability.h... no 2024-05-10T10:42:39.538 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fiemap.h usability... yes 2024-05-10T10:42:39.561 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fiemap.h presence... yes 2024-05-10T10:42:39.561 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/fiemap.h... yes 2024-05-10T10:42:39.723 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fallocate... yes 2024-05-10T10:42:39.906 INFO:tasks.workunit.client.0.smithi112.stdout:checking for open_by_handle_at... yes 2024-05-10T10:42:39.979 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fs.h usability... yes 2024-05-10T10:42:40.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:39 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:42:40.086 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fs.h presence... yes 2024-05-10T10:42:40.087 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/fs.h... yes 2024-05-10T10:42:40.155 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfsutil.h usability... no 2024-05-10T10:42:40.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:39 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:42:40.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:39 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:42:40.408 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfsutil.h presence... no 2024-05-10T10:42:40.408 INFO:tasks.workunit.client.0.smithi112.stdout:checking for btrfsutil.h... no 2024-05-10T10:42:40.486 INFO:tasks.workunit.client.0.smithi112.stdout:checking for copy_file_range... yes 2024-05-10T10:42:40.594 INFO:tasks.workunit.client.0.smithi112.stdout:checking for SEEK_DATA... yes 2024-05-10T10:42:40.680 INFO:tasks.workunit.client.0.smithi112.stdout:checking for BMV_OF_SHARED... yes 2024-05-10T10:42:40.767 INFO:tasks.workunit.client.0.smithi112.stdout:checking for nftw... yes 2024-05-10T10:42:40.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1986567256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108880 msgr2=0x7f89f4110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1986567256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108880 0x7f89f4110400 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00b0d0 tx=0x7f89ec00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1986567256 shutdown_connections 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1986567256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108880 0x7f89f4110400 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1986567256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89f41037b0 0x7f89f4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1986567256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f4104160 0x7f89f4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1986567256 >> 172.21.15.112:0/1986567256 conn(0x7f89f407a530 msgr2=0x7f89f407a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1986567256 shutdown_connections 2024-05-10T10:42:40.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.809+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1986567256 wait complete. 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 Processor -- start 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 -- start start 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41037b0 0x7f89f410f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89f4104160 0x7f89f410fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f4108880 0x7f89f41a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4113520 con 0x7f89f41037b0 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41133a0 con 0x7f89f4104160 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.810+0000 7f89fbeb1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41136a0 con 0x7f89f4108880 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41037b0 0x7f89f410f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41037b0 0x7f89f410f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33498/0 (socket says 172.21.15.112:33498) 2024-05-10T10:42:40.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1619109144 learned_addr learned my addr 172.21.15.112:0/1619109144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89fa44e700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f4108880 0x7f89f41a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f944c700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89f4104160 0x7f89f410fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1619109144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f4108880 msgr2=0x7f89f41a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f4108880 0x7f89f41a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1619109144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89f4104160 msgr2=0x7f89f410fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89f4104160 0x7f89f410fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1619109144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f41a4f70 con 0x7f89f41037b0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41037b0 0x7f89f410f590 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f89f000d950 tx=0x7f89f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89eaffd700 1 -- 172.21.15.112:0/1619109144 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0016070 con 0x7f89f41037b0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f41a5260 con 0x7f89f41037b0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89eaffd700 1 -- 172.21.15.112:0/1619109144 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89f0009dd0 con 0x7f89f41037b0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89eaffd700 1 -- 172.21.15.112:0/1619109144 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f000f3c0 con 0x7f89f41037b0 2024-05-10T10:42:40.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.811+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f41acae0 con 0x7f89f41037b0 2024-05-10T10:42:40.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.814+0000 7f89eaffd700 1 -- 172.21.15.112:0/1619109144 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f89f0019070 con 0x7f89f41037b0 2024-05-10T10:42:40.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.815+0000 7f89eaffd700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f89e00b2880 0x7f89e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.815+0000 7f8a06744b80 1 Processor -- start 2024-05-10T10:42:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f944c700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f89e00b2880 0x7f89e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89eaffd700 1 -- 172.21.15.112:0/1619109144 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f89f00e2200 con 0x7f89f41037b0 2024-05-10T10:42:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f8a06744b80 1 -- start start 2024-05-10T10:42:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f074964d0 0x559f074968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f07496df0 0x559f0749a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f8a06744b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f073e8860 con 0x559f07496df0 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f8a06744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559f073877c0 con 0x559f0749ac80 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f8a06744b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559f073e4820 con 0x559f074964d0 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f944c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f07496df0 0x559f0749a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f944c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f07496df0 0x559f0749a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33504/0 (socket says 172.21.15.112:33504) 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f944c700 1 -- 172.21.15.112:0/3559154529 learned_addr learned my addr 172.21.15.112:0/3559154529 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89fa44e700 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f074964d0 0x559f074968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f944c700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f89e00b2880 0x7f89e00b4d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f89f4103280 tx=0x7f89e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f9c4d700 1 -- 172.21.15.112:0/3559154529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 msgr2=0x559f0749b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f9c4d700 1 -- 172.21.15.112:0/3559154529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f07496df0 msgr2=0x559f0749a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f07496df0 0x559f0749a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.816+0000 7f89f9c4d700 1 -- 172.21.15.112:0/3559154529 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f073358d0 con 0x559f074964d0 2024-05-10T10:42:40.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.817+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f074964d0 0x559f074968b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89f00df9c0 tx=0x7f89f00a90a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd5caa6aa801d9f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.817+0000 7f89dffff700 1 -- 172.21.15.112:0/3559154529 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0020070 con 0x559f074964d0 2024-05-10T10:42:40.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.817+0000 7f89dffff700 1 -- 172.21.15.112:0/3559154529 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89f00a9490 con 0x559f074964d0 2024-05-10T10:42:40.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.817+0000 7f89dffff700 1 -- 172.21.15.112:0/3559154529 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00b1e60 con 0x559f074964d0 2024-05-10T10:42:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 -- 172.21.15.112:0/3559154529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f074964d0 msgr2=0x559f074968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f074964d0 0x559f074968b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89f00df9c0 tx=0x7f89f00a90a0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 -- 172.21.15.112:0/3559154529 shutdown_connections 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f074964d0 0x559f074968b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 --2- 172.21.15.112:0/3559154529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f07496df0 0x559f0749a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 -- 172.21.15.112:0/3559154529 >> 172.21.15.112:0/3559154529 conn(0x559f073c2100 msgr2=0x559f07490c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 -- 172.21.15.112:0/3559154529 shutdown_connections 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 -- 172.21.15.112:0/3559154529 wait complete. 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 -- start start 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.818+0000 7f8a06744b80 1 Processor -- start 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f0749f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f8a06744b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f074a2dd0 con 0x559f074964d0 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f8a06744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559f0734ec30 con 0x559f0749ac80 2024-05-10T10:42:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f8a06744b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559f074a2f50 con 0x559f07496df0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f8a06744b80 10 client.0 _ll_register_callbacks cb 0x559f07380920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33506/0 (socket says 172.21.15.112:33506) 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1298229989 learned_addr learned my addr 172.21.15.112:0/1298229989 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f944c700 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f0749f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89fa44e700 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 msgr2=0x559f0749f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f0749f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1298229989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 msgr2=0x559f0749fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 -- 172.21.15.112:0/1298229989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f07384690 con 0x559f074964d0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074a2140 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f89f00a6700 tx=0x7f89f00109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda5cb8be918ee50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.819+0000 7f89dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f89dd7fa700 1 -- 172.21.15.112:0/1298229989 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0020040 con 0x559f074964d0 2024-05-10T10:42:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f00a9490 con 0x559f074964d0 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f89dd7fa700 1 -- 172.21.15.112:0/1298229989 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f89f00b1e60 con 0x559f074964d0 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f89dd7fa700 1 -- 172.21.15.112:0/1298229989 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f89f00b2060 con 0x559f074964d0 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 10 client.16068 fetch_fsmap learned FSMap version 33 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559f074ad6a0 con 0x559f074964d0 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f89dd7fa700 1 -- 172.21.15.112:0/1298229989 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89f00a9490 con 0x559f074964d0 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f89dd7fa700 1 -- 172.21.15.112:0/1298229989 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f89f00ca310 con 0x559f074964d0 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 10 client.16068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 20 client.16068 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f074b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 4 client.16068 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.820+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f07462650 con 0x559f074afb70 2024-05-10T10:42:40.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.821+0000 7f89f944c700 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f074b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.821+0000 7f89f944c700 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f074b1f60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3cebe4ca526dcd4 server_cookie=7a5abc2ccc8731c2 in_seq=0 out_seq=0 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.821+0000 7f89dd7fa700 10 client.16068 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f89dd7fa700 1 -- 172.21.15.112:0/1298229989 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f07462650 con 0x559f074afb70 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f89dd7fa700 10 client.16068 handle_command_reply: tid=0 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 client.16068 shutdown 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 msgr2=0x559f074a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074a2140 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f89f00a6700 tx=0x7f89f00109e0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 shutdown_connections 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f074b1f60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0749ac80 0x559f0749fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f89dd7fa700 0 client.16068 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074a2140 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 --2- 172.21.15.112:0/1298229989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f0749f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 >> 172.21.15.112:0/1298229989 conn(0x559f073c2100 msgr2=0x559f0748b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 shutdown_connections 2024-05-10T10:42:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 1 -- 172.21.15.112:0/1298229989 wait complete. 2024-05-10T10:42:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.822+0000 7f8a06744b80 20 client.16068 trim_cache size 0 max 16384 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.838+0000 7f8a06744b80 1 Processor -- start 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.838+0000 7f8a06744b80 1 -- start start 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f074964d0 0x559f074959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f074b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0749ac80 0x559f074b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f8a06744b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41b56f0 con 0x559f0749ac80 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f8a06744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41b72c0 con 0x559f074964d0 2024-05-10T10:42:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f8a06744b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41b5dc0 con 0x559f07496df0 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f9c4d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f074964d0 0x559f074959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89fa44e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0749ac80 0x559f074b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89fa44e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0749ac80 0x559f074b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33514/0 (socket says 172.21.15.112:33514) 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89fa44e700 1 -- 172.21.15.112:0/4090357688 learned_addr learned my addr 172.21.15.112:0/4090357688 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f9c4d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f074964d0 0x559f074959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:38912/0 (socket says 172.21.15.112:38912) 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f944c700 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f074b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f944c700 1 -- 172.21.15.112:0/4090357688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f074964d0 msgr2=0x559f074959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f944c700 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f074964d0 0x559f074959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f944c700 1 -- 172.21.15.112:0/4090357688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0749ac80 msgr2=0x559f074b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f944c700 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0749ac80 0x559f074b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f944c700 1 -- 172.21.15.112:0/4090357688 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f074a2d70 con 0x559f07496df0 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.839+0000 7f89f944c700 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f074b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x559f074a2f70 tx=0x7f89e4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af563b2d4f3824b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.840+0000 7f89df7fe700 1 -- 172.21.15.112:0/4090357688 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4030070 con 0x559f07496df0 2024-05-10T10:42:40.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.840+0000 7f89df7fe700 1 -- 172.21.15.112:0/4090357688 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89e402bab0 con 0x559f07496df0 2024-05-10T10:42:40.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.840+0000 7f89df7fe700 1 -- 172.21.15.112:0/4090357688 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4012450 con 0x559f07496df0 2024-05-10T10:42:40.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.840+0000 7f8a06744b80 1 -- 172.21.15.112:0/4090357688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 msgr2=0x559f074b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.840+0000 7f8a06744b80 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f074b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x559f074a2f70 tx=0x7f89e4014000 comp rx=0 tx=0).stop 2024-05-10T10:42:40.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 -- 172.21.15.112:0/4090357688 shutdown_connections 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0749ac80 0x559f074b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f074964d0 0x559f074959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 --2- 172.21.15.112:0/4090357688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f07496df0 0x559f074b4f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 -- 172.21.15.112:0/4090357688 >> 172.21.15.112:0/4090357688 conn(0x559f073c2100 msgr2=0x559f074952e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 -- 172.21.15.112:0/4090357688 shutdown_connections 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 -- 172.21.15.112:0/4090357688 wait complete. 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 -- start start 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.841+0000 7f8a06744b80 1 Processor -- start 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f07496df0 0x559f074b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f8a06744b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f0749ac80 0x559f074b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f8a06744b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41b6b60 con 0x559f074964d0 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f8a06744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41b6490 con 0x559f07496df0 2024-05-10T10:42:40.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f8a06744b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41b3690 con 0x559f0749ac80 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f8a06744b80 10 client.0 _ll_register_callbacks cb 0x559f07380920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89fa44e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f0749ac80 0x559f074b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f07496df0 0x559f074b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f07496df0 0x559f074b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:38924/0 (socket says 172.21.15.112:38924) 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89fa44e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f0749ac80 0x559f074b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:39360/0 (socket says 172.21.15.112:39360) 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 -- 172.21.15.112:0/955488376 learned_addr learned my addr 172.21.15.112:0/955488376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 -- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f0749ac80 msgr2=0x559f074b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f0749ac80 0x559f074b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 -- 172.21.15.112:0/955488376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 msgr2=0x559f074b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 -- 172.21.15.112:0/955488376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f07519090 con 0x559f07496df0 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89fa44e700 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f0749ac80 0x559f074b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:40.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89f944c700 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f07496df0 0x559f074b3020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89e4011fd0 tx=0x7f89e4003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d53a659c84e83c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.842+0000 7f89dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.843+0000 7f89dd7fa700 1 -- 172.21.15.112:0/955488376 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4030040 con 0x559f07496df0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.843+0000 7f89dd7fa700 1 -- 172.21.15.112:0/955488376 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f89e40038e0 con 0x559f07496df0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.843+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f073f6240 con 0x559f07496df0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.843+0000 7f89dd7fa700 1 -- 172.21.15.112:0/955488376 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f89e4043050 con 0x559f07496df0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.843+0000 7f8a06744b80 10 client.25469 fetch_fsmap learned FSMap version 33 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.843+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559f0751a360 con 0x559f07496df0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.843+0000 7f89dd7fa700 1 -- 172.21.15.112:0/955488376 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559f073f6240 con 0x559f07496df0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f89dd7fa700 1 -- 172.21.15.112:0/955488376 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f89e4012830 con 0x559f07496df0 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f8a06744b80 10 client.25469 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f8a06744b80 10 client.25469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f8a06744b80 20 client.25469 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f8a06744b80 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f0751ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f8a06744b80 4 client.25469 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f073f02e0 con 0x559f074afb70 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.844+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f0751ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f89dd7fa700 10 client.25469 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f89f9c4d700 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f0751ca30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9f7654b5a5bb645 server_cookie=5fc18e5739cb549d in_seq=0 out_seq=0 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f89dd7fa700 1 -- 172.21.15.112:0/955488376 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f073f02e0 con 0x559f074afb70 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f89dd7fa700 10 client.25469 handle_command_reply: tid=0 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 client.25469 shutdown 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f07496df0 msgr2=0x559f074b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f07496df0 0x559f074b3020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89e4011fd0 tx=0x7f89e4003cc0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 shutdown_connections 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x559f074afb70 0x559f0751ca30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559f0749ac80 0x559f074b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f074964d0 0x559f074b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 --2- 172.21.15.112:0/955488376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f07496df0 0x559f074b3020 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 >> 172.21.15.112:0/955488376 conn(0x559f073c2100 msgr2=0x559f07495b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f89dd7fa700 0 client.25469 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:42:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 shutdown_connections 2024-05-10T10:42:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 1 -- 172.21.15.112:0/955488376 wait complete. 2024-05-10T10:42:40.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.845+0000 7f8a06744b80 20 client.25469 trim_cache size 0 max 16384 2024-05-10T10:42:40.852 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f89e00b2880 msgr2=0x7f89e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f89e00b2880 0x7f89e00b4d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f89f4103280 tx=0x7f89e4009260 comp rx=0 tx=0).stop 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41037b0 msgr2=0x7f89f410f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41037b0 0x7f89f410f590 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f89f000d950 tx=0x7f89f000dc60 comp rx=0 tx=0).stop 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 shutdown_connections 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f89e00b2880 0x7f89e00b4d40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f4108880 0x7f89f41a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41037b0 0x7f89f410f590 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 --2- 172.21.15.112:0/1619109144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89f4104160 0x7f89f410fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.846+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 >> 172.21.15.112:0/1619109144 conn(0x7f89f407a530 msgr2=0x7f89f41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.847+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 shutdown_connections 2024-05-10T10:42:40.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:40.847+0000 7f89fbeb1700 1 -- 172.21.15.112:0/1619109144 wait complete. 2024-05-10T10:42:40.855 INFO:tasks.workunit.client.0.smithi112.stdout:checking for RLIMIT_NOFILE... yes 2024-05-10T10:42:40.930 INFO:tasks.workunit.client.0.smithi112.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-05-10T10:42:41.000 INFO:tasks.workunit.client.0.smithi112.stdout:checking for FICLONE... yes 2024-05-10T10:42:41.072 INFO:tasks.workunit.client.0.smithi112.stdout:checking for renameat2... yes 2024-05-10T10:42:41.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:40 smithi112 ceph-mon[30100]: pgmap v503: 129 pgs: 129 active+clean; 31 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 961 KiB/s wr, 419 op/s 2024-05-10T10:42:41.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:40 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:41.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:42:41.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:42:41.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:40 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:42:41.147 INFO:tasks.workunit.client.0.smithi112.stdout:checking for reallocarray... yes 2024-05-10T10:42:41.250 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct mount_attr... yes 2024-05-10T10:42:41.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:40 smithi148 ceph-mon[33568]: pgmap v503: 129 pgs: 129 active+clean; 31 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 961 KiB/s wr, 419 op/s 2024-05-10T10:42:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:40 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:42:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:42:41.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:40 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:42:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:40 smithi195 ceph-mon[33773]: pgmap v503: 129 pgs: 129 active+clean; 31 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 961 KiB/s wr, 419 op/s 2024-05-10T10:42:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:40 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:42:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:42:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:40 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:42:41.359 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_qgroup_limit... yes 2024-05-10T10:42:41.452 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-05-10T10:42:41.544 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-05-10T10:42:41.637 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-05-10T10:42:41.752 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-05-10T10:42:41.780 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:42:41.844 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-05-10T10:42:41.938 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-05-10T10:42:42.009 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs.h usability... yes 2024-05-10T10:42:42.033 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs.h presence... yes 2024-05-10T10:42:42.034 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/btrfs.h... yes 2024-05-10T10:42:42.100 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:42.118 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs_tree.h usability... yes 2024-05-10T10:42:42.145 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs_tree.h presence... yes 2024-05-10T10:42:42.145 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/btrfs_tree.h... yes 2024-05-10T10:42:42.214 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-05-10T10:42:42.259 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-05-10T10:42:42.285 INFO:tasks.workunit.client.0.smithi112.stdout:configure: creating ./config.status 2024-05-10T10:42:42.684 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating include/builddefs 2024-05-10T10:42:42.729 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating include/config.h 2024-05-10T10:42:43.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:42 smithi112 ceph-mon[30100]: pgmap v504: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 763 KiB/s wr, 449 op/s 2024-05-10T10:42:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:42 smithi148 ceph-mon[33568]: pgmap v504: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 763 KiB/s wr, 449 op/s 2024-05-10T10:42:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:42 smithi195 ceph-mon[33773]: pgmap v504: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 763 KiB/s wr, 449 op/s 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.931+0000 7f8824315700 1 -- 172.21.15.112:0/3206252967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06bb40 msgr2=0x7f881c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.931+0000 7f8824315700 1 --2- 172.21.15.112:0/3206252967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06bb40 0x7f881c110330 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f881400b0a0 tx=0x7f881400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- 172.21.15.112:0/3206252967 shutdown_connections 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 --2- 172.21.15.112:0/3206252967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06bb40 0x7f881c110330 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 --2- 172.21.15.112:0/3206252967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f881c109510 0x7f881c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 --2- 172.21.15.112:0/3206252967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f881c06b180 0x7f881c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- 172.21.15.112:0/3206252967 >> 172.21.15.112:0/3206252967 conn(0x7f881c079350 msgr2=0x7f881c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- 172.21.15.112:0/3206252967 shutdown_connections 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- 172.21.15.112:0/3206252967 wait complete. 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 Processor -- start 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- start start 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06b180 0x7f881c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f881c06bb40 0x7f881c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f881c109510 0x7f881c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f881c113550 con 0x7f881c06b180 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1133d0 con 0x7f881c06bb40 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.932+0000 7f8824315700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1136d0 con 0x7f881c109510 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06b180 0x7f881c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06b180 0x7f881c102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33524/0 (socket says 172.21.15.112:33524) 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 -- 172.21.15.112:0/2365565717 learned_addr learned my addr 172.21.15.112:0/2365565717 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:43.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88218b0700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f881c06bb40 0x7f881c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88228b2700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f881c109510 0x7f881c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 -- 172.21.15.112:0/2365565717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f881c109510 msgr2=0x7f881c102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f881c109510 0x7f881c102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 -- 172.21.15.112:0/2365565717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f881c06bb40 msgr2=0x7f881c1029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f881c06bb40 0x7f881c1029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 -- 172.21.15.112:0/2365565717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c1a7fd0 con 0x7f881c06b180 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88220b1700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06b180 0x7f881c102470 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f881800b530 tx=0x7f881800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88137fe700 1 -- 172.21.15.112:0/2365565717 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818015020 con 0x7f881c06b180 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c1a8260 con 0x7f881c06b180 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88137fe700 1 -- 172.21.15.112:0/2365565717 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8818011cc0 con 0x7f881c06b180 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f88137fe700 1 -- 172.21.15.112:0/2365565717 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818010c80 con 0x7f881c06b180 2024-05-10T10:42:43.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.933+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1a87a0 con 0x7f881c06b180 2024-05-10T10:42:43.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.936+0000 7f88137fe700 1 -- 172.21.15.112:0/2365565717 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8818018020 con 0x7f881c06b180 2024-05-10T10:42:43.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.936+0000 7f88137fe700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2880 0x7f88080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f88218b0700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2880 0x7f88080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f88137fe700 1 -- 172.21.15.112:0/2365565717 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f88180e1c80 con 0x7f881c06b180 2024-05-10T10:42:43.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 Processor -- start 2024-05-10T10:42:43.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f88218b0700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2880 0x7f88080b4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f881c079a70 tx=0x7f880c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 -- start start 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b75685e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b756863100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b7567b0860 con 0x55b756862c80 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b75674f7c0 con 0x55b75685edf0 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f882eba8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b7567ac820 con 0x55b75685e4d0 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f88228b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b756863100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f88228b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b756863100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33532/0 (socket says 172.21.15.112:33532) 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f88228b2700 1 -- 172.21.15.112:0/4274784315 learned_addr learned my addr 172.21.15.112:0/4274784315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.937+0000 7f88218b0700 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f88220b1700 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b75685e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f88228b2700 1 -- 172.21.15.112:0/4274784315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 msgr2=0x55b75685e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f88228b2700 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b75685e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f88228b2700 1 -- 172.21.15.112:0/4274784315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 msgr2=0x55b7568626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f88228b2700 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f88228b2700 1 -- 172.21.15.112:0/4274784315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7566fd8d0 con 0x55b756862c80 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f88228b2700 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b756863100 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8814009230 tx=0x7f8814009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b4b2fa91d4b589 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f8810d70700 1 -- 172.21.15.112:0/4274784315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814020070 con 0x55b756862c80 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f8810d70700 1 -- 172.21.15.112:0/4274784315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8814009480 con 0x55b756862c80 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.938+0000 7f8810d70700 1 -- 172.21.15.112:0/4274784315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140052a0 con 0x55b756862c80 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 -- 172.21.15.112:0/4274784315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 msgr2=0x55b756863100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b756863100 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8814009230 tx=0x7f8814009260 comp rx=0 tx=0).stop 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 -- 172.21.15.112:0/4274784315 shutdown_connections 2024-05-10T10:42:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b756863100 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b75685e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 --2- 172.21.15.112:0/4274784315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 -- 172.21.15.112:0/4274784315 >> 172.21.15.112:0/4274784315 conn(0x55b75678a100 msgr2=0x55b756858c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 -- 172.21.15.112:0/4274784315 shutdown_connections 2024-05-10T10:42:43.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 -- 172.21.15.112:0/4274784315 wait complete. 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 -- start start 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.939+0000 7f882eba8b80 1 Processor -- start 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b75686a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b756862c80 0x55b756867900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f882eba8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b75686ae30 con 0x55b75685e4d0 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f882eba8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b756716c30 con 0x55b75685edf0 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f882eba8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b75686afb0 con 0x55b756862c80 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f882eba8b80 10 client.0 _ll_register_callbacks cb 0x55b756748920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b75686a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b75686a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33546/0 (socket says 172.21.15.112:33546) 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 -- 172.21.15.112:0/1251092943 learned_addr learned my addr 172.21.15.112:0/1251092943 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88218b0700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88228b2700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b756862c80 0x55b756867900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 -- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b756862c80 msgr2=0x55b756867900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b756862c80 0x55b756867900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 -- 172.21.15.112:0/1251092943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 msgr2=0x55b7568673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 -- 172.21.15.112:0/1251092943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75674c690 con 0x55b75685e4d0 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88218b0700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f88220b1700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b75686a020 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f88180df440 tx=0x7f88180a8af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed12573ee9f63a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.940+0000 7f8805ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f8805ffb700 1 -- 172.21.15.112:0/1251092943 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88180a8f10 con 0x55b75685e4d0 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f8805ffb700 1 -- 172.21.15.112:0/1251092943 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f88180a9380 con 0x55b75685e4d0 2024-05-10T10:42:43.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b75682a650 con 0x55b75685e4d0 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f8805ffb700 1 -- 172.21.15.112:0/1251092943 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f88180ad2c0 con 0x55b75685e4d0 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 10 client.16086 fetch_fsmap learned FSMap version 33 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7568756a0 con 0x55b75685e4d0 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f8805ffb700 1 -- 172.21.15.112:0/1251092943 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b75682a650 con 0x55b75685e4d0 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f8805ffb700 1 -- 172.21.15.112:0/1251092943 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f88180a82e0 con 0x55b75685e4d0 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 20 client.16086 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b756877ce0 0x55b75687a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 4 client.16086 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7567be240 con 0x55b756877ce0 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.941+0000 7f88218b0700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b756877ce0 0x55b75687a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.942+0000 7f8805ffb700 10 client.16086 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.942+0000 7f88218b0700 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b756877ce0 0x55b75687a0d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25442f90aa0d5acb server_cookie=8ea7041bb363a051 in_seq=0 out_seq=0 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f8805ffb700 1 -- 172.21.15.112:0/1251092943 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7567be240 con 0x55b756877ce0 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f8805ffb700 10 client.16086 handle_command_reply: tid=0 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 client.16086 shutdown 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 msgr2=0x55b75686a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b75686a020 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f88180df440 tx=0x7f88180a8af0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 shutdown_connections 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b756877ce0 0x55b75687a0d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b756862c80 0x55b756867900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b75686a020 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f8805ffb700 0 client.16086 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:43.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1251092943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 >> 172.21.15.112:0/1251092943 conn(0x55b75678a100 msgr2=0x55b756863080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 shutdown_connections 2024-05-10T10:42:43.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 1 -- 172.21.15.112:0/1251092943 wait complete. 2024-05-10T10:42:43.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.943+0000 7f882eba8b80 20 client.16086 trim_cache size 0 max 16384 2024-05-10T10:42:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 Processor -- start 2024-05-10T10:42:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 -- start start 2024-05-10T10:42:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b756873ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685edf0 0x55b7568535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b756862c80 0x55b7568605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1b13c0 con 0x55b75685e4d0 2024-05-10T10:42:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1b2f90 con 0x55b756862c80 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f882eba8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1b1a90 con 0x55b75685edf0 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b756873ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b756873ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33562/0 (socket says 172.21.15.112:33562) 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 -- 172.21.15.112:0/1531097495 learned_addr learned my addr 172.21.15.112:0/1531097495 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88228b2700 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b756862c80 0x55b7568605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88218b0700 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685edf0 0x55b7568535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 -- 172.21.15.112:0/1531097495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685edf0 msgr2=0x55b7568535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685edf0 0x55b7568535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 -- 172.21.15.112:0/1531097495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b756862c80 msgr2=0x55b7568605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b756862c80 0x55b7568605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 -- 172.21.15.112:0/1531097495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b756855810 con 0x55b75685e4d0 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.958+0000 7f88220b1700 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b756873ec0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f88180a8ea0 tx=0x7f88180a8730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922837a8541db1ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f8807fff700 1 -- 172.21.15.112:0/1531097495 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88180a8f10 con 0x55b75685e4d0 2024-05-10T10:42:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f8807fff700 1 -- 172.21.15.112:0/1531097495 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88180b1d70 con 0x55b75685e4d0 2024-05-10T10:42:43.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f8807fff700 1 -- 172.21.15.112:0/1531097495 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88180a7cb0 con 0x55b75685e4d0 2024-05-10T10:42:43.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 -- 172.21.15.112:0/1531097495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 msgr2=0x55b756873ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b756873ec0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f88180a8ea0 tx=0x7f88180a8730 comp rx=0 tx=0).stop 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 -- 172.21.15.112:0/1531097495 shutdown_connections 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b756862c80 0x55b7568605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b75685e4d0 0x55b756873ec0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1531097495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685edf0 0x55b7568535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 -- 172.21.15.112:0/1531097495 >> 172.21.15.112:0/1531097495 conn(0x55b75678a100 msgr2=0x55b756852e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 -- 172.21.15.112:0/1531097495 shutdown_connections 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.959+0000 7f882eba8b80 1 -- 172.21.15.112:0/1531097495 wait complete. 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 -- start start 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 Processor -- start 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b7568703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b75686b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1b2830 con 0x55b756862c80 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1b2160 con 0x55b75685edf0 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1af360 con 0x55b75685e4d0 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f882eba8b80 10 client.0 _ll_register_callbacks cb 0x55b756748920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b75686b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88218b0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b75686b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33570/0 (socket says 172.21.15.112:33570) 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 -- 172.21.15.112:0/1620635708 learned_addr learned my addr 172.21.15.112:0/1620635708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88220b1700 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b7568703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 -- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 msgr2=0x55b7568703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b7568703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 -- 172.21.15.112:0/1620635708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 msgr2=0x55b7568708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 -- 172.21.15.112:0/1620635708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75686c200 con 0x55b756862c80 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f88228b2700 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b75686b990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f88140092d0 tx=0x7f8814007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82dee7cfe54555e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.960+0000 7f8805ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f8805ffb700 1 -- 172.21.15.112:0/1620635708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814020040 con 0x55b756862c80 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f8805ffb700 1 -- 172.21.15.112:0/1620635708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f881400f070 con 0x55b756862c80 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88180b1d70 con 0x55b756862c80 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f8805ffb700 1 -- 172.21.15.112:0/1620635708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f88140076b0 con 0x55b756862c80 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 10 client.16092 fetch_fsmap learned FSMap version 33 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b7568e2190 con 0x55b756862c80 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f8805ffb700 1 -- 172.21.15.112:0/1620635708 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88180b1d70 con 0x55b756862c80 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f8805ffb700 1 -- 172.21.15.112:0/1620635708 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f8814011070 con 0x55b756862c80 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 20 client.16092 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b7568e24c0 0x55b7568e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 4 client.16092 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:42:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7567b82e0 con 0x55b7568e24c0 2024-05-10T10:42:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.961+0000 7f88220b1700 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b7568e24c0 0x55b7568e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f88220b1700 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b7568e24c0 0x55b7568e48b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acdc963bd8a6dfd9 server_cookie=604a48c658b8063f in_seq=0 out_seq=0 2024-05-10T10:42:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f8805ffb700 10 client.16092 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f8805ffb700 1 -- 172.21.15.112:0/1620635708 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7567b82e0 con 0x55b7568e24c0 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f8805ffb700 10 client.16092 handle_command_reply: tid=0 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 client.16092 shutdown 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 msgr2=0x55b75686b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b75686b990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f88140092d0 tx=0x7f8814007c20 comp rx=0 tx=0).stop 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 shutdown_connections 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b7568e24c0 0x55b7568e48b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b756862c80 0x55b75686b990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f8805ffb700 0 client.16092 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b75685e4d0 0x55b7568703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 --2- 172.21.15.112:0/1620635708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b75685edf0 0x55b7568708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 >> 172.21.15.112:0/1620635708 conn(0x55b75678a100 msgr2=0x55b75685d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 shutdown_connections 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 1 -- 172.21.15.112:0/1620635708 wait complete. 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.962+0000 7f882eba8b80 20 client.16092 trim_cache size 0 max 16384 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:42:43.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2880 msgr2=0x7f88080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2880 0x7f88080b4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f881c079a70 tx=0x7f880c00b410 comp rx=0 tx=0).stop 2024-05-10T10:42:43.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06b180 msgr2=0x7f881c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06b180 0x7f881c102470 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f881800b530 tx=0x7f881800b840 comp rx=0 tx=0).stop 2024-05-10T10:42:43.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 shutdown_connections 2024-05-10T10:42:43.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88080b2880 0x7f88080b4d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f881c109510 0x7f881c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c06b180 0x7f881c102470 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 --2- 172.21.15.112:0/2365565717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f881c06bb40 0x7f881c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 >> 172.21.15.112:0/2365565717 conn(0x7f881c079350 msgr2=0x7f881c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 shutdown_connections 2024-05-10T10:42:43.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:43.963+0000 7f8824315700 1 -- 172.21.15.112:0/2365565717 wait complete. 2024-05-10T10:42:45.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:44 smithi112 ceph-mon[30100]: pgmap v505: 129 pgs: 129 active+clean; 31 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 605 KiB/s wr, 448 op/s 2024-05-10T10:42:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:44 smithi148 ceph-mon[33568]: pgmap v505: 129 pgs: 129 active+clean; 31 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 605 KiB/s wr, 448 op/s 2024-05-10T10:42:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:44 smithi195 ceph-mon[33773]: pgmap v505: 129 pgs: 129 active+clean; 31 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 605 KiB/s wr, 448 op/s 2024-05-10T10:42:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 -- 172.21.15.112:0/2630730227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8340101910 msgr2=0x7f8340101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 --2- 172.21.15.112:0/2630730227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8340101910 0x7f8340101d30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8330009ca0 tx=0x7f833002a8c0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 -- 172.21.15.112:0/2630730227 shutdown_connections 2024-05-10T10:42:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 --2- 172.21.15.112:0/2630730227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83401034c0 0x7f8340109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 --2- 172.21.15.112:0/2630730227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8340102b00 0x7f8340102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 --2- 172.21.15.112:0/2630730227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8340101910 0x7f8340101d30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 -- 172.21.15.112:0/2630730227 >> 172.21.15.112:0/2630730227 conn(0x7f83400fce80 msgr2=0x7f83400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 -- 172.21.15.112:0/2630730227 shutdown_connections 2024-05-10T10:42:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.374+0000 7f8344abd640 1 -- 172.21.15.112:0/2630730227 wait complete. 2024-05-10T10:42:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 Processor -- start 2024-05-10T10:42:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 -- start start 2024-05-10T10:42:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8340101910 0x7f8340198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8340102b00 0x7f8340198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83401034c0 0x7f83401a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f834010d310 con 0x7f83401034c0 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f834010d190 con 0x7f8340102b00 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f8344abd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f834010d490 con 0x7f8340101910 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83401034c0 0x7f83401a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83401034c0 0x7f83401a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33584/0 (socket says 172.21.15.112:33584) 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 -- 172.21.15.112:0/958782883 learned_addr learned my addr 172.21.15.112:0/958782883 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833e59c640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8340101910 0x7f8340198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833dd9b640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8340102b00 0x7f8340198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 -- 172.21.15.112:0/958782883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8340101910 msgr2=0x7f8340198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8340101910 0x7f8340198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 -- 172.21.15.112:0/958782883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8340102b00 msgr2=0x7f8340198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8340102b00 0x7f8340198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 -- 172.21.15.112:0/958782883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83401a0690 con 0x7f83401034c0 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.375+0000 7f833ed9d640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83401034c0 0x7f83401a0020 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f833800d750 tx=0x7f833800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.376+0000 7f832f7fe640 1 -- 172.21.15.112:0/958782883 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338017070 con 0x7f83401034c0 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.376+0000 7f832f7fe640 1 -- 172.21.15.112:0/958782883 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83380054b0 con 0x7f83401034c0 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.376+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83401a0980 con 0x7f83401034c0 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.376+0000 7f832f7fe640 1 -- 172.21.15.112:0/958782883 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338004b90 con 0x7f83401034c0 2024-05-10T10:42:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.376+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83401a0e80 con 0x7f83401034c0 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.378+0000 7f832f7fe640 1 -- 172.21.15.112:0/958782883 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8338004d30 con 0x7f83401034c0 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.378+0000 7f83477d2740 1 Processor -- start 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f83477d2740 1 -- start start 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f83477d2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc4501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f83477d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc4552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f83477d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc457da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f83477d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557afc3b45d0 con 0x557afc455910 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f83477d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557afc36d890 con 0x557afc4507a0 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f83477d2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557afc3b0590 con 0x557afc44fdb0 2024-05-10T10:42:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc457da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc457da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33588/0 (socket says 172.21.15.112:33588) 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 -- 172.21.15.112:0/2314085617 learned_addr learned my addr 172.21.15.112:0/2314085617 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc4501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f832f7fe640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f83280b28a0 0x7f83280b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833e59c640 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc4552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 -- 172.21.15.112:0/2314085617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 msgr2=0x557afc4501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc4501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 -- 172.21.15.112:0/2314085617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 msgr2=0x557afc4552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc4552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833dd9b640 1 -- 172.21.15.112:0/2314085617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afc0c6210 con 0x557afc455910 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.379+0000 7f833ed9d640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f83280b28a0 0x7f83280b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.380+0000 7f832f7fe640 1 -- 172.21.15.112:0/958782883 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f83380e2100 con 0x7f83401034c0 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.380+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc457da0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f833400bfd0 tx=0x7f833400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84fd84462c2ffac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.380+0000 7f832cff9640 1 -- 172.21.15.112:0/2314085617 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833400e950 con 0x557afc455910 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.380+0000 7f832cff9640 1 -- 172.21.15.112:0/2314085617 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8334010430 con 0x557afc455910 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.380+0000 7f832cff9640 1 -- 172.21.15.112:0/2314085617 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833400f620 con 0x557afc455910 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- 172.21.15.112:0/2314085617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 msgr2=0x557afc457da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f833ed9d640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f83280b28a0 0x7f83280b4d60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83380a53b0 tx=0x7f8338075cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc457da0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f833400bfd0 tx=0x7f833400b460 comp rx=0 tx=0).stop 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- 172.21.15.112:0/2314085617 shutdown_connections 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc457da0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc4552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 --2- 172.21.15.112:0/2314085617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc4501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- 172.21.15.112:0/2314085617 >> 172.21.15.112:0/2314085617 conn(0x557afc44b420 msgr2=0x557afc44b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- 172.21.15.112:0/2314085617 shutdown_connections 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- 172.21.15.112:0/2314085617 wait complete. 2024-05-10T10:42:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- start start 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 Processor -- start 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc44fdb0 0x557afc459fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc4507a0 0x557afc45a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc455910 0x557afc461520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557afc4588c0 con 0x557afc4507a0 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557afc371370 con 0x557afc44fdb0 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557afc458a40 con 0x557afc455910 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc4507a0 0x557afc45a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.381+0000 7f83477d2740 10 client.0 _ll_register_callbacks cb 0x557afc0c5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc4507a0 0x557afc45a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33602/0 (socket says 172.21.15.112:33602) 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 -- 172.21.15.112:0/2047327797 learned_addr learned my addr 172.21.15.112:0/2047327797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc44fdb0 0x557afc459fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc455910 0x557afc461520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 -- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc455910 msgr2=0x557afc461520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc455910 0x557afc461520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 -- 172.21.15.112:0/2047327797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc44fdb0 msgr2=0x557afc459fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc44fdb0 0x557afc459fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 -- 172.21.15.112:0/2047327797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afc192620 con 0x557afc4507a0 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc455910 0x557afc461520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f833e59c640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc4507a0 0x557afc45a4e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8330009ca0 tx=0x7f8330031d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5090fb887aa0e832 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f8325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f8325ffb640 1 -- 172.21.15.112:0/2047327797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330003d80 con 0x557afc4507a0 2024-05-10T10:42:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8334010430 con 0x557afc4507a0 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f8325ffb640 1 -- 172.21.15.112:0/2047327797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8330004040 con 0x557afc4507a0 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f8325ffb640 1 -- 172.21.15.112:0/2047327797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f833004f050 con 0x557afc4507a0 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f83477d2740 10 client.16104 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.382+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557afc464500 con 0x557afc4507a0 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f8325ffb640 1 -- 172.21.15.112:0/2047327797 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8334010430 con 0x557afc4507a0 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f8325ffb640 1 -- 172.21.15.112:0/2047327797 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f83300308f0 con 0x557afc4507a0 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f83477d2740 10 client.16104 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f83477d2740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f83477d2740 20 client.16104 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f83477d2740 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc465d70 0x557afc4681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f83477d2740 4 client.16104 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557afc3c21e0 con 0x557afc465d70 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc465d70 0x557afc4681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f8325ffb640 10 client.16104 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.383+0000 7f833dd9b640 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc465d70 0x557afc4681f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6b637a00f2df1a server_cookie=c7f3cac00d3c3917 in_seq=0 out_seq=0 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f8325ffb640 1 -- 172.21.15.112:0/2047327797 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557afc3c21e0 con 0x557afc465d70 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f8325ffb640 10 client.16104 handle_command_reply: tid=0 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 client.16104 shutdown 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc4507a0 msgr2=0x557afc45a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc4507a0 0x557afc45a4e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8330009ca0 tx=0x7f8330031d30 comp rx=0 tx=0).stop 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 shutdown_connections 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc455910 0x557afc461520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc4507a0 0x557afc45a4e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc465d70 0x557afc4681f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 --2- 172.21.15.112:0/2047327797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc44fdb0 0x557afc459fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f8325ffb640 0 client.16104 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 >> 172.21.15.112:0/2047327797 conn(0x557afc44b420 msgr2=0x557afc44a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 shutdown_connections 2024-05-10T10:42:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.384+0000 7f83477d2740 1 -- 172.21.15.112:0/2047327797 wait complete. 2024-05-10T10:42:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.385+0000 7f83477d2740 20 client.16104 trim_cache size 0 max 16384 2024-05-10T10:42:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.399+0000 7f83477d2740 1 Processor -- start 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.399+0000 7f83477d2740 1 -- start start 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f83477d2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc450e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f83477d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc454fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f83477d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc451980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f83477d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83401ab390 con 0x557afc455910 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f83477d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83401acf90 con 0x557afc4507a0 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f83477d2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f83401aba60 con 0x557afc44fdb0 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc451980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc451980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33610/0 (socket says 172.21.15.112:33610) 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 -- 172.21.15.112:0/289865970 learned_addr learned my addr 172.21.15.112:0/289865970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc450e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833e59c640 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc454fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 -- 172.21.15.112:0/289865970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 msgr2=0x557afc450e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc450e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 -- 172.21.15.112:0/289865970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 msgr2=0x557afc454fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc454fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 -- 172.21.15.112:0/289865970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afc44b250 con 0x557afc455910 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833e59c640 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc454fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f833dd9b640 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc451980 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8334013900 tx=0x7f8334015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1004714ce770dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f8327fff640 1 -- 172.21.15.112:0/289865970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833400bbf0 con 0x557afc455910 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f8327fff640 1 -- 172.21.15.112:0/289865970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83340054d0 con 0x557afc455910 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.400+0000 7f8327fff640 1 -- 172.21.15.112:0/289865970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833402d870 con 0x557afc455910 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 -- 172.21.15.112:0/289865970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 msgr2=0x557afc451980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc451980 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8334013900 tx=0x7f8334015040 comp rx=0 tx=0).stop 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 -- 172.21.15.112:0/289865970 shutdown_connections 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc455910 0x557afc451980 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc4507a0 0x557afc454fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 --2- 172.21.15.112:0/289865970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc44fdb0 0x557afc450e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 -- 172.21.15.112:0/289865970 >> 172.21.15.112:0/289865970 conn(0x557afc44b420 msgr2=0x557afc458de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 -- 172.21.15.112:0/289865970 shutdown_connections 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 -- 172.21.15.112:0/289865970 wait complete. 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 -- start start 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.401+0000 7f83477d2740 1 Processor -- start 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc44fdb0 0x557afc4cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc4507a0 0x557afc4cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc455910 0x557afc4d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83401ac800 con 0x557afc44fdb0 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83401ac130 con 0x557afc455910 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f83401a9370 con 0x557afc4507a0 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 10 client.0 _ll_register_callbacks cb 0x7f8330009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc44fdb0 0x557afc4cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc44fdb0 0x557afc4cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33616/0 (socket says 172.21.15.112:33616) 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 -- 172.21.15.112:0/834103856 learned_addr learned my addr 172.21.15.112:0/834103856 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833e59c640 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc4507a0 0x557afc4cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc455910 0x557afc4d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 -- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc4507a0 msgr2=0x557afc4cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc4507a0 0x557afc4cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 -- 172.21.15.112:0/834103856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc455910 msgr2=0x557afc4d2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc455910 0x557afc4d2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 -- 172.21.15.112:0/834103856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afc458b80 con 0x557afc44fdb0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f833dd9b640 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc44fdb0 0x557afc4cb570 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8334000c00 tx=0x7f8334005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21a5f9b3a8f9d74 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f8325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f8325ffb640 1 -- 172.21.15.112:0/834103856 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334033020 con 0x557afc44fdb0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83340054d0 con 0x557afc44fdb0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.402+0000 7f8325ffb640 1 -- 172.21.15.112:0/834103856 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8334005110 con 0x557afc44fdb0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f8325ffb640 1 -- 172.21.15.112:0/834103856 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8334044050 con 0x557afc44fdb0 2024-05-10T10:42:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 10 client.16110 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557afc4d4190 con 0x557afc44fdb0 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f8325ffb640 1 -- 172.21.15.112:0/834103856 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83340054d0 con 0x557afc44fdb0 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f8325ffb640 1 -- 172.21.15.112:0/834103856 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f833400ebe0 con 0x557afc44fdb0 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 10 client.16110 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 20 client.16110 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc4d4ac0 0x557afc4d6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 4 client.16110 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557afc3bc050 con 0x557afc4d4ac0 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f833e59c640 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc4d4ac0 0x557afc4d6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f8325ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.403+0000 7f833e59c640 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc4d4ac0 0x557afc4d6ee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45b7cb49345c394 server_cookie=b3cc019addcd8d6c in_seq=0 out_seq=0 2024-05-10T10:42:45.430 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: executing libtool commands 2024-05-10T10:42:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f8325ffb640 1 -- 172.21.15.112:0/834103856 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557afc3bc050 con 0x557afc4d4ac0 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f8325ffb640 10 client.16110 handle_command_reply: tid=0 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 client.16110 shutdown 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc44fdb0 msgr2=0x557afc4cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc44fdb0 0x557afc4cb570 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8334000c00 tx=0x7f8334005d60 comp rx=0 tx=0).stop 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 shutdown_connections 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557afc4d4ac0 0x557afc4d6ee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557afc455910 0x557afc4d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557afc4507a0 0x557afc4cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 --2- 172.21.15.112:0/834103856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557afc44fdb0 0x557afc4cb570 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 >> 172.21.15.112:0/834103856 conn(0x557afc44b420 msgr2=0x557afc458de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f8325ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 shutdown_connections 2024-05-10T10:42:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.431+0000 7f83477d2740 1 -- 172.21.15.112:0/834103856 wait complete. 2024-05-10T10:42:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f83477d2740 20 client.16110 trim_cache size 0 max 16384 2024-05-10T10:42:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:42:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:42:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:42:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:42:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f83280b28a0 msgr2=0x7f83280b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f83280b28a0 0x7f83280b4d60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83380a53b0 tx=0x7f8338075cb0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83401034c0 msgr2=0x7f83401a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83401034c0 0x7f83401a0020 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f833800d750 tx=0x7f833800dc10 comp rx=0 tx=0).stop 2024-05-10T10:42:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 shutdown_connections 2024-05-10T10:42:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f83280b28a0 0x7f83280b4d60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83401034c0 0x7f83401a0020 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8340102b00 0x7f8340198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 --2- 172.21.15.112:0/958782883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8340101910 0x7f8340198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 >> 172.21.15.112:0/958782883 conn(0x7f83400fce80 msgr2=0x7f834006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 shutdown_connections 2024-05-10T10:42:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:45.432+0000 7f8344abd640 1 -- 172.21.15.112:0/958782883 wait complete. 2024-05-10T10:42:45.620 INFO:tasks.workunit.client.0.smithi112.stdout:Building lib 2024-05-10T10:42:45.621 INFO:tasks.workunit.client.0.smithi112.stdout:Building include 2024-05-10T10:42:45.621 INFO:tasks.workunit.client.0.smithi112.stdout:Building tests 2024-05-10T10:42:45.643 INFO:tasks.workunit.client.0.smithi112.stdout: [LTDEP] 2024-05-10T10:42:45.696 INFO:tasks.workunit.client.0.smithi112.stdout:Building btrfs 2024-05-10T10:42:45.696 INFO:tasks.workunit.client.0.smithi112.stdout:Building cifs 2024-05-10T10:42:45.696 INFO:tasks.workunit.client.0.smithi112.stdout:Building ceph 2024-05-10T10:42:45.748 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-05-10T10:42:45.752 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-05-10T10:42:45.771 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-05-10T10:42:45.798 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:42:45.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:42:45.833 INFO:tasks.workunit.client.0.smithi112.stdout:Building ext4 2024-05-10T10:42:45.868 INFO:tasks.workunit.client.0.smithi112.stdout:Building f2fs 2024-05-10T10:42:45.904 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-05-10T10:42:45.914 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-05-10T10:42:46.011 INFO:tasks.workunit.client.0.smithi112.stdout:Building generic 2024-05-10T10:42:46.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:45 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:46.130 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:46.187 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-05-10T10:42:46.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:45 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:46.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:45 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:42:46.437 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dataascii.lo 2024-05-10T10:42:46.635 INFO:tasks.workunit.client.0.smithi112.stdout:Building nfs 2024-05-10T10:42:46.677 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-05-10T10:42:46.697 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] databin.lo 2024-05-10T10:42:46.751 INFO:tasks.workunit.client.0.smithi112.stdout:Building ocfs2 2024-05-10T10:42:46.791 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-05-10T10:42:46.863 INFO:tasks.workunit.client.0.smithi112.stdout:Building overlay 2024-05-10T10:42:46.939 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-05-10T10:42:47.063 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] datapid.lo 2024-05-10T10:42:47.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:46 smithi112 ceph-mon[30100]: pgmap v506: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 889 KiB/s wr, 390 op/s 2024-05-10T10:42:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:46 smithi148 ceph-mon[33568]: pgmap v506: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 889 KiB/s wr, 390 op/s 2024-05-10T10:42:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:46 smithi195 ceph-mon[33773]: pgmap v506: 129 pgs: 129 active+clean; 34 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 889 KiB/s wr, 390 op/s 2024-05-10T10:42:47.345 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] file_lock.lo 2024-05-10T10:42:47.735 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] forker.lo 2024-05-10T10:42:47.885 INFO:tasks.workunit.client.0.smithi112.stdout:Building perf 2024-05-10T10:42:47.933 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-05-10T10:42:48.003 INFO:tasks.workunit.client.0.smithi112.stdout:Building selftest 2024-05-10T10:42:48.047 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-05-10T10:42:48.069 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] pattern.lo 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 -- 172.21.15.112:0/245335066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90102d50 msgr2=0x7f7e9010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 --2- 172.21.15.112:0/245335066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90102d50 0x7f7e9010f240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009f90 tx=0x7f7e8800b330 comp rx=0 tx=0).stop 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 -- 172.21.15.112:0/245335066 shutdown_connections 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 --2- 172.21.15.112:0/245335066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90102d50 0x7f7e9010f240 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 --2- 172.21.15.112:0/245335066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e90108400 0x7f7e901087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 --2- 172.21.15.112:0/245335066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102390 0x7f7e90102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 -- 172.21.15.112:0/245335066 >> 172.21.15.112:0/245335066 conn(0x7f7e90079310 msgr2=0x7f7e90079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 -- 172.21.15.112:0/245335066 shutdown_connections 2024-05-10T10:42:48.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.111+0000 7f7e963cc700 1 -- 172.21.15.112:0/245335066 wait complete. 2024-05-10T10:42:48.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.112+0000 7f7e963cc700 1 Processor -- start 2024-05-10T10:42:48.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.112+0000 7f7e963cc700 1 -- start start 2024-05-10T10:42:48.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e963cc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e90102390 0x7f7e9019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e963cc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102d50 0x7f7e9019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e963cc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90108400 0x7f7e901987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e963cc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90112400 con 0x7f7e90108400 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e963cc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90112280 con 0x7f7e90102390 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e963cc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90112580 con 0x7f7e90102d50 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102d50 0x7f7e9019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102d50 0x7f7e9019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:34000/0 (socket says 172.21.15.112:34000) 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/170637122 learned_addr learned my addr 172.21.15.112:0/170637122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e90102390 0x7f7e9019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e94969700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90108400 0x7f7e901987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/170637122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e90102390 msgr2=0x7f7e9019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e90102390 0x7f7e9019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90108400 msgr2=0x7f7e901987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90108400 0x7f7e901987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/170637122 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e90198fa0 con 0x7f7e90102d50 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.113+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e90102390 0x7f7e9019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.114+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102d50 0x7f7e9019eb80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9c0 tx=0x7f7e7800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.114+0000 7f7e8d7fa700 1 -- 172.21.15.112:0/170637122 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800cbd0 con 0x7f7e90102d50 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.114+0000 7f7e8d7fa700 1 -- 172.21.15.112:0/170637122 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e7800ee60 con 0x7f7e90102d50 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.114+0000 7f7e8d7fa700 1 -- 172.21.15.112:0/170637122 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78005540 con 0x7f7e90102d50 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.114+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e90199290 con 0x7f7e90102d50 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.114+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9006ba10 con 0x7f7e90102d50 2024-05-10T10:42:48.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.116+0000 7f7e8d7fa700 1 -- 172.21.15.112:0/170637122 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7e78004750 con 0x7f7e90102d50 2024-05-10T10:42:48.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.117+0000 7f7e8d7fa700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7e7c0b2900 0x7f7e7c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.118+0000 7f7e8d7fa700 1 -- 172.21.15.112:0/170637122 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7e780e2770 con 0x7f7e90102d50 2024-05-10T10:42:48.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.118+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7e7c0b2900 0x7f7e7c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.118+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7e7c0b2900 0x7f7e7c0b4dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e80005fd0 tx=0x7f7e80009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.119+0000 7f7ea0c5fb80 1 Processor -- start 2024-05-10T10:42:48.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.119+0000 7f7ea0c5fb80 1 -- start start 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.119+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702a2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.119+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702a6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.119+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.119+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.119+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702a2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:34010/0 (socket says 172.21.15.112:34010) 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:39620/0 (socket says 172.21.15.112:39620) 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e94969700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e94969700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47724/0 (socket says 172.21.15.112:47724) 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e8ffff700 1 -- 172.21.15.112:0/1522139402 learned_addr learned my addr 172.21.15.112:0/1522139402 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e701f48e0 con 0x559e702a6d00 2024-05-10T10:42:48.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559e70193840 con 0x559e702a2e70 2024-05-10T10:42:48.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e701f08a0 con 0x559e702a2550 2024-05-10T10:42:48.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e8ffff700 1 -- 172.21.15.112:0/1522139402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 msgr2=0x559e702a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.120+0000 7f7e8ffff700 1 -- 172.21.15.112:0/1522139402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 msgr2=0x559e702a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e8ffff700 1 -- 172.21.15.112:0/1522139402 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e70141950 con 0x559e702a2550 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e94969700 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702a2930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000e6d0 tx=0x7f7e80014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f3ed934b158fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e867fc700 1 -- 172.21.15.112:0/1522139402 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80004220 con 0x559e702a2550 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e867fc700 1 -- 172.21.15.112:0/1522139402 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e800043c0 con 0x559e702a2550 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.121+0000 7f7e867fc700 1 -- 172.21.15.112:0/1522139402 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8001c720 con 0x559e702a2550 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 msgr2=0x559e702a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702a2930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000e6d0 tx=0x7f7e80014000 comp rx=0 tx=0).stop 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 shutdown_connections 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702a2930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1522139402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 >> 172.21.15.112:0/1522139402 conn(0x559e701ce180 msgr2=0x559e7029ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 shutdown_connections 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.122+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1522139402 wait complete. 2024-05-10T10:42:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.123+0000 7f7ea0c5fb80 1 -- start start 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.123+0000 7f7ea0c5fb80 1 Processor -- start 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.123+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e702aef10 con 0x559e702a6d00 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559e7015acb0 con 0x559e702a2e70 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e702af090 con 0x559e702a2550 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7ea0c5fb80 10 client.0 _ll_register_callbacks cb 0x559e7018c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47728/0 (socket says 172.21.15.112:47728) 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 -- 172.21.15.112:0/1113169383 learned_addr learned my addr 172.21.15.112:0/1113169383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 -- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 msgr2=0x559e702ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 -- 172.21.15.112:0/1113169383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 msgr2=0x559e702ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 -- 172.21.15.112:0/1113169383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e70190710 con 0x559e702a6d00 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e94969700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702aba80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009550 tx=0x7f7e88003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98693c8de894e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.124+0000 7f7e6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/1113169383 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800be50 con 0x559e702a6d00 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/1113169383 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7e88007420 con 0x559e702a6d00 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e800043c0 con 0x559e702a6d00 2024-05-10T10:42:48.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/1113169383 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7e8802f050 con 0x559e702a6d00 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 10 client.16122 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e702c1d90 con 0x559e702a6d00 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/1113169383 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e800043c0 con 0x559e702a6d00 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/1113169383 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f7e88011070 con 0x559e702a6d00 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 10 client.16122 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 20 client.16122 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702c4420 0x559e702c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 4 client.16122 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e7026e6d0 con 0x559e702c4420 2024-05-10T10:42:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.125+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702c4420 0x559e702c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.126+0000 7f7e6f7fe700 10 client.16122 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.126+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702c4420 0x559e702c6810 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a907e53fe72433 server_cookie=8988356ff4ba1afe in_seq=0 out_seq=0 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/1113169383 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e7026e6d0 con 0x559e702c4420 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7e6f7fe700 10 client.16122 handle_command_reply: tid=0 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7ea0c5fb80 1 client.16122 shutdown 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 msgr2=0x559e702aba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702aba80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009550 tx=0x7f7e88003f90 comp rx=0 tx=0).stop 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 shutdown_connections 2024-05-10T10:42:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702c4420 0x559e702c6810 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7e6f7fe700 0 client.16122 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.127+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a6d00 0x559e702aba80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.128+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e702a2550 0x559e702ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.128+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1113169383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.128+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 >> 172.21.15.112:0/1113169383 conn(0x559e701ce180 msgr2=0x559e702977a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.128+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 shutdown_connections 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.128+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1113169383 wait complete. 2024-05-10T10:42:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.128+0000 7f7ea0c5fb80 20 client.16122 trim_cache size 0 max 16384 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 Processor -- start 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 -- start start 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 0x559e702c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2550 0x559e702abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901acf10 con 0x559e702a2550 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901aeb10 con 0x559e702a2e70 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901ad5e0 con 0x559e701ce180 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.159+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2550 0x559e702abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2550 0x559e702abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47740/0 (socket says 172.21.15.112:47740) 2024-05-10T10:42:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/1548092694 learned_addr learned my addr 172.21.15.112:0/1548092694 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/1548092694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 msgr2=0x559e702c0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/1548092694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 0x559e702c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/1548092694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 msgr2=0x559e702cc0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:42:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/1548092694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702cc0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/1548092694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7029d8d0 con 0x559e702a2550 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/1548092694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2550 0x559e702abaf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a5a20 tx=0x7f7e780b21c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f0c664b722231a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e85ffb700 1 -- 172.21.15.112:0/1548092694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780c3070 con 0x559e702a2550 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.160+0000 7f7e85ffb700 1 -- 172.21.15.112:0/1548092694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e780a99a0 con 0x559e702a2550 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7e85ffb700 1 -- 172.21.15.112:0/1548092694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780a5c30 con 0x559e702a2550 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1548092694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2550 msgr2=0x559e702abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1548092694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2550 0x559e702abaf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a5a20 tx=0x7f7e780b21c0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1548092694 shutdown_connections 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1548092694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2e70 0x559e702cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1548092694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 0x559e702c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/1548092694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2550 0x559e702abaf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.161+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1548092694 >> 172.21.15.112:0/1548092694 conn(0x559e70297b30 msgr2=0x559e702a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.162+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1548092694 shutdown_connections 2024-05-10T10:42:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.162+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/1548092694 wait complete. 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.162+0000 7f7ea0c5fb80 1 -- start start 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 1 Processor -- start 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 0x559e702bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2550 0x559e702bd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2e70 0x559e702bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901ae380 con 0x559e702a2e70 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901adcb0 con 0x559e702a2550 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901aaeb0 con 0x559e701ce180 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2550 0x559e702bd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2550 0x559e702bd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:39654/0 (socket says 172.21.15.112:39654) 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/604718005 learned_addr learned my addr 172.21.15.112:0/604718005 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 0x559e702bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7ea0c5fb80 10 client.0 _ll_register_callbacks cb 0x559e7018c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:48.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.163+0000 7f7e94969700 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2e70 0x559e702bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 msgr2=0x559e702bcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 0x559e702bcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/604718005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2e70 msgr2=0x559e702bb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2e70 0x559e702bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e8f7fe700 1 -- 172.21.15.112:0/604718005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e702bbb80 con 0x559e702a2550 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e8f7fe700 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2550 0x559e702bd2d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a9650 tx=0x7f7e780a58e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51192c22d2de04e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/604718005 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780c3040 con 0x559e702a2550 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/604718005 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7e780a5f90 con 0x559e702a2550 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e780a99a0 con 0x559e702a2550 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/604718005 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7e7808d050 con 0x559e702a2550 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7ea0c5fb80 10 client.25517 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.164+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e7032fa00 con 0x559e702a2550 2024-05-10T10:42:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/604718005 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e780b2800 con 0x559e702a2550 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/604718005 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f7e78004ad0 con 0x559e702a2550 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7ea0c5fb80 10 client.25517 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7ea0c5fb80 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7ea0c5fb80 20 client.25517 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702a6d00 0x559e70332160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7ea0c5fb80 4 client.25517 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7e780a99a0 con 0x559e702a6d00 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.165+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702a6d00 0x559e70332160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.166+0000 7f7e6f7fe700 10 client.25517 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.166+0000 7f7e8ffff700 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702a6d00 0x559e70332160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5132ecf68832188 server_cookie=45daeb7137c42fea in_seq=0 out_seq=0 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.166+0000 7f7e6f7fe700 1 -- 172.21.15.112:0/604718005 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7e780a99a0 con 0x559e702a6d00 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.166+0000 7f7e6f7fe700 10 client.25517 handle_command_reply: tid=0 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.166+0000 7f7ea0c5fb80 1 client.25517 shutdown 2024-05-10T10:42:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2550 msgr2=0x559e702bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2550 0x559e702bd2d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a9650 tx=0x7f7e780a58e0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 shutdown_connections 2024-05-10T10:42:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559e702a6d00 0x559e70332160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e702a2e70 0x559e702bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e701ce180 0x559e702bcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 --2- 172.21.15.112:0/604718005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e702a2550 0x559e702bd2d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 >> 172.21.15.112:0/604718005 conn(0x559e70297b30 msgr2=0x559e702acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.167+0000 7f7e6f7fe700 0 client.25517 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.168+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 shutdown_connections 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.168+0000 7f7ea0c5fb80 1 -- 172.21.15.112:0/604718005 wait complete. 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.168+0000 7f7ea0c5fb80 20 client.25517 trim_cache size 0 max 16384 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "38b58198-816b-417e-8076-c94ee7a46221", 2024-05-10T10:42:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:42:48.175 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:42:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7e7c0b2900 msgr2=0x7f7e7c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7e7c0b2900 0x7f7e7c0b4dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e80005fd0 tx=0x7f7e80009310 comp rx=0 tx=0).stop 2024-05-10T10:42:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102d50 msgr2=0x7f7e9019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102d50 0x7f7e9019eb80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9c0 tx=0x7f7e7800ecd0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 shutdown_connections 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90108400 0x7f7e901987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e90102390 0x7f7e9019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7e7c0b2900 0x7f7e7c0b4dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 --2- 172.21.15.112:0/170637122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e90102d50 0x7f7e9019eb80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 >> 172.21.15.112:0/170637122 conn(0x7f7e90079310 msgr2=0x7f7e901010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 shutdown_connections 2024-05-10T10:42:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:48.169+0000 7f7e963cc700 1 -- 172.21.15.112:0/170637122 wait complete. 2024-05-10T10:42:48.176 INFO:tasks.workunit.client.0.smithi112.stdout:Building shared 2024-05-10T10:42:48.367 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-05-10T10:42:48.440 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] open_flags.lo 2024-05-10T10:42:48.485 INFO:tasks.workunit.client.0.smithi112.stdout:Building tmpfs 2024-05-10T10:42:48.485 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] random_range.lo 2024-05-10T10:42:48.531 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-05-10T10:42:48.614 INFO:tasks.workunit.client.0.smithi112.stdout:Building udf 2024-05-10T10:42:48.661 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-05-10T10:42:48.727 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] string_to_tokens.lo 2024-05-10T10:42:48.812 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] str_to_bytes.lo 2024-05-10T10:42:48.913 INFO:tasks.workunit.client.0.smithi112.stdout:Building xfs 2024-05-10T10:42:48.970 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] tlibio.lo 2024-05-10T10:42:49.056 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:42:49.094 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-05-10T10:42:49.223 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] write_log.lo 2024-05-10T10:42:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: pgmap v507: 129 pgs: 129 active+clean; 34 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 658 KiB/s wr, 227 op/s 2024-05-10T10:42:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:42:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:42:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:42:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:42:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:42:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:48 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: pgmap v507: 129 pgs: 129 active+clean; 34 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 658 KiB/s wr, 227 op/s 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:48 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: pgmap v507: 129 pgs: 129 active+clean; 34 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 658 KiB/s wr, 227 op/s 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:48 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:49.438 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:42:49.520 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] random.lo 2024-05-10T10:42:49.760 INFO:tasks.workunit.client.0.smithi112.stdout: [LD] 2024-05-10T10:42:50.091 INFO:tasks.workunit.client.0.smithi112.stdout:Building src 2024-05-10T10:42:50.091 INFO:tasks.workunit.client.0.smithi112.stdout:Building ltp 2024-05-10T10:42:50.119 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-10T10:42:50.176 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-10T10:42:50.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:49 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:50.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:49 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:50.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:49 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:50.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:49 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:50.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:49 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:50.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:49 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:42:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:50 smithi148 ceph-mon[33568]: pgmap v508: 129 pgs: 129 active+clean; 31 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 666 KiB/s wr, 159 op/s 2024-05-10T10:42:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:50 smithi195 ceph-mon[33773]: pgmap v508: 129 pgs: 129 active+clean; 31 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 666 KiB/s wr, 159 op/s 2024-05-10T10:42:51.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:50 smithi112 ceph-mon[30100]: pgmap v508: 129 pgs: 129 active+clean; 31 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 666 KiB/s wr, 159 op/s 2024-05-10T10:42:51.847 INFO:tasks.workunit.client.0.smithi112.stdout:Building m4 2024-05-10T10:42:51.876 INFO:tasks.workunit.client.0.smithi112.stdout:Building common 2024-05-10T10:42:52.710 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] doio 2024-05-10T10:42:52.710 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsstress 2024-05-10T10:42:53.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:52 smithi148 ceph-mon[33568]: pgmap v509: 129 pgs: 129 active+clean; 32 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 541 KiB/s wr, 398 op/s 2024-05-10T10:42:53.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:52 smithi195 ceph-mon[33773]: pgmap v509: 129 pgs: 129 active+clean; 32 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 541 KiB/s wr, 398 op/s 2024-05-10T10:42:53.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:52 smithi112 ceph-mon[30100]: pgmap v509: 129 pgs: 129 active+clean; 32 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 541 KiB/s wr, 398 op/s 2024-05-10T10:42:53.781 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsx 2024-05-10T10:42:54.050 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] growfiles 2024-05-10T10:42:55.017 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] iogen 2024-05-10T10:42:55.027 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-stress 2024-05-10T10:42:55.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:54 smithi148 ceph-mon[33568]: pgmap v510: 129 pgs: 129 active+clean; 32 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 506 KiB/s wr, 456 op/s 2024-05-10T10:42:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:54 smithi195 ceph-mon[33773]: pgmap v510: 129 pgs: 129 active+clean; 32 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 506 KiB/s wr, 456 op/s 2024-05-10T10:42:55.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:54 smithi112 ceph-mon[30100]: pgmap v510: 129 pgs: 129 active+clean; 32 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 506 KiB/s wr, 456 op/s 2024-05-10T10:42:56.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 -- 172.21.15.112:0/2951372375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 msgr2=0x7f67c41077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 --2- 172.21.15.112:0/2951372375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c41077b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009990 tx=0x7f67c0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 -- 172.21.15.112:0/2951372375 shutdown_connections 2024-05-10T10:42:56.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 --2- 172.21.15.112:0/2951372375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c4108a80 0x7f67c410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 --2- 172.21.15.112:0/2951372375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c41077b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 --2- 172.21.15.112:0/2951372375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c41080c0 0x7f67c4108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 -- 172.21.15.112:0/2951372375 >> 172.21.15.112:0/2951372375 conn(0x7f67c4077740 msgr2=0x7f67c4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 -- 172.21.15.112:0/2951372375 shutdown_connections 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.566+0000 7f67cb909700 1 -- 172.21.15.112:0/2951372375 wait complete. 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.567+0000 7f67cb909700 1 Processor -- start 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.567+0000 7f67cb909700 1 -- start start 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.568+0000 7f67cb909700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.568+0000 7f67cb909700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41080c0 0x7f67c419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.568+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.568+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38352/0 (socket says 172.21.15.112:38352) 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.568+0000 7f67c96a5700 1 -- 172.21.15.112:0/1708889109 learned_addr learned my addr 172.21.15.112:0/1708889109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.568+0000 7f67cb909700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c4108a80 0x7f67c41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.569+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4113570 con 0x7f67c41073d0 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.569+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41080c0 0x7f67c419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.569+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41133f0 con 0x7f67c4108a80 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.569+0000 7f67c9ea6700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c4108a80 0x7f67c41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.569+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41136f0 con 0x7f67c41080c0 2024-05-10T10:42:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.570+0000 7f67c8ea4700 1 -- 172.21.15.112:0/1708889109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c4108a80 msgr2=0x7f67c41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.570+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c4108a80 0x7f67c41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.570+0000 7f67c8ea4700 1 -- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 msgr2=0x7f67c419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.570+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.570+0000 7f67c8ea4700 1 -- 172.21.15.112:0/1708889109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c419a030 con 0x7f67c41080c0 2024-05-10T10:42:56.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.570+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.571+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41080c0 0x7f67c419fc50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9f0 tx=0x7f67b400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.571+0000 7f67ba7fc700 1 -- 172.21.15.112:0/1708889109 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b400c9c0 con 0x7f67c41080c0 2024-05-10T10:42:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.571+0000 7f67ba7fc700 1 -- 172.21.15.112:0/1708889109 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67b4010430 con 0x7f67c41080c0 2024-05-10T10:42:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.571+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c419a320 con 0x7f67c41080c0 2024-05-10T10:42:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.571+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a65e0 con 0x7f67c41080c0 2024-05-10T10:42:56.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.571+0000 7f67ba7fc700 1 -- 172.21.15.112:0/1708889109 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4018680 con 0x7f67c41080c0 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.575+0000 7f67ba7fc700 1 -- 172.21.15.112:0/1708889109 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f67b4018820 con 0x7f67c41080c0 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.576+0000 7f67d619cb80 1 Processor -- start 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.576+0000 7f67d619cb80 1 -- start start 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.576+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f45590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67d619cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2f8e91940 con 0x55a2f8f3fd50 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67d619cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2f8df6da0 con 0x55a2f8f45ad0 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67d619cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a2f8e8d900 con 0x55a2f8f451b0 2024-05-10T10:42:56.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c9ea6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f45590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c9ea6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f45590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60838/0 (socket says 172.21.15.112:60838) 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c9ea6700 1 -- 172.21.15.112:0/205855071 learned_addr learned my addr 172.21.15.112:0/205855071 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c9ea6700 1 -- 172.21.15.112:0/205855071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 msgr2=0x55a2f8f3f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c9ea6700 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c9ea6700 1 -- 172.21.15.112:0/205855071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 msgr2=0x55a2f8f40190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.577+0000 7f67c9ea6700 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f40190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67c9ea6700 1 -- 172.21.15.112:0/205855071 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f8dde8d0 con 0x55a2f8f451b0 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67c9ea6700 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f45590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67bc003f90 tx=0x7f67bc009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4eb9bbc1cf513f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67ba7fc700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67b00b2980 0x7f67b00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67b00b2980 0x7f67b00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67af7fe700 1 -- 172.21.15.112:0/205855071 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc009e60 con 0x55a2f8f451b0 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67ba7fc700 1 -- 172.21.15.112:0/1708889109 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f67b40e26a0 con 0x7f67c41080c0 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67af7fe700 1 -- 172.21.15.112:0/205855071 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67bc01a070 con 0x55a2f8f451b0 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.578+0000 7f67af7fe700 1 -- 172.21.15.112:0/205855071 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0156e0 con 0x55a2f8f451b0 2024-05-10T10:42:56.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67d619cb80 1 -- 172.21.15.112:0/205855071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 msgr2=0x55a2f8f45590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67d619cb80 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f45590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67bc003f90 tx=0x7f67bc009c20 comp rx=0 tx=0).stop 2024-05-10T10:42:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67b00b2980 0x7f67b00b4e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55a2f8f3bed0 tx=0x7f67c001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67d619cb80 1 -- 172.21.15.112:0/205855071 shutdown_connections 2024-05-10T10:42:56.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67d619cb80 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67d619cb80 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f45590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67d619cb80 1 --2- 172.21.15.112:0/205855071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.580+0000 7f67d619cb80 1 -- 172.21.15.112:0/205855071 >> 172.21.15.112:0/205855071 conn(0x55a2f8e6bb70 msgr2=0x55a2f8f3a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.581+0000 7f67d619cb80 1 -- 172.21.15.112:0/205855071 shutdown_connections 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.581+0000 7f67d619cb80 1 -- 172.21.15.112:0/205855071 wait complete. 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.581+0000 7f67d619cb80 1 -- start start 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.581+0000 7f67d619cb80 1 Processor -- start 2024-05-10T10:42:56.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f4b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f4ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60848/0 (socket says 172.21.15.112:60848) 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c8ea4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f4b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c8ea4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f4b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:54964/0 (socket says 172.21.15.112:54964) 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c9ea6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f4ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c9ea6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f4ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38368/0 (socket says 172.21.15.112:38368) 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2f8f50860 con 0x55a2f8f45ad0 2024-05-10T10:42:56.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2f8e4aed0 con 0x55a2f8f451b0 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a2f8f509e0 con 0x55a2f8f3fd50 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c96a5700 1 -- 172.21.15.112:0/1058346773 learned_addr learned my addr 172.21.15.112:0/1058346773 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c8ea4700 1 -- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 msgr2=0x55a2f8f4b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c8ea4700 1 -- 172.21.15.112:0/1058346773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 msgr2=0x55a2f8f4ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.583+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f4ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67c8ea4700 1 -- 172.21.15.112:0/1058346773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f8e48760 con 0x55a2f8f451b0 2024-05-10T10:42:56.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67d619cb80 10 client.0 _ll_register_callbacks cb 0x55a2f8e255b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:56.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67bc01a070 con 0x55a2f8f451b0 2024-05-10T10:42:56.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67c9ea6700 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f4ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:56.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f4b9b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9c0 tx=0x7f67b4010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe3bea262650415 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67acff9700 10 client.25526 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:42:56.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67acff9700 1 -- 172.21.15.112:0/1058346773 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40a93e0 con 0x55a2f8f451b0 2024-05-10T10:42:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67acff9700 1 -- 172.21.15.112:0/1058346773 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f67b40e2360 con 0x55a2f8f451b0 2024-05-10T10:42:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67acff9700 1 -- 172.21.15.112:0/1058346773 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f67b40a9fd0 con 0x55a2f8f451b0 2024-05-10T10:42:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67d619cb80 10 client.25526 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.584+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55a2f8f5f4f0 con 0x55a2f8f451b0 2024-05-10T10:42:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67acff9700 1 -- 172.21.15.112:0/1058346773 <== mon.2 v2:172.21.15.148:3300/0 4 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f67b40a9b30 con 0x55a2f8f451b0 2024-05-10T10:42:56.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67d619cb80 10 client.25526 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67d619cb80 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67d619cb80 20 client.25526 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8f63db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67d619cb80 4 client.25526 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2f8e9f320 con 0x55a2f8f619c0 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8f63db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67acff9700 10 client.25526 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.585+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8f63db0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df28bcb25a7a6d3 server_cookie=f0c0465b7b98f209 in_seq=0 out_seq=0 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.586+0000 7f67acff9700 1 -- 172.21.15.112:0/1058346773 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2f8e9f320 con 0x55a2f8f619c0 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.586+0000 7f67acff9700 10 client.25526 handle_command_reply: tid=0 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.586+0000 7f67d619cb80 1 client.25526 shutdown 2024-05-10T10:42:56.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.586+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 msgr2=0x55a2f8f4b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.290 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:42:56.290 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:42:56.290 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-10T10:42:56.290 INFO:teuthology.orchestra.run.smithi112.stdout: "38b58198-816b-417e-8076-c94ee7a46221": { 2024-05-10T10:42:56.290 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-10T10:42:56.291 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "38b58198-816b-417e-8076-c94ee7a46221", 2024-05-10T10:42:56.291 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-10T10:42:56.291 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:42:56.291 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:42:56.291 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:42:56.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f4b9b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9c0 tx=0x7f67b4010b10 comp rx=0 tx=0).stop 2024-05-10T10:42:56.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 shutdown_connections 2024-05-10T10:42:56.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8f63db0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f4ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67acff9700 0 client.25526 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1058346773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f4b9b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 >> 172.21.15.112:0/1058346773 conn(0x55a2f8e6bb70 msgr2=0x55a2f8f34b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 shutdown_connections 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.587+0000 7f67d619cb80 1 -- 172.21.15.112:0/1058346773 wait complete. 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.588+0000 7f67d619cb80 20 client.25526 trim_cache size 0 max 16384 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.628+0000 7f67d619cb80 1 Processor -- start 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.628+0000 7f67d619cb80 1 -- start start 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.628+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.628+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f34c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c8ea4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f34c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c9ea6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c8ea4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f34c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:54974/0 (socket says 172.21.15.112:54974) 2024-05-10T10:42:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60852/0 (socket says 172.21.15.112:60852) 2024-05-10T10:42:56.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c9ea6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f69680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38370/0 (socket says 172.21.15.112:38370) 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c8ea4700 1 -- 172.21.15.112:0/1350148925 learned_addr learned my addr 172.21.15.112:0/1350148925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41ad040 con 0x55a2f8f45ad0 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41aec40 con 0x55a2f8f451b0 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41ad710 con 0x55a2f8f3fd50 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c96a5700 1 -- 172.21.15.112:0/1350148925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 msgr2=0x55a2f8f34c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f34c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c96a5700 1 -- 172.21.15.112:0/1350148925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 msgr2=0x55a2f8f69680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f69680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c96a5700 1 -- 172.21.15.112:0/1350148925 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f8f4bd20 con 0x55a2f8f3fd50 2024-05-10T10:42:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.629+0000 7f67c9ea6700 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f69680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.630+0000 7f67c96a5700 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4c810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67c00093f0 tx=0x7f67c001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac76bae19bf627c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.630+0000 7f67aeffd700 1 -- 172.21.15.112:0/1350148925 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0036070 con 0x55a2f8f3fd50 2024-05-10T10:42:56.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.630+0000 7f67aeffd700 1 -- 172.21.15.112:0/1350148925 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67c0031a50 con 0x55a2f8f3fd50 2024-05-10T10:42:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.630+0000 7f67aeffd700 1 -- 172.21.15.112:0/1350148925 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0026430 con 0x55a2f8f3fd50 2024-05-10T10:42:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 msgr2=0x55a2f8f4c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4c810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67c00093f0 tx=0x7f67c001f000 comp rx=0 tx=0).stop 2024-05-10T10:42:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 shutdown_connections 2024-05-10T10:42:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f69680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f4c810 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 --2- 172.21.15.112:0/1350148925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f451b0 0x55a2f8f34c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 >> 172.21.15.112:0/1350148925 conn(0x55a2f8e6bb70 msgr2=0x55a2f8f4c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 shutdown_connections 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.631+0000 7f67d619cb80 1 -- 172.21.15.112:0/1350148925 wait complete. 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.632+0000 7f67d619cb80 1 -- start start 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.632+0000 7f67d619cb80 1 Processor -- start 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.632+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f5e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f5ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67d619cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f5ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67d619cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41ae4b0 con 0x55a2f8f3fd50 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67d619cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41adde0 con 0x55a2f8f45ad0 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67d619cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41aafe0 con 0x55a2f8f451b0 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f5e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67d619cb80 10 client.0 _ll_register_callbacks cb 0x55a2f8e255b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c96a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f5e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38382/0 (socket says 172.21.15.112:38382) 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c96a5700 1 -- 172.21.15.112:0/2521670236 learned_addr learned my addr 172.21.15.112:0/2521670236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c9ea6700 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f5ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f5ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c8ea4700 1 -- 172.21.15.112:0/2521670236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 msgr2=0x55a2f8f5ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f5ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c8ea4700 1 -- 172.21.15.112:0/2521670236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 msgr2=0x55a2f8f5e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f5e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.633+0000 7f67c8ea4700 1 -- 172.21.15.112:0/2521670236 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f8f5b7a0 con 0x55a2f8f451b0 2024-05-10T10:42:56.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.634+0000 7f67c8ea4700 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f5ea40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009fd0 tx=0x7f67b40b2070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acaf4aa71de1c798 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.634+0000 7f67acff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:42:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.634+0000 7f67acff9700 1 -- 172.21.15.112:0/2521670236 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40a9040 con 0x55a2f8f451b0 2024-05-10T10:42:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.634+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c0031a50 con 0x55a2f8f451b0 2024-05-10T10:42:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.634+0000 7f67acff9700 1 -- 172.21.15.112:0/2521670236 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f67b40e2360 con 0x55a2f8f451b0 2024-05-10T10:42:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.634+0000 7f67acff9700 1 -- 172.21.15.112:0/2521670236 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f67b40a93e0 con 0x55a2f8f451b0 2024-05-10T10:42:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 10 client.25573 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:56.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a2f8fcc790 con 0x55a2f8f451b0 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67acff9700 1 -- 172.21.15.112:0/2521670236 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67c0031a50 con 0x55a2f8f451b0 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67acff9700 1 -- 172.21.15.112:0/2521670236 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f67b40a8b30 con 0x55a2f8f451b0 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 10 client.25573 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 10 client.25573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 20 client.25573 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8fced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 4 client.25573 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2f8e993c0 con 0x55a2f8f619c0 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.635+0000 7f67c96a5700 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8fced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67acff9700 10 client.25573 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67c96a5700 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8fced80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390da7857aac7ae9 server_cookie=f0c7bc614f2621cd in_seq=0 out_seq=0 2024-05-10T10:42:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67acff9700 1 -- 172.21.15.112:0/2521670236 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a2f8e993c0 con 0x55a2f8f619c0 2024-05-10T10:42:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67acff9700 10 client.25573 handle_command_reply: tid=0 2024-05-10T10:42:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67d619cb80 1 client.25573 shutdown 2024-05-10T10:42:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 msgr2=0x55a2f8f5ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67d619cb80 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f5ea40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009fd0 tx=0x7f67b40b2070 comp rx=0 tx=0).stop 2024-05-10T10:42:56.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 shutdown_connections 2024-05-10T10:42:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.636+0000 7f67d619cb80 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2f8f619c0 0x55a2f8fced80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67d619cb80 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2f8f45ad0 0x55a2f8f5ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67d619cb80 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2f8f3fd50 0x55a2f8f5e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67d619cb80 1 --2- 172.21.15.112:0/2521670236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2f8f451b0 0x55a2f8f5ea40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 >> 172.21.15.112:0/2521670236 conn(0x55a2f8e6bb70 msgr2=0x55a2f8f43cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67acff9700 0 client.25573 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 shutdown_connections 2024-05-10T10:42:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67d619cb80 1 -- 172.21.15.112:0/2521670236 wait complete. 2024-05-10T10:42:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.637+0000 7f67d619cb80 20 client.25573 trim_cache size 0 max 16384 2024-05-10T10:42:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67b00b2980 msgr2=0x7f67b00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67b00b2980 0x7f67b00b4e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55a2f8f3bed0 tx=0x7f67c001a040 comp rx=0 tx=0).stop 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41080c0 msgr2=0x7f67c419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41080c0 0x7f67c419fc50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9f0 tx=0x7f67b400ed00 comp rx=0 tx=0).stop 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 shutdown_connections 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c4108a80 0x7f67c41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c41073d0 0x7f67c419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67b00b2980 0x7f67b00b4e40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 --2- 172.21.15.112:0/1708889109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67c41080c0 0x7f67c419fc50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 >> 172.21.15.112:0/1708889109 conn(0x7f67c4077740 msgr2=0x7f67c4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 shutdown_connections 2024-05-10T10:42:56.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:42:55.638+0000 7f67cb909700 1 -- 172.21.15.112:0/1708889109 wait complete. 2024-05-10T10:42:57.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:56 smithi112 ceph-mon[30100]: pgmap v511: 129 pgs: 129 active+clean; 32 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 497 KiB/s wr, 450 op/s 2024-05-10T10:42:57.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:56 smithi148 ceph-mon[33568]: pgmap v511: 129 pgs: 129 active+clean; 32 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 497 KiB/s wr, 450 op/s 2024-05-10T10:42:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:56 smithi195 ceph-mon[33773]: pgmap v511: 129 pgs: 129 active+clean; 32 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 497 KiB/s wr, 450 op/s 2024-05-10T10:42:57.508 INFO:tasks.cephfs.filesystem:scrub status for tag:38b58198-816b-417e-8076-c94ee7a46221 - {'path': '/', 'tag': '38b58198-816b-417e-8076-c94ee7a46221', 'options': 'recursive,force'} 2024-05-10T10:42:58.378 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-10T10:42:58.380 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-10T10:42:58.384 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-10T10:42:58.389 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-10T10:42:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 -- 172.21.15.112:0/3373485588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078103cd0 msgr2=0x7fe078104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 --2- 172.21.15.112:0/3373485588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078103cd0 0x7fe078104150 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe068009930 tx=0x7fe06802a420 comp rx=0 tx=0).stop 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 -- 172.21.15.112:0/3373485588 shutdown_connections 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 --2- 172.21.15.112:0/3373485588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe078104690 0x7fe07810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 --2- 172.21.15.112:0/3373485588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078103cd0 0x7fe078104150 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 --2- 172.21.15.112:0/3373485588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe078100d00 0x7fe078101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 -- 172.21.15.112:0/3373485588 >> 172.21.15.112:0/3373485588 conn(0x7fe0780fc630 msgr2=0x7fe0780fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 -- 172.21.15.112:0/3373485588 shutdown_connections 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.534+0000 7fe0800c4640 1 -- 172.21.15.112:0/3373485588 wait complete. 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 Processor -- start 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 -- start start 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe078100d00 0x7fe078198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe078103cd0 0x7fe078199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078104690 0x7fe0781a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e440 con 0x7fe078104690 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e2c0 con 0x7fe078100d00 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe0800c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e5c0 con 0x7fe078103cd0 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078104690 0x7fe0781a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078104690 0x7fe0781a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38404/0 (socket says 172.21.15.112:38404) 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 -- 172.21.15.112:0/2378143119 learned_addr learned my addr 172.21.15.112:0/2378143119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07de60640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe078100d00 0x7fe078198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07d65f640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe078103cd0 0x7fe078199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 -- 172.21.15.112:0/2378143119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe078103cd0 msgr2=0x7fe078199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe078103cd0 0x7fe078199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 -- 172.21.15.112:0/2378143119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe078100d00 msgr2=0x7fe078198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe078100d00 0x7fe078198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07e661640 1 -- 172.21.15.112:0/2378143119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0781a0730 con 0x7fe078104690 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.535+0000 7fe07d65f640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe078103cd0 0x7fe078199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.536+0000 7fe07e661640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078104690 0x7fe0781a00c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe07000eb10 tx=0x7fe07000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.536+0000 7fe06effd640 1 -- 172.21.15.112:0/2378143119 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070004b20 con 0x7fe078104690 2024-05-10T10:42:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.536+0000 7fe06effd640 1 -- 172.21.15.112:0/2378143119 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe070004cc0 con 0x7fe078104690 2024-05-10T10:42:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.536+0000 7fe06effd640 1 -- 172.21.15.112:0/2378143119 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070010690 con 0x7fe078104690 2024-05-10T10:42:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.536+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0781a0a20 con 0x7fe078104690 2024-05-10T10:42:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.536+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0781a0fc0 con 0x7fe078104690 2024-05-10T10:42:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.538+0000 7fe06effd640 1 -- 172.21.15.112:0/2378143119 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe070005020 con 0x7fe078104690 2024-05-10T10:42:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 Processor -- start 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 -- start start 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eb9d2275d0 con 0x55eb9d2ccd20 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb9d1e0890 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe082dd8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb9d223590 con 0x55eb9d2c71c0 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe07d65f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe07d65f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38416/0 (socket says 172.21.15.112:38416) 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe07d65f640 1 -- 172.21.15.112:0/805385036 learned_addr learned my addr 172.21.15.112:0/805385036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe07d65f640 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.539+0000 7fe07de60640 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe06effd640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe0640b2920 0x7fe0640b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe07d65f640 1 -- 172.21.15.112:0/805385036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 msgr2=0x55eb9d2c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe07d65f640 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe07d65f640 1 -- 172.21.15.112:0/805385036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 msgr2=0x55eb9d2cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe07d65f640 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe07d65f640 1 -- 172.21.15.112:0/805385036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9cf39210 con 0x55eb9d2ccd20 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe07e661640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe0640b2920 0x7fe0640b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe07d65f640 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2cf1b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe068005c20 tx=0x7fe068000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1e17f8a6fadbaa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe06effd640 1 -- 172.21.15.112:0/2378143119 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe0700e2bb0 con 0x7fe078104690 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe063fff640 1 -- 172.21.15.112:0/805385036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068035050 con 0x55eb9d2ccd20 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe063fff640 1 -- 172.21.15.112:0/805385036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0680044c0 con 0x55eb9d2ccd20 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.540+0000 7fe063fff640 1 -- 172.21.15.112:0/805385036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0680039b0 con 0x55eb9d2ccd20 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 -- 172.21.15.112:0/805385036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2ccd20 msgr2=0x55eb9d2cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2cf1b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe068005c20 tx=0x7fe068000bc0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe07e661640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe0640b2920 0x7fe0640b4de0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe0700049d0 tx=0x7fe070000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 -- 172.21.15.112:0/805385036 shutdown_connections 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2cf1b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 --2- 172.21.15.112:0/805385036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 -- 172.21.15.112:0/805385036 >> 172.21.15.112:0/805385036 conn(0x55eb9d2be420 msgr2=0x55eb9d2be820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 -- 172.21.15.112:0/805385036 shutdown_connections 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 -- 172.21.15.112:0/805385036 wait complete. 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.541+0000 7fe082dd8740 1 -- start start 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 1 Processor -- start 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eb9d2cfc70 con 0x55eb9d2c71c0 2024-05-10T10:42:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb9d1e4370 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb9d2cfdf0 con 0x55eb9d2ccd20 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe082dd8740 10 client.0 _ll_register_callbacks cb 0x55eb9cf38a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38424/0 (socket says 172.21.15.112:38424) 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 -- 172.21.15.112:0/1707183462 learned_addr learned my addr 172.21.15.112:0/1707183462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07de60640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 -- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 msgr2=0x55eb9d2d89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 -- 172.21.15.112:0/1707183462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 msgr2=0x55eb9d2d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07d65f640 1 -- 172.21.15.112:0/1707183462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9d005620 con 0x55eb9d2c71c0 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.542+0000 7fe07de60640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe07d65f640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d1430 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe068005ec0 tx=0x7fe068001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3474639cbba23a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe0617fa640 1 -- 172.21.15.112:0/1707183462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068035070 con 0x55eb9d2c71c0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb9d2351e0 con 0x55eb9d2c71c0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe0617fa640 1 -- 172.21.15.112:0/1707183462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe06803d890 con 0x55eb9d2c71c0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe0617fa640 1 -- 172.21.15.112:0/1707183462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe06805b050 con 0x55eb9d2c71c0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 10 client.16155 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb9d2db990 con 0x55eb9d2c71c0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe0617fa640 1 -- 172.21.15.112:0/1707183462 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55eb9d2351e0 con 0x55eb9d2c71c0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe0617fa640 1 -- 172.21.15.112:0/1707183462 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fe06803a040 con 0x55eb9d2c71c0 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 10 client.16155 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 20 client.16155 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d2dcce0 0x55eb9d2df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 4 client.16155 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.543+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb9d22f050 con 0x55eb9d2dcce0 2024-05-10T10:42:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.544+0000 7fe07de60640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d2dcce0 0x55eb9d2df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.544+0000 7fe07de60640 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d2dcce0 0x55eb9d2df0d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70454d99eafa2e2 server_cookie=7919ef0a46d33d00 in_seq=0 out_seq=0 2024-05-10T10:42:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.544+0000 7fe0617fa640 10 client.16155 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe0617fa640 1 -- 172.21.15.112:0/1707183462 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb9d22f050 con 0x55eb9d2dcce0 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe0617fa640 10 client.16155 handle_command_reply: tid=0 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 client.16155 shutdown 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 msgr2=0x55eb9d2d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d1430 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe068005ec0 tx=0x7fe068001e70 comp rx=0 tx=0).stop 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 shutdown_connections 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d2dcce0 0x55eb9d2df0d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 --2- 172.21.15.112:0/1707183462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d1430 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe0617fa640 0 client.16155 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 >> 172.21.15.112:0/1707183462 conn(0x55eb9d2be420 msgr2=0x55eb9d2bd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 shutdown_connections 2024-05-10T10:42:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 1 -- 172.21.15.112:0/1707183462 wait complete. 2024-05-10T10:42:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.545+0000 7fe082dd8740 20 client.16155 trim_cache size 0 max 16384 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 Processor -- start 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 -- start start 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781a9280 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781aae80 con 0x55eb9d2ccd20 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.560+0000 7fe082dd8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781a9950 con 0x55eb9d2c71c0 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38428/0 (socket says 172.21.15.112:38428) 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 -- 172.21.15.112:0/680559759 learned_addr learned my addr 172.21.15.112:0/680559759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07d65f640 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07d65f640 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 -- 172.21.15.112:0/680559759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 msgr2=0x55eb9d2c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 -- 172.21.15.112:0/680559759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2ccd20 msgr2=0x55eb9d2d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 -- 172.21.15.112:0/680559759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9d2be250 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07d65f640 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe07de60640 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55eb9d2c8620 tx=0x7fe074005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86131f588f327 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe0637fe640 1 -- 172.21.15.112:0/680559759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074004050 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe0637fe640 1 -- 172.21.15.112:0/680559759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe074005020 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.561+0000 7fe0637fe640 1 -- 172.21.15.112:0/680559759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0740049e0 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 -- 172.21.15.112:0/680559759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c7bb0 msgr2=0x55eb9d2d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55eb9d2c8620 tx=0x7fe074005f90 comp rx=0 tx=0).stop 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 -- 172.21.15.112:0/680559759 shutdown_connections 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d1040 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 --2- 172.21.15.112:0/680559759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 -- 172.21.15.112:0/680559759 >> 172.21.15.112:0/680559759 conn(0x55eb9d2be420 msgr2=0x55eb9d2c1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 -- 172.21.15.112:0/680559759 shutdown_connections 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.562+0000 7fe082dd8740 1 -- 172.21.15.112:0/680559759 wait complete. 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 -- start start 2024-05-10T10:42:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 Processor -- start 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781aa6f0 con 0x55eb9d2c71c0 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781aa020 con 0x55eb9d2c7bb0 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781a7260 con 0x55eb9d2ccd20 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe082dd8740 10 client.0 _ll_register_callbacks cb 0x7fe074009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38434/0 (socket says 172.21.15.112:38434) 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 -- 172.21.15.112:0/839473858 learned_addr learned my addr 172.21.15.112:0/839473858 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07de60640 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 -- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 msgr2=0x55eb9d2d6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 -- 172.21.15.112:0/839473858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 msgr2=0x55eb9d2d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.563+0000 7fe07d65f640 1 -- 172.21.15.112:0/839473858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9d2d7510 con 0x55eb9d2c71c0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe07d65f640 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d59d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe068009930 tx=0x7fe06804e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906937f81610755c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe0617fa640 1 -- 172.21.15.112:0/839473858 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068067020 con 0x55eb9d2c71c0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe0617fa640 1 -- 172.21.15.112:0/839473858 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe06803d690 con 0x55eb9d2c71c0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe074005020 con 0x55eb9d2c71c0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe0617fa640 1 -- 172.21.15.112:0/839473858 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe06806f050 con 0x55eb9d2c71c0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 10 client.16161 fetch_fsmap learned FSMap version 34 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb9d3477c0 con 0x55eb9d2c71c0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe0617fa640 1 -- 172.21.15.112:0/839473858 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe074005020 con 0x55eb9d2c71c0 2024-05-10T10:42:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe0617fa640 1 -- 172.21.15.112:0/839473858 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fe06804a420 con 0x55eb9d2c71c0 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 10 client.16161 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 20 client.16161 populate_metadata read hostname 'smithi112' 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d348190 0x55eb9d34a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 4 client.16161 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.564+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb9d1aa5f0 con 0x55eb9d348190 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.565+0000 7fe07de60640 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d348190 0x55eb9d34a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.565+0000 7fe07de60640 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d348190 0x55eb9d34a5b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15dc101e6ddc9c64 server_cookie=716497e7c0d15d9f in_seq=0 out_seq=0 2024-05-10T10:42:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.565+0000 7fe0617fa640 10 client.16161 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.583+0000 7fe0617fa640 1 -- 172.21.15.112:0/839473858 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb9d1aa5f0 con 0x55eb9d348190 2024-05-10T10:42:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.583+0000 7fe0617fa640 10 client.16161 handle_command_reply: tid=0 2024-05-10T10:42:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.583+0000 7fe082dd8740 1 client.16161 shutdown 2024-05-10T10:42:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.583+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 msgr2=0x55eb9d2d59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.583+0000 7fe082dd8740 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d59d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe068009930 tx=0x7fe06804e040 comp rx=0 tx=0).stop 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 shutdown_connections 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb9d2ccd20 0x55eb9d2d6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55eb9d348190 0x55eb9d34a5b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9d2c7bb0 0x55eb9d2d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 --2- 172.21.15.112:0/839473858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eb9d2c71c0 0x55eb9d2d59d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0617fa640 0 client.16161 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 >> 172.21.15.112:0/839473858 conn(0x55eb9d2be420 msgr2=0x55eb9d2d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 shutdown_connections 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 1 -- 172.21.15.112:0/839473858 wait complete. 2024-05-10T10:42:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe082dd8740 20 client.16161 trim_cache size 0 max 16384 2024-05-10T10:42:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:42:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:42:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:42:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe0640b2920 msgr2=0x7fe0640b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe0640b2920 0x7fe0640b4de0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe0700049d0 tx=0x7fe070000f40 comp rx=0 tx=0).stop 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078104690 msgr2=0x7fe0781a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078104690 0x7fe0781a00c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe07000eb10 tx=0x7fe07000c2d0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 shutdown_connections 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe0640b2920 0x7fe0640b4de0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe078104690 0x7fe0781a00c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe078103cd0 0x7fe078199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 --2- 172.21.15.112:0/2378143119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe078100d00 0x7fe078198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.584+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 >> 172.21.15.112:0/2378143119 conn(0x7fe0780fc630 msgr2=0x7fe078107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.585+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 shutdown_connections 2024-05-10T10:42:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:42:58.585+0000 7fe0800c4640 1 -- 172.21.15.112:0/2378143119 wait complete. 2024-05-10T10:42:59.015 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirstress 2024-05-10T10:42:59.015 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fill 2024-05-10T10:42:59.015 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fill2 2024-05-10T10:42:59.016 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] getpagesize 2024-05-10T10:42:59.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:58 smithi148 ceph-mon[33568]: pgmap v512: 129 pgs: 129 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 228 KiB/s wr, 418 op/s 2024-05-10T10:42:59.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:58 smithi195 ceph-mon[33773]: pgmap v512: 129 pgs: 129 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 228 KiB/s wr, 418 op/s 2024-05-10T10:42:59.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:58 smithi112 ceph-mon[30100]: pgmap v512: 129 pgs: 129 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 228 KiB/s wr, 418 op/s 2024-05-10T10:42:59.382 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] holes 2024-05-10T10:42:59.384 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] lstat64 2024-05-10T10:42:59.514 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] nametest 2024-05-10T10:42:59.529 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] permname 2024-05-10T10:42:59.760 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] randholes 2024-05-10T10:42:59.803 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] runas 2024-05-10T10:42:59.942 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] truncfile 2024-05-10T10:42:59.974 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] usemem 2024-05-10T10:43:00.205 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmapcat 2024-05-10T10:43:00.266 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] append_reader 2024-05-10T10:43:00.286 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] append_writer 2024-05-10T10:43:00.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:59 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:43:00.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:59 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:43:00.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:42:59 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:43:00.315 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirperf 2024-05-10T10:43:00.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:59 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:43:00.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:59 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:43:00.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:42:59 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:43:00.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:59 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:43:00.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:59 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:43:00.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:42:59 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:43:00.521 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] metaperf 2024-05-10T10:43:00.623 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] devzero 2024-05-10T10:43:00.638 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] feature 2024-05-10T10:43:00.785 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] alloc 2024-05-10T10:43:01.003 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fault 2024-05-10T10:43:01.028 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fstest 2024-05-10T10:43:01.057 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_access_root 2024-05-10T10:43:01.245 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] godown 2024-05-10T10:43:01.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:00 smithi148 ceph-mon[33568]: pgmap v513: 129 pgs: 129 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 245 KiB/s wr, 417 op/s 2024-05-10T10:43:01.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:00 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:43:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:00 smithi195 ceph-mon[33773]: pgmap v513: 129 pgs: 129 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 245 KiB/s wr, 417 op/s 2024-05-10T10:43:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:00 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:43:01.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:00 smithi112 ceph-mon[30100]: pgmap v513: 129 pgs: 129 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 245 KiB/s wr, 417 op/s 2024-05-10T10:43:01.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:00 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:43:01.363 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] resvtest 2024-05-10T10:43:01.386 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] writemod 2024-05-10T10:43:01.484 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] writev_on_pagefault 2024-05-10T10:43:01.630 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] makeextents 2024-05-10T10:43:01.702 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] itrash 2024-05-10T10:43:01.768 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] rename 2024-05-10T10:43:01.825 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] multi_open_unlink 2024-05-10T10:43:02.028 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] unwritten_sync 2024-05-10T10:43:02.029 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] genhashnames 2024-05-10T10:43:02.058 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_holes 2024-05-10T10:43:02.163 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_writev 2024-05-10T10:43:02.378 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_truncate_cmtime 2024-05-10T10:43:02.420 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirhash_collide 2024-05-10T10:43:02.433 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_rename_overwrite 2024-05-10T10:43:02.474 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] holetest 2024-05-10T10:43:02.738 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_truncate_self 2024-05-10T10:43:02.747 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] af_unix 2024-05-10T10:43:02.790 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_stale_pmd 2024-05-10T10:43:02.915 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_cow_race 2024-05-10T10:43:03.046 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_fallocate 2024-05-10T10:43:03.103 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsync-err 2024-05-10T10:43:03.133 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_write_ro 2024-05-10T10:43:03.272 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ext4_dax_journal_corruption 2024-05-10T10:43:03.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:02 smithi148 ceph-mon[33568]: pgmap v514: 129 pgs: 129 active+clean; 33 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 248 KiB/s wr, 398 op/s 2024-05-10T10:43:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:02 smithi195 ceph-mon[33773]: pgmap v514: 129 pgs: 129 active+clean; 33 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 248 KiB/s wr, 398 op/s 2024-05-10T10:43:03.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:02 smithi112 ceph-mon[30100]: pgmap v514: 129 pgs: 129 active+clean; 33 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 248 KiB/s wr, 398 op/s 2024-05-10T10:43:03.363 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ext4_dax_inline_corruption 2024-05-10T10:43:03.459 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ofd_locks 2024-05-10T10:43:03.486 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_collision 2024-05-10T10:43:03.608 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmap-write-concurrent 2024-05-10T10:43:03.699 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_get_file_time 2024-05-10T10:43:03.853 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_create_short_dirs 2024-05-10T10:43:03.904 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_create_long_dirs 2024-05-10T10:43:03.935 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_enospc 2024-05-10T10:43:04.016 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_writev_overlap 2024-05-10T10:43:04.211 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] checkpoint_journal 2024-05-10T10:43:04.258 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmap-rw-fault 2024-05-10T10:43:04.322 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] allocstale 2024-05-10T10:43:04.363 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_cow_memory_failure 2024-05-10T10:43:04.554 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fake-dump-rootino 2024-05-10T10:43:04.630 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-buf-fault 2024-05-10T10:43:04.694 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] rewinddir-test 2024-05-10T10:43:04.723 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] readdir-while-renames 2024-05-10T10:43:04.887 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] xfsctl 2024-05-10T10:43:04.952 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bstat 2024-05-10T10:43:05.057 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mtab 2024-05-10T10:43:05.068 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] getdevicesize 2024-05-10T10:43:05.287 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] preallo_rw_pattern_reader 2024-05-10T10:43:05.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:04 smithi148 ceph-mon[33568]: pgmap v515: 129 pgs: 129 active+clean; 33 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 169 KiB/s wr, 148 op/s 2024-05-10T10:43:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:04 smithi195 ceph-mon[33773]: pgmap v515: 129 pgs: 129 active+clean; 33 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 169 KiB/s wr, 148 op/s 2024-05-10T10:43:05.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:04 smithi112 ceph-mon[30100]: pgmap v515: 129 pgs: 129 active+clean; 33 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 169 KiB/s wr, 148 op/s 2024-05-10T10:43:05.366 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] preallo_rw_pattern_writer 2024-05-10T10:43:05.371 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] ftrunc 2024-05-10T10:43:05.443 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] trunc 2024-05-10T10:43:05.657 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fs_perms 2024-05-10T10:43:05.701 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] testx 2024-05-10T10:43:05.730 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] looptest 2024-05-10T10:43:05.828 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] locktest 2024-05-10T10:43:06.007 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] unwritten_mmap 2024-05-10T10:43:06.016 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_unlink_test 2024-05-10T10:43:06.114 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] deduperace 2024-05-10T10:43:06.374 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_unlink_test_modified 2024-05-10T10:43:06.381 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_offset 2024-05-10T10:43:06.422 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_futimens 2024-05-10T10:43:06.486 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_immutable 2024-05-10T10:43:06.727 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] stale_handle 2024-05-10T10:43:06.743 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] pwrite_mmap_blocked 2024-05-10T10:43:06.785 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_offset2 2024-05-10T10:43:07.097 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] seek_sanity_test 2024-05-10T10:43:07.102 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] seek_copy_test 2024-05-10T10:43:07.166 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_1 2024-05-10T10:43:07.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:06 smithi148 ceph-mon[33568]: pgmap v516: 129 pgs: 129 active+clean; 34 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 256 KiB/s wr, 93 op/s 2024-05-10T10:43:07.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:06 smithi195 ceph-mon[33773]: pgmap v516: 129 pgs: 129 active+clean; 34 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 256 KiB/s wr, 93 op/s 2024-05-10T10:43:07.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:06 smithi112 ceph-mon[30100]: pgmap v516: 129 pgs: 129 active+clean; 34 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 256 KiB/s wr, 93 op/s 2024-05-10T10:43:07.443 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_2 2024-05-10T10:43:07.463 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsync-tester 2024-05-10T10:43:07.480 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] nsexec 2024-05-10T10:43:07.769 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] cloner 2024-05-10T10:43:07.824 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] renameat2 2024-05-10T10:43:07.833 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_getcwd 2024-05-10T10:43:07.906 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] e4compact 2024-05-10T10:43:08.161 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] test-nextquota 2024-05-10T10:43:08.181 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] punch-alternating 2024-05-10T10:43:08.207 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] attr-list-by-handle-cursor-test 2024-05-10T10:43:08.325 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] listxattr 2024-05-10T10:43:08.575 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-interleaved 2024-05-10T10:43:08.576 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_type 2024-05-10T10:43:08.582 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-invalidate-cache 2024-05-10T10:43:08.621 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] stat_test 2024-05-10T10:43:08.948 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_encrypted_d_revalidate 2024-05-10T10:43:08.951 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] attr_replace_test 2024-05-10T10:43:09.012 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] swapon 2024-05-10T10:43:09.142 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mkswap 2024-05-10T10:43:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:08 smithi148 ceph-mon[33568]: pgmap v517: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 285 KiB/s wr, 109 op/s 2024-05-10T10:43:09.309 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_attr_corruption 2024-05-10T10:43:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:08 smithi195 ceph-mon[33773]: pgmap v517: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 285 KiB/s wr, 109 op/s 2024-05-10T10:43:09.316 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_open_tmpfiles 2024-05-10T10:43:09.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:08 smithi112 ceph-mon[30100]: pgmap v517: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 285 KiB/s wr, 109 op/s 2024-05-10T10:43:09.357 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fscrypt-crypt-util 2024-05-10T10:43:09.461 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_null_ocount 2024-05-10T10:43:09.673 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] splice-test 2024-05-10T10:43:09.736 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] chprojid_fail 2024-05-10T10:43:09.800 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] detached_mounts_propagation 2024-05-10T10:43:10.038 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] ext4_resize 2024-05-10T10:43:10.094 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_3 2024-05-10T10:43:10.194 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] splice2pipe 2024-05-10T10:43:10.320 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] uuid_ioctl 2024-05-10T10:43:10.350 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_snapshot_deleted_subvolume 2024-05-10T10:43:10.483 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fiemap-fault 2024-05-10T10:43:10.531 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fiemap-tester 2024-05-10T10:43:10.664 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_stripealign 2024-05-10T10:43:10.729 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] open_by_handle 2024-05-10T10:43:10.826 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_dio 2024-05-10T10:43:10.997 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] xfsfind 2024-05-10T10:43:11.033 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_reflink_read_race 2024-05-10T10:43:11.179 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fssum 2024-05-10T10:43:11.245 INFO:tasks.workunit.client.0.smithi112.stdout:Building log-writes 2024-05-10T10:43:11.288 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] replay-log 2024-05-10T10:43:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:10 smithi148 ceph-mon[33568]: pgmap v518: 129 pgs: 129 active+clean; 35 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 277 KiB/s wr, 123 op/s 2024-05-10T10:43:11.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:10 smithi195 ceph-mon[33773]: pgmap v518: 129 pgs: 129 active+clean; 35 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 277 KiB/s wr, 123 op/s 2024-05-10T10:43:11.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:10 smithi112 ceph-mon[30100]: pgmap v518: 129 pgs: 129 active+clean; 35 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 277 KiB/s wr, 123 op/s 2024-05-10T10:43:11.452 INFO:tasks.workunit.client.0.smithi112.stdout:Building perf 2024-05-10T10:43:11.455 INFO:tasks.workunit.client.0.smithi112.stdout:Building aio-dio-regress 2024-05-10T10:43:11.472 INFO:tasks.workunit.client.0.smithi112.stdout:Building vfs 2024-05-10T10:43:11.495 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aiocp 2024-05-10T10:43:11.505 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] vfstest 2024-05-10T10:43:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 -- 172.21.15.112:0/2277892874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff08076990 msgr2=0x7eff08074de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 --2- 172.21.15.112:0/2277892874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff08076990 0x7eff08074de0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efef8009900 tx=0x7efef802a480 comp rx=0 tx=0).stop 2024-05-10T10:43:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 -- 172.21.15.112:0/2277892874 shutdown_connections 2024-05-10T10:43:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 --2- 172.21.15.112:0/2277892874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0810a8c0 0x7eff0810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 --2- 172.21.15.112:0/2277892874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff080753b0 0x7eff08075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 --2- 172.21.15.112:0/2277892874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff08076990 0x7eff08074de0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 -- 172.21.15.112:0/2277892874 >> 172.21.15.112:0/2277892874 conn(0x7eff080fdfa0 msgr2=0x7eff081003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 -- 172.21.15.112:0/2277892874 shutdown_connections 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.701+0000 7eff0e015640 1 -- 172.21.15.112:0/2277892874 wait complete. 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.702+0000 7eff0e015640 1 Processor -- start 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.702+0000 7eff0e015640 1 -- start start 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff0e015640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff080753b0 0x7eff0819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff0e015640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff08076990 0x7eff0819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff0e015640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0810a8c0 0x7eff081a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff0e015640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff081106b0 con 0x7eff080753b0 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff0e015640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff08110530 con 0x7eff0810a8c0 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff0e015640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff08110830 con 0x7eff08076990 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff080753b0 0x7eff0819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff080753b0 0x7eff0819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34250/0 (socket says 172.21.15.112:34250) 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff077fe640 1 -- 172.21.15.112:0/3115801836 learned_addr learned my addr 172.21.15.112:0/3115801836 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff07fff640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0810a8c0 0x7eff081a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff07fff640 1 -- 172.21.15.112:0/3115801836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff08076990 msgr2=0x7eff0819b280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff07fff640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff08076990 0x7eff0819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff07fff640 1 -- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff080753b0 msgr2=0x7eff0819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff07fff640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff080753b0 0x7eff0819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff07fff640 1 -- 172.21.15.112:0/3115801836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff081a2950 con 0x7eff0810a8c0 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff077fe640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff080753b0 0x7eff0819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.703+0000 7eff07fff640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0810a8c0 0x7eff081a22b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff0000be20 tx=0x7eff0000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.704+0000 7eff04ff9640 1 -- 172.21.15.112:0/3115801836 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00019070 con 0x7eff0810a8c0 2024-05-10T10:43:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.704+0000 7eff04ff9640 1 -- 172.21.15.112:0/3115801836 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff000148d0 con 0x7eff0810a8c0 2024-05-10T10:43:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.704+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff081a2c40 con 0x7eff0810a8c0 2024-05-10T10:43:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.704+0000 7eff04ff9640 1 -- 172.21.15.112:0/3115801836 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000127e0 con 0x7eff0810a8c0 2024-05-10T10:43:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.704+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff081a3140 con 0x7eff0810a8c0 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 Processor -- start 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff04ff9640 1 -- 172.21.15.112:0/3115801836 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7eff0001c070 con 0x7eff0810a8c0 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 -- start start 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa35c0 0x55893bfa39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfa8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa9120 0x55893bfab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55893beff5d0 con 0x55893bfa9120 2024-05-10T10:43:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55893beb8890 con 0x55893bfa3fb0 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.706+0000 7eff10e07740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55893befb590 con 0x55893bfa35c0 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfa8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfa8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:40818/0 (socket says 172.21.15.112:40818) 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 -- 172.21.15.112:0/38187186 learned_addr learned my addr 172.21.15.112:0/38187186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff06ffd640 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa9120 0x55893bfab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 -- 172.21.15.112:0/38187186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa35c0 msgr2=0x55893bfa39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa35c0 0x55893bfa39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 -- 172.21.15.112:0/38187186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa9120 msgr2=0x55893bfab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa9120 0x55893bfab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 -- 172.21.15.112:0/38187186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893bc11210 con 0x55893bfa3fb0 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff06ffd640 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa9120 0x55893bfab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff04ff9640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efef00b2920 0x7efef00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff077fe640 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfa8ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efef80057c0 tx=0x7efef8004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106f3d9f2cb07f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff04ff9640 1 -- 172.21.15.112:0/3115801836 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7eff000e2040 con 0x7eff0810a8c0 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.707+0000 7eff06ffd640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efef00b2920 0x7efef00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7efeedffb640 1 -- 172.21.15.112:0/38187186 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8035030 con 0x55893bfa3fb0 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7efeedffb640 1 -- 172.21.15.112:0/38187186 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efef8040070 con 0x55893bfa3fb0 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7efeedffb640 1 -- 172.21.15.112:0/38187186 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8031da0 con 0x55893bfa3fb0 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 -- 172.21.15.112:0/38187186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 msgr2=0x55893bfa8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfa8ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efef80057c0 tx=0x7efef8004610 comp rx=0 tx=0).stop 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff06ffd640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efef00b2920 0x7efef00b4de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55893bfa4810 tx=0x7efefc00a4f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 -- 172.21.15.112:0/38187186 shutdown_connections 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa9120 0x55893bfab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfa8ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 --2- 172.21.15.112:0/38187186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa35c0 0x55893bfa39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 -- 172.21.15.112:0/38187186 >> 172.21.15.112:0/38187186 conn(0x55893bf96420 msgr2=0x55893bf96820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 -- 172.21.15.112:0/38187186 shutdown_connections 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.708+0000 7eff10e07740 1 -- 172.21.15.112:0/38187186 wait complete. 2024-05-10T10:43:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 -- start start 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 Processor -- start 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfadd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bfb4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55893bfac070 con 0x55893bfa35c0 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55893bebc370 con 0x55893bfa3fb0 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55893bfac1f0 con 0x55893bfa9120 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff10e07740 10 client.0 _ll_register_callbacks cb 0x55893bc10a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff06ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfadd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff06ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfadd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:40824/0 (socket says 172.21.15.112:40824) 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.709+0000 7eff06ffd640 1 -- 172.21.15.112:0/3012441008 learned_addr learned my addr 172.21.15.112:0/3012441008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff07fff640 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bfb4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff06ffd640 1 -- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 msgr2=0x55893bfb4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff06ffd640 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bfb4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff06ffd640 1 -- 172.21.15.112:0/3012441008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 msgr2=0x55893bfad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff06ffd640 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff06ffd640 1 -- 172.21.15.112:0/3012441008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893bcdd620 con 0x55893bfa3fb0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff077fe640 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff06ffd640 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfadd10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efefc0116c0 tx=0x7efefc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecddf0470a1f8ea0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7efee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7efee6ffd640 1 -- 172.21.15.112:0/3012441008 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc01e8f0 con 0x55893bfa3fb0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7efee6ffd640 1 -- 172.21.15.112:0/3012441008 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efefc01ea90 con 0x55893bfa3fb0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef8040070 con 0x55893bfa3fb0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7efee6ffd640 1 -- 172.21.15.112:0/3012441008 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efefc01ec90 con 0x55893bfa3fb0 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.710+0000 7eff10e07740 10 client.25562 fetch_fsmap learned FSMap version 35 2024-05-10T10:43:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55893bfb7df0 con 0x55893bfa3fb0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7efee6ffd640 1 -- 172.21.15.112:0/3012441008 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efef8040070 con 0x55893bfa3fb0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7efee6ffd640 1 -- 172.21.15.112:0/3012441008 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7efefc01c3b0 con 0x55893bfa3fb0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff10e07740 10 client.25562 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff10e07740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff10e07740 20 client.25562 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff10e07740 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893bfbba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff10e07740 4 client.25562 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55893bf0d1e0 con 0x55893bfb95a0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.711+0000 7eff077fe640 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893bfbba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.712+0000 7eff077fe640 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893bfbba60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b82e6124fb10c2 server_cookie=57f9c25a3ca7de80 in_seq=0 out_seq=0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.712+0000 7efee6ffd640 10 client.25562 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.712+0000 7efee6ffd640 1 -- 172.21.15.112:0/3012441008 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55893bf0d1e0 con 0x55893bfb95a0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.712+0000 7efee6ffd640 10 client.25562 handle_command_reply: tid=0 2024-05-10T10:43:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.712+0000 7eff10e07740 1 client.25562 shutdown 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 msgr2=0x55893bfadd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfadd10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efefc0116c0 tx=0x7efefc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 shutdown_connections 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bfb4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893bfbba60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7efee6ffd640 0 client.25562 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa3fb0 0x55893bfadd10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 --2- 172.21.15.112:0/3012441008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 >> 172.21.15.112:0/3012441008 conn(0x55893bf96420 msgr2=0x55893bf95580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 shutdown_connections 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 1 -- 172.21.15.112:0/3012441008 wait complete. 2024-05-10T10:43:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.713+0000 7eff10e07740 20 client.25562 trim_cache size 0 max 16384 2024-05-10T10:43:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 Processor -- start 2024-05-10T10:43:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 -- start start 2024-05-10T10:43:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa35c0 0x55893bfaccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa3fb0 0x55893bf99330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bf99870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff081a9170 con 0x55893bfa3fb0 2024-05-10T10:43:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff081aada0 con 0x55893bfa35c0 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff06ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa3fb0 0x55893bf99330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff06ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa3fb0 0x55893bf99330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34272/0 (socket says 172.21.15.112:34272) 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff06ffd640 1 -- 172.21.15.112:0/2382183942 learned_addr learned my addr 172.21.15.112:0/2382183942 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.731+0000 7eff10e07740 1 -- 172.21.15.112:0/2382183942 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff081a9840 con 0x55893bfa9120 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff077fe640 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa35c0 0x55893bfaccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff07fff640 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bf99870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff07fff640 1 -- 172.21.15.112:0/2382183942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa35c0 msgr2=0x55893bfaccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff07fff640 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa35c0 0x55893bfaccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff07fff640 1 -- 172.21.15.112:0/2382183942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa3fb0 msgr2=0x55893bf99330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff07fff640 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa3fb0 0x55893bf99330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff07fff640 1 -- 172.21.15.112:0/2382183942 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893bfb5e40 con 0x55893bfa9120 2024-05-10T10:43:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff077fe640 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa35c0 0x55893bfaccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff06ffd640 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa3fb0 0x55893bf99330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7eff07fff640 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bf99870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff000df750 tx=0x7eff000a9a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44b3528b9f4710b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7efeed7fa640 1 -- 172.21.15.112:0/2382183942 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000b20d0 con 0x55893bfa9120 2024-05-10T10:43:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7efeed7fa640 1 -- 172.21.15.112:0/2382183942 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff00014a80 con 0x55893bfa9120 2024-05-10T10:43:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.732+0000 7efeed7fa640 1 -- 172.21.15.112:0/2382183942 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000b1ed0 con 0x55893bfa9120 2024-05-10T10:43:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.733+0000 7eff10e07740 1 -- 172.21.15.112:0/2382183942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 msgr2=0x55893bf99870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.733+0000 7eff10e07740 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bf99870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff000df750 tx=0x7eff000a9a30 comp rx=0 tx=0).stop 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 -- 172.21.15.112:0/2382183942 shutdown_connections 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa9120 0x55893bf99870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa3fb0 0x55893bf99330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 --2- 172.21.15.112:0/2382183942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa35c0 0x55893bfaccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 -- 172.21.15.112:0/2382183942 >> 172.21.15.112:0/2382183942 conn(0x55893bf96420 msgr2=0x55893bfac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 -- 172.21.15.112:0/2382183942 shutdown_connections 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 -- 172.21.15.112:0/2382183942 wait complete. 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 -- start start 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.734+0000 7eff10e07740 1 Processor -- start 2024-05-10T10:43:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfb1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa3fb0 0x55893bfb19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa9120 0x55893c021ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff081aa610 con 0x55893bfa35c0 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff081a9f40 con 0x55893bfa9120 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff081a7150 con 0x55893bfa3fb0 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 10 client.0 _ll_register_callbacks cb 0x7efefc010e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa9120 0x55893c021ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa9120 0x55893c021ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:40834/0 (socket says 172.21.15.112:40834) 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfb1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 -- 172.21.15.112:0/1478895790 learned_addr learned my addr 172.21.15.112:0/1478895790 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff06ffd640 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa3fb0 0x55893bfb19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 -- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa3fb0 msgr2=0x55893bfb19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa3fb0 0x55893bfb19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 -- 172.21.15.112:0/1478895790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 msgr2=0x55893bfb1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfb1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 -- 172.21.15.112:0/1478895790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893c0224c0 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff07fff640 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa9120 0x55893c021ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff000a9200 tx=0x7eff000a7470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306e9ee79b4b1a65 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7efee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7efee6ffd640 1 -- 172.21.15.112:0/1478895790 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000c8060 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7efee6ffd640 1 -- 172.21.15.112:0/1478895790 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7eff00014650 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.735+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff00014a80 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7efee6ffd640 1 -- 172.21.15.112:0/1478895790 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff000ca300 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 10 client.25568 fetch_fsmap learned FSMap version 35 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55893c0237d0 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7efee6ffd640 1 -- 172.21.15.112:0/1478895790 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff00014a80 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7efee6ffd640 1 -- 172.21.15.112:0/1478895790 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7eff000a84d0 con 0x55893bfa9120 2024-05-10T10:43:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 10 client.25568 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 20 client.25568 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893c0261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 4 client.25568 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55893bf07050 con 0x55893bfb95a0 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.736+0000 7eff077fe640 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893c0261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.737+0000 7efee6ffd640 10 client.25568 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.737+0000 7eff077fe640 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893c0261d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cabcb21f2b021a server_cookie=eb582d7026c135ea in_seq=0 out_seq=0 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.753+0000 7efee6ffd640 1 -- 172.21.15.112:0/1478895790 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55893bf07050 con 0x55893bfb95a0 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.753+0000 7efee6ffd640 10 client.25568 handle_command_reply: tid=0 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.753+0000 7eff10e07740 1 client.25568 shutdown 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.753+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa9120 msgr2=0x55893c021ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.753+0000 7eff10e07740 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa9120 0x55893c021ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff000a9200 tx=0x7eff000a7470 comp rx=0 tx=0).stop 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 shutdown_connections 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55893bfb95a0 0x55893c0261d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55893bfa9120 0x55893c021ed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55893bfa3fb0 0x55893bfb19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 --2- 172.21.15.112:0/1478895790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893bfa35c0 0x55893bfb1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 >> 172.21.15.112:0/1478895790 conn(0x55893bf96420 msgr2=0x55893bf99330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7efee6ffd640 0 client.25568 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 shutdown_connections 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 1 -- 172.21.15.112:0/1478895790 wait complete. 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff10e07740 20 client.25568 trim_cache size 0 max 16384 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:43:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:43:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efef00b2920 msgr2=0x7efef00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff0e015640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efef00b2920 0x7efef00b4de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55893bfa4810 tx=0x7efefc00a4f0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0810a8c0 msgr2=0x7eff081a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.754+0000 7eff0e015640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0810a8c0 0x7eff081a22b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff0000be20 tx=0x7eff0000bf10 comp rx=0 tx=0).stop 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 shutdown_connections 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efef00b2920 0x7efef00b4de0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0810a8c0 0x7eff081a22b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff08076990 0x7eff0819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 --2- 172.21.15.112:0/3115801836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff080753b0 0x7eff0819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 >> 172.21.15.112:0/3115801836 conn(0x7eff080fdfa0 msgr2=0x7eff08106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 shutdown_connections 2024-05-10T10:43:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:11.755+0000 7eff0e015640 1 -- 172.21.15.112:0/3115801836 wait complete. 2024-05-10T10:43:11.783 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mount-idmapped 2024-05-10T10:43:11.877 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-append-write-fallocate-race 2024-05-10T10:43:11.898 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-append-write-read-race 2024-05-10T10:43:12.192 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-cow-race 2024-05-10T10:43:12.220 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-cycle-write 2024-05-10T10:43:12.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:11 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:43:12.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:11 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:43:12.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:11 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:43:12.397 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-eof-race 2024-05-10T10:43:12.485 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-extend-stat 2024-05-10T10:43:12.538 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-fcntl-race 2024-05-10T10:43:12.701 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-hole-filling-race 2024-05-10T10:43:12.782 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-invalidate-failure 2024-05-10T10:43:12.842 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-invalidate-readahead 2024-05-10T10:43:12.976 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aiodio_sparse2 2024-05-10T10:43:13.067 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-subblock-eof-read 2024-05-10T10:43:13.162 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-write-verify 2024-05-10T10:43:13.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:12 smithi148 ceph-mon[33568]: pgmap v519: 129 pgs: 129 active+clean; 37 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 394 KiB/s wr, 163 op/s 2024-05-10T10:43:13.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:12 smithi195 ceph-mon[33773]: pgmap v519: 129 pgs: 129 active+clean; 37 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 394 KiB/s wr, 163 op/s 2024-05-10T10:43:13.323 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-05-10T10:43:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:12 smithi112 ceph-mon[30100]: pgmap v519: 129 pgs: 129 active+clean; 37 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 394 KiB/s wr, 163 op/s 2024-05-10T10:43:13.330 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-05-10T10:43:13.485 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-last-ref-held-by-io 2024-05-10T10:43:14.189 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd .. 2024-05-10T10:43:14.190 INFO:tasks.workunit.client.0.smithi112.stderr:+ cp xfstests-dev/ltp/fsx . 2024-05-10T10:43:14.197 INFO:tasks.workunit.client.0.smithi112.stderr:+ OPTIONS=-z 2024-05-10T10:43:14.197 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-05-10T10:43:14.198 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-05-10T10:43:14.231 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-10T10:43:14.231 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-10T10:43:14.231 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-05-10T10:43:14.232 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-10T10:43:14.235 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-10T10:43:14.236 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-10T10:43:14.236 INFO:tasks.workunit.client.0.smithi112.stdout:fallocating to largest ever: 0x9abd6 2024-05-10T10:43:14.282 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xaeaec 2024-05-10T10:43:14.318 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xf3194 2024-05-10T10:43:14.366 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xf3ceb 2024-05-10T10:43:14.558 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xfaccc 2024-05-10T10:43:14.564 INFO:tasks.workunit.client.0.smithi112.stdout:fallocating to largest ever: 0xfbb6d 2024-05-10T10:43:15.011 INFO:tasks.workunit.client.0.smithi112.stdout:fallocating to largest ever: 0xffee7 2024-05-10T10:43:15.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:14 smithi148 ceph-mon[33568]: pgmap v520: 129 pgs: 129 active+clean; 37 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 360 KiB/s wr, 155 op/s 2024-05-10T10:43:15.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:14 smithi195 ceph-mon[33773]: pgmap v520: 129 pgs: 129 active+clean; 37 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 360 KiB/s wr, 155 op/s 2024-05-10T10:43:15.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:14 smithi112 ceph-mon[30100]: pgmap v520: 129 pgs: 129 active+clean; 37 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 360 KiB/s wr, 155 op/s 2024-05-10T10:43:17.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:17 smithi148 ceph-mon[33568]: pgmap v521: 129 pgs: 129 active+clean; 38 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 464 KiB/s wr, 175 op/s 2024-05-10T10:43:17.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:17 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:43:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:17 smithi195 ceph-mon[33773]: pgmap v521: 129 pgs: 129 active+clean; 38 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 464 KiB/s wr, 175 op/s 2024-05-10T10:43:17.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:17 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:43:17.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:17 smithi112 ceph-mon[30100]: pgmap v521: 129 pgs: 129 active+clean; 38 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 464 KiB/s wr, 175 op/s 2024-05-10T10:43:17.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:17 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:43:17.958 INFO:tasks.workunit.client.0.smithi112.stdout:fallocating to largest ever: 0x100000 2024-05-10T10:43:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:19 smithi148 ceph-mon[33568]: pgmap v522: 129 pgs: 129 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 393 KiB/s wr, 140 op/s 2024-05-10T10:43:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:19 smithi195 ceph-mon[33773]: pgmap v522: 129 pgs: 129 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 393 KiB/s wr, 140 op/s 2024-05-10T10:43:19.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:19 smithi112 ceph-mon[30100]: pgmap v522: 129 pgs: 129 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 393 KiB/s wr, 140 op/s 2024-05-10T10:43:21.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:21 smithi148 ceph-mon[33568]: pgmap v523: 129 pgs: 129 active+clean; 41 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2024-05-10T10:43:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:21 smithi195 ceph-mon[33773]: pgmap v523: 129 pgs: 129 active+clean; 41 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2024-05-10T10:43:21.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:21 smithi112 ceph-mon[30100]: pgmap v523: 129 pgs: 129 active+clean; 41 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2024-05-10T10:43:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:23 smithi148 ceph-mon[33568]: pgmap v524: 129 pgs: 129 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-05-10T10:43:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:23 smithi195 ceph-mon[33773]: pgmap v524: 129 pgs: 129 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-05-10T10:43:23.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:23 smithi112 ceph-mon[30100]: pgmap v524: 129 pgs: 129 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 -- 172.21.15.112:0/1722091264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650108750 msgr2=0x7f165010ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 --2- 172.21.15.112:0/1722091264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650108750 0x7f165010ab40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1648009ca0 tx=0x7f164802a8c0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 -- 172.21.15.112:0/1722091264 shutdown_connections 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 --2- 172.21.15.112:0/1722091264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650108750 0x7f165010ab40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 --2- 172.21.15.112:0/1722091264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1650105e20 0x7f1650108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 --2- 172.21.15.112:0/1722091264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16501034f0 0x7f16501058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 -- 172.21.15.112:0/1722091264 >> 172.21.15.112:0/1722091264 conn(0x7f16500fce80 msgr2=0x7f16500ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 -- 172.21.15.112:0/1722091264 shutdown_connections 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 -- 172.21.15.112:0/1722091264 wait complete. 2024-05-10T10:43:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.856+0000 7f1655fc1640 1 Processor -- start 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f1655fc1640 1 -- start start 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f1655fc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16501034f0 0x7f1650198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f1655fc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650105e20 0x7f1650199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f1655fc1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1650108750 0x7f16501a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f1655fc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f165010e460 con 0x7f1650105e20 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f1655fc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f165010e2e0 con 0x7f16501034f0 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f1655fc1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f165010e5e0 con 0x7f1650108750 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650105e20 0x7f1650199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650105e20 0x7f1650199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39534/0 (socket says 172.21.15.112:39534) 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 -- 172.21.15.112:0/2354301707 learned_addr learned my addr 172.21.15.112:0/2354301707 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164ffff640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1650108750 0x7f16501a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164f7fe640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16501034f0 0x7f1650198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 -- 172.21.15.112:0/2354301707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1650108750 msgr2=0x7f16501a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1650108750 0x7f16501a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 -- 172.21.15.112:0/2354301707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16501034f0 msgr2=0x7f1650198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16501034f0 0x7f1650198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 -- 172.21.15.112:0/2354301707 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16501a06f0 con 0x7f1650105e20 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164effd640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650105e20 0x7f1650199010 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f164400e9f0 tx=0x7f164400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.857+0000 7f164cff9640 1 -- 172.21.15.112:0/2354301707 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164400ccc0 con 0x7f1650105e20 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.858+0000 7f164cff9640 1 -- 172.21.15.112:0/2354301707 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16440054b0 con 0x7f1650105e20 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.858+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16501a09e0 con 0x7f1650105e20 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.858+0000 7f164cff9640 1 -- 172.21.15.112:0/2354301707 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644010430 con 0x7f1650105e20 2024-05-10T10:43:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.858+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16501a0ee0 con 0x7f1650105e20 2024-05-10T10:43:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f164cff9640 1 -- 172.21.15.112:0/2354301707 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f16440046c0 con 0x7f1650105e20 2024-05-10T10:43:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 Processor -- start 2024-05-10T10:43:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 -- start start 2024-05-10T10:43:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352ae910 0x5558352b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55583520d5d0 con 0x5558352a8db0 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5558351c6890 con 0x5558352a97a0 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f1658db3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555835209590 con 0x5558352ae910 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f164ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f164f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f164ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39550/0 (socket says 172.21.15.112:39550) 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.860+0000 7f164ffff640 1 -- 172.21.15.112:0/1003824045 learned_addr learned my addr 172.21.15.112:0/1003824045 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164ffff640 1 -- 172.21.15.112:0/1003824045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352ae910 msgr2=0x5558352b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164ffff640 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352ae910 0x5558352b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164ffff640 1 -- 172.21.15.112:0/1003824045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 msgr2=0x5558352ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164ffff640 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164ffff640 1 -- 172.21.15.112:0/1003824045 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555834f1f210 con 0x5558352a8db0 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164f7fe640 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164cff9640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f16380b28a0 0x7f16380b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164ffff640 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352a91d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f164802ade0 tx=0x7f1648005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e384d26ebd8654 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164f7fe640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f16380b28a0 0x7f16380b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f164cff9640 1 -- 172.21.15.112:0/2354301707 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f16440e28d0 con 0x7f1650105e20 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f1635ffb640 1 -- 172.21.15.112:0/1003824045 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480316d0 con 0x5558352a8db0 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f1635ffb640 1 -- 172.21.15.112:0/1003824045 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1648036030 con 0x5558352a8db0 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.861+0000 7f1635ffb640 1 -- 172.21.15.112:0/1003824045 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648004460 con 0x5558352a8db0 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 -- 172.21.15.112:0/1003824045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 msgr2=0x5558352a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352a91d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f164802ade0 tx=0x7f1648005ef0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f164f7fe640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f16380b28a0 0x7f16380b4d60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5558352a9ba0 tx=0x7f1640005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 -- 172.21.15.112:0/1003824045 shutdown_connections 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352ae910 0x5558352b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 --2- 172.21.15.112:0/1003824045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352a91d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 -- 172.21.15.112:0/1003824045 >> 172.21.15.112:0/1003824045 conn(0x5558352a4420 msgr2=0x5558352a4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 -- 172.21.15.112:0/1003824045 shutdown_connections 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.862+0000 7f1658db3740 1 -- 172.21.15.112:0/1003824045 wait complete. 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 -- start start 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 Processor -- start 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352ba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558352b16e0 con 0x5558352ae910 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5558351ca370 con 0x5558352a97a0 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5558352b1860 con 0x5558352a8db0 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f1658db3740 10 client.0 _ll_register_callbacks cb 0x555834f1ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f164f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f164effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f164f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:59612/0 (socket says 172.21.15.112:59612) 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.863+0000 7f164f7fe640 1 -- 172.21.15.112:0/3468758828 learned_addr learned my addr 172.21.15.112:0/3468758828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164ffff640 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352ba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164effd640 1 -- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a8db0 msgr2=0x5558352b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164effd640 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164effd640 1 -- 172.21.15.112:0/3468758828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 msgr2=0x5558352ba3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164effd640 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352ba3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164effd640 1 -- 172.21.15.112:0/3468758828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555834feb620 con 0x5558352a97a0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164f7fe640 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f164effd640 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352b33b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16440e0090 tx=0x7f16440a92d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16260f91e0f88e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f162effd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f162effd640 1 -- 172.21.15.112:0/3468758828 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16440aa260 con 0x5558352a97a0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1648036030 con 0x5558352a97a0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f162effd640 1 -- 172.21.15.112:0/3468758828 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f16440b2740 con 0x5558352a97a0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f162effd640 1 -- 172.21.15.112:0/3468758828 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f16440a9b60 con 0x5558352a97a0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f1658db3740 10 client.25580 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.864+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5558352bd490 con 0x5558352a97a0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f162effd640 1 -- 172.21.15.112:0/3468758828 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1648036030 con 0x5558352a97a0 2024-05-10T10:43:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f162effd640 1 -- 172.21.15.112:0/3468758828 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f16440a8830 con 0x5558352a97a0 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f1658db3740 10 client.25580 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f1658db3740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f1658db3740 20 client.25580 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f1658db3740 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558352beb20 0x5558352c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f1658db3740 4 client.25580 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55583521b1e0 con 0x5558352beb20 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.865+0000 7f164f7fe640 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558352beb20 0x5558352c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f162effd640 10 client.25580 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f164f7fe640 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558352beb20 0x5558352c0f40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed9e84b6cdc299f server_cookie=f225d5735b2d9868 in_seq=0 out_seq=0 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f162effd640 1 -- 172.21.15.112:0/3468758828 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55583521b1e0 con 0x5558352beb20 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f162effd640 10 client.25580 handle_command_reply: tid=0 2024-05-10T10:43:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f1658db3740 1 client.25580 shutdown 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 msgr2=0x5558352b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f1658db3740 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352b33b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16440e0090 tx=0x7f16440a92d0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.866+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 shutdown_connections 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352ba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558352beb20 0x5558352c0f40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a97a0 0x5558352b33b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f162effd640 0 client.25580 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 1 --2- 172.21.15.112:0/3468758828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 >> 172.21.15.112:0/3468758828 conn(0x5558352a4420 msgr2=0x5558352a3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 shutdown_connections 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 1 -- 172.21.15.112:0/3468758828 wait complete. 2024-05-10T10:43:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.867+0000 7f1658db3740 20 client.25580 trim_cache size 0 max 16384 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 Processor -- start 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 -- start start 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352ae910 0x5558352aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16501a90e0 con 0x5558352a8db0 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16501aada0 con 0x5558352ae910 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f1658db3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f16501a97b0 con 0x5558352a97a0 2024-05-10T10:43:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39566/0 (socket says 172.21.15.112:39566) 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 -- 172.21.15.112:0/1772499843 learned_addr learned my addr 172.21.15.112:0/1772499843 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164effd640 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164ffff640 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352ae910 0x5558352aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 -- 172.21.15.112:0/1772499843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 msgr2=0x5558352b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 -- 172.21.15.112:0/1772499843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352ae910 msgr2=0x5558352aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352ae910 0x5558352aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 -- 172.21.15.112:0/1772499843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558352a89f0 con 0x5558352a8db0 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.884+0000 7f164f7fe640 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352b2e70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5558352c17b0 tx=0x7f164001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f297851ba8ed576c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f16357fa640 1 -- 172.21.15.112:0/1772499843 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164002b4e0 con 0x5558352a8db0 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f16357fa640 1 -- 172.21.15.112:0/1772499843 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1640005020 con 0x5558352a8db0 2024-05-10T10:43:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f16357fa640 1 -- 172.21.15.112:0/1772499843 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640004800 con 0x5558352a8db0 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 -- 172.21.15.112:0/1772499843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 msgr2=0x5558352b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352b2e70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5558352c17b0 tx=0x7f164001b000 comp rx=0 tx=0).stop 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 -- 172.21.15.112:0/1772499843 shutdown_connections 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352ae910 0x5558352aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 --2- 172.21.15.112:0/1772499843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352a8db0 0x5558352b2e70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 -- 172.21.15.112:0/1772499843 >> 172.21.15.112:0/1772499843 conn(0x5558352a4420 msgr2=0x55583529f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 -- 172.21.15.112:0/1772499843 shutdown_connections 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.885+0000 7f1658db3740 1 -- 172.21.15.112:0/1772499843 wait complete. 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 -- start start 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 Processor -- start 2024-05-10T10:43:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a8db0 0x5558352b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352b8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16501aa610 con 0x5558352ae910 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16501a9f40 con 0x5558352a8db0 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f16501a70c0 con 0x5558352a97a0 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f1658db3740 10 client.0 _ll_register_callbacks cb 0x7f164400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f164ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f164effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f164f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a8db0 0x5558352b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f164effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:59638/0 (socket says 172.21.15.112:59638) 2024-05-10T10:43:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.886+0000 7f164effd640 1 -- 172.21.15.112:0/3589354764 learned_addr learned my addr 172.21.15.112:0/3589354764 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f164effd640 1 -- 172.21.15.112:0/3589354764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a8db0 msgr2=0x5558352b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f164effd640 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a8db0 0x5558352b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f164effd640 1 -- 172.21.15.112:0/3589354764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 msgr2=0x5558352b8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f164effd640 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352b8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f164effd640 1 -- 172.21.15.112:0/3589354764 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558352b8920 con 0x5558352a97a0 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f164ffff640 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f164effd640 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b7370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16440a6dd0 tx=0x7f1644009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ed1fbc77c8fb28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f162effd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f162effd640 1 -- 172.21.15.112:0/3589354764 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164408b030 con 0x5558352a97a0 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1640005020 con 0x5558352a97a0 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f162effd640 1 -- 172.21.15.112:0/3589354764 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f16440a6150 con 0x5558352a97a0 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f162effd640 1 -- 172.21.15.112:0/3589354764 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f16440a5a20 con 0x5558352a97a0 2024-05-10T10:43:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f1658db3740 10 client.25612 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5558352b9ca0 con 0x5558352a97a0 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f162effd640 1 -- 172.21.15.112:0/3589354764 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1640005020 con 0x5558352a97a0 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f162effd640 1 -- 172.21.15.112:0/3589354764 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f1644004f20 con 0x5558352a97a0 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f1658db3740 10 client.25612 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.887+0000 7f1658db3740 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.888+0000 7f1658db3740 20 client.25612 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.888+0000 7f1658db3740 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558353298c0 0x55583532bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.888+0000 7f1658db3740 4 client.25612 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.888+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555835215050 con 0x5558353298c0 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.888+0000 7f164f7fe640 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558353298c0 0x55583532bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.888+0000 7f164f7fe640 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558353298c0 0x55583532bce0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da23ea556f58edc server_cookie=ddf618ba97ce4b39 in_seq=0 out_seq=0 2024-05-10T10:43:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.888+0000 7f162effd640 10 client.25612 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.910+0000 7f162effd640 1 -- 172.21.15.112:0/3589354764 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555835215050 con 0x5558353298c0 2024-05-10T10:43:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.910+0000 7f162effd640 10 client.25612 handle_command_reply: tid=0 2024-05-10T10:43:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.910+0000 7f1658db3740 1 client.25612 shutdown 2024-05-10T10:43:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 msgr2=0x5558352b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b7370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16440a6dd0 tx=0x7f1644009310 comp rx=0 tx=0).stop 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 shutdown_connections 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558352ae910 0x5558352b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5558352a97a0 0x5558352b7370 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5558353298c0 0x55583532bce0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 --2- 172.21.15.112:0/3589354764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558352a8db0 0x5558352b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f162effd640 0 client.25612 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 >> 172.21.15.112:0/3589354764 conn(0x5558352a4420 msgr2=0x5558352aaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 shutdown_connections 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.911+0000 7f1658db3740 1 -- 172.21.15.112:0/3589354764 wait complete. 2024-05-10T10:43:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.912+0000 7f1658db3740 20 client.25612 trim_cache size 0 max 16384 2024-05-10T10:43:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:43:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:43:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:43:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.912+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f16380b28a0 msgr2=0x7f16380b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.912+0000 7f1655fc1640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f16380b28a0 0x7f16380b4d60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5558352a9ba0 tx=0x7f1640005f90 comp rx=0 tx=0).stop 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.912+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650105e20 msgr2=0x7f1650199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.912+0000 7f1655fc1640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650105e20 0x7f1650199010 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f164400e9f0 tx=0x7f164400eeb0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 shutdown_connections 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f16380b28a0 0x7f16380b4d60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1650108750 0x7f16501a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1650105e20 0x7f1650199010 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 --2- 172.21.15.112:0/2354301707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16501034f0 0x7f1650198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 >> 172.21.15.112:0/2354301707 conn(0x7f16500fce80 msgr2=0x7f1650103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 shutdown_connections 2024-05-10T10:43:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:24.913+0000 7f1655fc1640 1 -- 172.21.15.112:0/2354301707 wait complete. 2024-05-10T10:43:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:25 smithi148 ceph-mon[33568]: pgmap v525: 129 pgs: 129 active+clean; 42 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-05-10T10:43:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:25 smithi195 ceph-mon[33773]: pgmap v525: 129 pgs: 129 active+clean; 42 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-05-10T10:43:25.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:25 smithi112 ceph-mon[30100]: pgmap v525: 129 pgs: 129 active+clean; 42 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 483 op/s 2024-05-10T10:43:25.900 INFO:tasks.workunit.client.0.smithi112.stdout:10000 punch from 0x713e to 0x9907, (0x27c9 bytes) 2024-05-10T10:43:27.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:27 smithi148 ceph-mon[33568]: pgmap v526: 129 pgs: 129 active+clean; 43 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 477 op/s 2024-05-10T10:43:27.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:27 smithi195 ceph-mon[33773]: pgmap v526: 129 pgs: 129 active+clean; 43 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 477 op/s 2024-05-10T10:43:27.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:27 smithi112 ceph-mon[30100]: pgmap v526: 129 pgs: 129 active+clean; 43 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 477 op/s 2024-05-10T10:43:27.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:43:27.790 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:29.277 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:29 smithi112 ceph-mon[30100]: pgmap v527: 129 pgs: 129 active+clean; 43 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 445 op/s 2024-05-10T10:43:29.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:29 smithi148 ceph-mon[33568]: pgmap v527: 129 pgs: 129 active+clean; 43 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 445 op/s 2024-05-10T10:43:29.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:29 smithi195 ceph-mon[33773]: pgmap v527: 129 pgs: 129 active+clean; 43 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 445 op/s 2024-05-10T10:43:29.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 -- 172.21.15.112:0/2825856448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 msgr2=0x7f803c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 --2- 172.21.15.112:0/2825856448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c102820 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f802c009990 tx=0x7f802c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 -- 172.21.15.112:0/2825856448 shutdown_connections 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 --2- 172.21.15.112:0/2825856448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 0x7f803c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 --2- 172.21.15.112:0/2825856448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c108410 0x7f803c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 --2- 172.21.15.112:0/2825856448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c102820 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 -- 172.21.15.112:0/2825856448 >> 172.21.15.112:0/2825856448 conn(0x7f803c079350 msgr2=0x7f803c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 -- 172.21.15.112:0/2825856448 shutdown_connections 2024-05-10T10:43:29.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.694+0000 7f8043bb3700 1 -- 172.21.15.112:0/2825856448 wait complete. 2024-05-10T10:43:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 Processor -- start 2024-05-10T10:43:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 -- start start 2024-05-10T10:43:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 0x7f803c1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c108410 0x7f803c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f803c112400 con 0x7f803c1023a0 2024-05-10T10:43:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f803c112280 con 0x7f803c102d60 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8043bb3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f803c112580 con 0x7f803c108410 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f804194f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f804114e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 0x7f803c1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f804194f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c1a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60446/0 (socket says 172.21.15.112:60446) 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f804194f700 1 -- 172.21.15.112:0/1290930156 learned_addr learned my addr 172.21.15.112:0/1290930156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f804114e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 0x7f803c1a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:47926/0 (socket says 172.21.15.112:47926) 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.695+0000 7f8042150700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c108410 0x7f803c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f804194f700 1 -- 172.21.15.112:0/1290930156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c108410 msgr2=0x7f803c19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f804194f700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c108410 0x7f803c19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f804194f700 1 -- 172.21.15.112:0/1290930156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 msgr2=0x7f803c1a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f804194f700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 0x7f803c1a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f804194f700 1 -- 172.21.15.112:0/1290930156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c19c260 con 0x7f803c1023a0 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f804114e700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 0x7f803c1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:29.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f8042150700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c108410 0x7f803c19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:29.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f804194f700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c1a18f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f803800e9c0 tx=0x7f803800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f8032ffd700 1 -- 172.21.15.112:0/1290930156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800cbd0 con 0x7f803c1023a0 2024-05-10T10:43:29.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c19c550 con 0x7f803c1023a0 2024-05-10T10:43:29.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f8032ffd700 1 -- 172.21.15.112:0/1290930156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f803800ee60 con 0x7f803c1023a0 2024-05-10T10:43:29.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f8032ffd700 1 -- 172.21.15.112:0/1290930156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380054b0 con 0x7f803c1023a0 2024-05-10T10:43:29.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.696+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1a87c0 con 0x7f803c1023a0 2024-05-10T10:43:29.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.699+0000 7f8032ffd700 1 -- 172.21.15.112:0/1290930156 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8038004750 con 0x7f803c1023a0 2024-05-10T10:43:29.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.700+0000 7f8032ffd700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f80280b2880 0x7f80280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.700+0000 7f804114e700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f80280b2880 0x7f80280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.700+0000 7f8032ffd700 1 -- 172.21.15.112:0/1290930156 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f80380e2810 con 0x7f803c1023a0 2024-05-10T10:43:29.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 Processor -- start 2024-05-10T10:43:29.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 -- start start 2024-05-10T10:43:29.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804114e700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f80280b2880 0x7f80280b4d40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f803c100aa0 tx=0x7f802c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a17100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562026964860 con 0x562026a16c80 2024-05-10T10:43:29.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5620269037c0 con 0x562026a124d0 2024-05-10T10:43:29.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.701+0000 7f804e446b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562026960820 con 0x562026a12df0 2024-05-10T10:43:29.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f8042150700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a17100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:47928/0 (socket says 172.21.15.112:47928) 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 -- 172.21.15.112:0/2679708568 learned_addr learned my addr 172.21.15.112:0/2679708568 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804114e700 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 -- 172.21.15.112:0/2679708568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 msgr2=0x562026a166f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a166f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 -- 172.21.15.112:0/2679708568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 msgr2=0x562026a17100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a17100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 -- 172.21.15.112:0/2679708568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620268b18d0 con 0x562026a124d0 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f8042150700 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a17100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804114e700 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f804194f700 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a128b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f80380a97a0 tx=0x7f80380a92e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe95b9fe35b34e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f8027fff700 1 -- 172.21.15.112:0/2679708568 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380aa1a0 con 0x562026a124d0 2024-05-10T10:43:29.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f8027fff700 1 -- 172.21.15.112:0/2679708568 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80380a9aa0 con 0x562026a124d0 2024-05-10T10:43:29.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.702+0000 7f8027fff700 1 -- 172.21.15.112:0/2679708568 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380a9d00 con 0x562026a124d0 2024-05-10T10:43:29.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 -- 172.21.15.112:0/2679708568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 msgr2=0x562026a128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a128b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f80380a97a0 tx=0x7f80380a92e0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 -- 172.21.15.112:0/2679708568 shutdown_connections 2024-05-10T10:43:29.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a17100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a128b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 --2- 172.21.15.112:0/2679708568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 -- 172.21.15.112:0/2679708568 >> 172.21.15.112:0/2679708568 conn(0x56202693e100 msgr2=0x562026a0cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:29.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 -- 172.21.15.112:0/2679708568 shutdown_connections 2024-05-10T10:43:29.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.703+0000 7f804e446b80 1 -- 172.21.15.112:0/2679708568 wait complete. 2024-05-10T10:43:29.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.704+0000 7f804e446b80 1 -- start start 2024-05-10T10:43:29.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.704+0000 7f804e446b80 1 Processor -- start 2024-05-10T10:43:29.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804e446b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a124d0 0x562026a1e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804e446b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a12df0 0x562026a1b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804e446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a1b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804e446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562026a1ee30 con 0x562026a16c80 2024-05-10T10:43:29.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804e446b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5620268cac30 con 0x562026a12df0 2024-05-10T10:43:29.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804e446b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562026a1efb0 con 0x562026a124d0 2024-05-10T10:43:29.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804e446b80 10 client.0 _ll_register_callbacks cb 0x5620268fc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:29.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f8042150700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a1b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a12df0 0x562026a1b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804194f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a124d0 0x562026a1e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a12df0 0x562026a1b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:47936/0 (socket says 172.21.15.112:47936) 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 -- 172.21.15.112:0/3914317137 learned_addr learned my addr 172.21.15.112:0/3914317137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f8042150700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a1b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60468/0 (socket says 172.21.15.112:60468) 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 -- 172.21.15.112:0/3914317137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a124d0 msgr2=0x562026a1e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a124d0 0x562026a1e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 -- 172.21.15.112:0/3914317137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 msgr2=0x562026a1b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a1b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 -- 172.21.15.112:0/3914317137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562026900690 con 0x562026a12df0 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.705+0000 7f804114e700 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a12df0 0x562026a1b450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f802c015000 tx=0x7f802c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff1bbbe6ef5302c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f80257fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f80257fa700 1 -- 172.21.15.112:0/3914317137 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004d30 con 0x562026a12df0 2024-05-10T10:43:29.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f80257fa700 1 -- 172.21.15.112:0/3914317137 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f802c026430 con 0x562026a12df0 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80380a9aa0 con 0x562026a12df0 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f80257fa700 1 -- 172.21.15.112:0/3914317137 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f802c026630 con 0x562026a12df0 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 10 client.25598 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562026a29670 con 0x562026a12df0 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f80257fa700 1 -- 172.21.15.112:0/3914317137 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f802c026c60 con 0x562026a12df0 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f80257fa700 1 -- 172.21.15.112:0/3914317137 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f802c031930 con 0x562026a12df0 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 20 client.25598 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a2bcb0 0x562026a2e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 4 client.25598 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:29.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.706+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f80380a9aa0 con 0x562026a2bcb0 2024-05-10T10:43:29.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.707+0000 7f804194f700 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a2bcb0 0x562026a2e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.707+0000 7f80257fa700 10 client.25598 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:29.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.707+0000 7f804194f700 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a2bcb0 0x562026a2e0a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4448af30f03886 server_cookie=17330ee2d560d011 in_seq=0 out_seq=0 2024-05-10T10:43:29.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f80257fa700 1 -- 172.21.15.112:0/3914317137 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f80380a9aa0 con 0x562026a2bcb0 2024-05-10T10:43:29.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f80257fa700 10 client.25598 handle_command_reply: tid=0 2024-05-10T10:43:29.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 client.25598 shutdown 2024-05-10T10:43:29.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a12df0 msgr2=0x562026a1b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a12df0 0x562026a1b450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f802c015000 tx=0x7f802c009310 comp rx=0 tx=0).stop 2024-05-10T10:43:29.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 shutdown_connections 2024-05-10T10:43:29.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a2bcb0 0x562026a2e0a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a1b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a124d0 0x562026a1e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 --2- 172.21.15.112:0/3914317137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a12df0 0x562026a1b450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f80257fa700 0 client.25598 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:29.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 >> 172.21.15.112:0/3914317137 conn(0x56202693e100 msgr2=0x562026a07720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:29.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 shutdown_connections 2024-05-10T10:43:29.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 1 -- 172.21.15.112:0/3914317137 wait complete. 2024-05-10T10:43:29.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.708+0000 7f804e446b80 20 client.25598 trim_cache size 0 max 16384 2024-05-10T10:43:29.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 Processor -- start 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 -- start start 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a12df0 0x562026a33a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a16c80 0x562026a33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b13e0 con 0x562026a12df0 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b30a0 con 0x562026a124d0 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.732+0000 7f804e446b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b1ab0 con 0x562026a16c80 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:47946/0 (socket says 172.21.15.112:47946) 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 -- 172.21.15.112:0/2453167605 learned_addr learned my addr 172.21.15.112:0/2453167605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804114e700 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a12df0 0x562026a33a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f8042150700 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a16c80 0x562026a33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 -- 172.21.15.112:0/2453167605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a16c80 msgr2=0x562026a33f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a16c80 0x562026a33f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 -- 172.21.15.112:0/2453167605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a12df0 msgr2=0x562026a33a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a12df0 0x562026a33a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 -- 172.21.15.112:0/2453167605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562026a0e4b0 con 0x562026a124d0 2024-05-10T10:43:29.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804114e700 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a12df0 0x562026a33a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:29.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f8042150700 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a16c80 0x562026a33f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:29.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f804194f700 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a075a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f80380a6d10 tx=0x7f803800e940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe0cedde0d79748c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f80277fe700 1 -- 172.21.15.112:0/2453167605 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803808e020 con 0x562026a124d0 2024-05-10T10:43:29.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f80277fe700 1 -- 172.21.15.112:0/2453167605 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f803808ae50 con 0x562026a124d0 2024-05-10T10:43:29.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.733+0000 7f80277fe700 1 -- 172.21.15.112:0/2453167605 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380a9d00 con 0x562026a124d0 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 -- 172.21.15.112:0/2453167605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 msgr2=0x562026a075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a075a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f80380a6d10 tx=0x7f803800e940 comp rx=0 tx=0).stop 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 -- 172.21.15.112:0/2453167605 shutdown_connections 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a16c80 0x562026a33f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a075a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 --2- 172.21.15.112:0/2453167605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a12df0 0x562026a33a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 -- 172.21.15.112:0/2453167605 >> 172.21.15.112:0/2453167605 conn(0x56202693e100 msgr2=0x562026a06e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:29.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 -- 172.21.15.112:0/2453167605 shutdown_connections 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.734+0000 7f804e446b80 1 -- 172.21.15.112:0/2453167605 wait complete. 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 -- start start 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 Processor -- start 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a24100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a24640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b2910 con 0x562026a16c80 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b2240 con 0x562026a124d0 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1af380 con 0x562026a12df0 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804194f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804e446b80 10 client.0 _ll_register_callbacks cb 0x5620268fc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804194f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a24100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:47952/0 (socket says 172.21.15.112:47952) 2024-05-10T10:43:29.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804194f700 1 -- 172.21.15.112:0/843844818 learned_addr learned my addr 172.21.15.112:0/843844818 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f804114e700 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a24640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.735+0000 7f8042150700 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804194f700 1 -- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 msgr2=0x562026a24640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804194f700 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a24640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804194f700 1 -- 172.21.15.112:0/843844818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 msgr2=0x562026a226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804194f700 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804194f700 1 -- 172.21.15.112:0/843844818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562026a22d80 con 0x562026a124d0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804194f700 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a24100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f803808c040 tx=0x7f8038005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a670fac1efe320 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f80257fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f80257fa700 1 -- 172.21.15.112:0/843844818 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803806f020 con 0x562026a124d0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f80257fa700 1 -- 172.21.15.112:0/843844818 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f80380a6090 con 0x562026a124d0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803808ae50 con 0x562026a124d0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f80257fa700 1 -- 172.21.15.112:0/843844818 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f803808e050 con 0x562026a124d0 2024-05-10T10:43:29.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804e446b80 10 client.25604 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.736+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562026a97170 con 0x562026a124d0 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f80257fa700 1 -- 172.21.15.112:0/843844818 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f803808ae50 con 0x562026a124d0 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f80257fa700 1 -- 172.21.15.112:0/843844818 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f80380a9de0 con 0x562026a124d0 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f804e446b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f804e446b80 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f804e446b80 20 client.25604 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f804e446b80 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a974a0 0x562026a99890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f804e446b80 4 client.25604 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620269de650 con 0x562026a974a0 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.737+0000 7f804114e700 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a974a0 0x562026a99890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f80257fa700 10 client.25604 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804114e700 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a974a0 0x562026a99890 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504f55564f9b477b server_cookie=7e708e51a55ee3a5 in_seq=0 out_seq=0 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f80257fa700 1 -- 172.21.15.112:0/843844818 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5620269de650 con 0x562026a974a0 2024-05-10T10:43:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f80257fa700 10 client.25604 handle_command_reply: tid=0 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 client.25604 shutdown 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 msgr2=0x562026a24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a24100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f803808c040 tx=0x7f8038005b40 comp rx=0 tx=0).stop 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 shutdown_connections 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562026a974a0 0x562026a99890 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562026a16c80 0x562026a226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562026a124d0 0x562026a24100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 --2- 172.21.15.112:0/843844818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562026a12df0 0x562026a24640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 >> 172.21.15.112:0/843844818 conn(0x56202693e100 msgr2=0x562026a07720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f80257fa700 0 client.25604 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 shutdown_connections 2024-05-10T10:43:29.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.738+0000 7f804e446b80 1 -- 172.21.15.112:0/843844818 wait complete. 2024-05-10T10:43:29.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f804e446b80 20 client.25604 trim_cache size 0 max 16384 2024-05-10T10:43:29.746 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:43:29.746 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:43:29.746 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:43:29.747 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f80280b2880 msgr2=0x7f80280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f80280b2880 0x7f80280b4d40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f803c100aa0 tx=0x7f802c01a040 comp rx=0 tx=0).stop 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 msgr2=0x7f803c1a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c1a18f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f803800e9c0 tx=0x7f803800ecd0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 shutdown_connections 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f80280b2880 0x7f80280b4d40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c108410 0x7f803c19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f803c1023a0 0x7f803c1a18f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 --2- 172.21.15.112:0/1290930156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c102d60 0x7f803c1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 >> 172.21.15.112:0/1290930156 conn(0x7f803c079350 msgr2=0x7f803c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 shutdown_connections 2024-05-10T10:43:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:29.739+0000 7f8043bb3700 1 -- 172.21.15.112:0/1290930156 wait complete. 2024-05-10T10:43:30.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:43:30.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:43:30.956 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:31.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:31 smithi148 ceph-mon[33568]: pgmap v528: 129 pgs: 129 active+clean; 40 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-05-10T10:43:31.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:31 smithi195 ceph-mon[33773]: pgmap v528: 129 pgs: 129 active+clean; 40 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-05-10T10:43:31.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:31 smithi112 ceph-mon[30100]: pgmap v528: 129 pgs: 129 active+clean; 40 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-05-10T10:43:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 -- 172.21.15.112:0/74251639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 msgr2=0x7fb6f410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/74251639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f410f250 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00b0a0 tx=0x7fb6ec00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 -- 172.21.15.112:0/74251639 shutdown_connections 2024-05-10T10:43:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/74251639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f410f250 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/74251639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6f4108410 0x7fb6f41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/74251639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f41023a0 0x7fb6f4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 -- 172.21.15.112:0/74251639 >> 172.21.15.112:0/74251639 conn(0x7fb6f4079350 msgr2=0x7fb6f4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 -- 172.21.15.112:0/74251639 shutdown_connections 2024-05-10T10:43:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.922+0000 7fb6f96a1700 1 -- 172.21.15.112:0/74251639 wait complete. 2024-05-10T10:43:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 Processor -- start 2024-05-10T10:43:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 -- start start 2024-05-10T10:43:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6f41023a0 0x7fb6f4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f4108410 0x7fb6f4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f41123e0 con 0x7fb6f4102d60 2024-05-10T10:43:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f4112260 con 0x7fb6f4108410 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.923+0000 7fb6f96a1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f4112560 con 0x7fb6f41023a0 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f4108410 0x7fb6f4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6f41023a0 0x7fb6f4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f41045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60498/0 (socket says 172.21.15.112:60498) 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 -- 172.21.15.112:0/164223148 learned_addr learned my addr 172.21.15.112:0/164223148 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 -- 172.21.15.112:0/164223148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6f41023a0 msgr2=0x7fb6f4104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6f41023a0 0x7fb6f4104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 -- 172.21.15.112:0/164223148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f4108410 msgr2=0x7fb6f4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f4108410 0x7fb6f4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 -- 172.21.15.112:0/164223148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f41a7f70 con 0x7fb6f4102d60 2024-05-10T10:43:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.924+0000 7fb6f27fc700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f41045c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9a0 tx=0x7fb6e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.925+0000 7fb6dbfff700 1 -- 172.21.15.112:0/164223148 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800ca10 con 0x7fb6f4102d60 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.925+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f41a8200 con 0x7fb6f4102d60 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.925+0000 7fb6dbfff700 1 -- 172.21.15.112:0/164223148 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e800cbb0 con 0x7fb6f4102d60 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.925+0000 7fb6dbfff700 1 -- 172.21.15.112:0/164223148 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8018680 con 0x7fb6f4102d60 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.925+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f41a8790 con 0x7fb6f4102d60 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.926+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f406a190 con 0x7fb6f4102d60 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.928+0000 7fb6dbfff700 1 -- 172.21.15.112:0/164223148 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb6e8010430 con 0x7fb6f4102d60 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.929+0000 7fb6dbfff700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6dc0b2880 0x7fb6dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.929+0000 7fb6f2ffd700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6dc0b2880 0x7fb6dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.929+0000 7fb6dbfff700 1 -- 172.21.15.112:0/164223148 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb6e80e2b40 con 0x7fb6f4102d60 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.930+0000 7fb6f2ffd700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6dc0b2880 0x7fb6dc0b4d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009be0 tx=0x7fb6e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:32.934+0000 7fb6dbfff700 1 -- 172.21.15.112:0/164223148 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb6e80aa4d0 con 0x7fb6f4102d60 2024-05-10T10:43:33.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:33 smithi112 ceph-mon[30100]: pgmap v529: 129 pgs: 129 active+clean; 40 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 510 op/s 2024-05-10T10:43:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.257+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f41a89f0 con 0x7fb6f4102d60 2024-05-10T10:43:33.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.259+0000 7fb6dbfff700 1 -- 172.21.15.112:0/164223148 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11967 (secure 0 0 0) 0x7fb6e80a9dd0 con 0x7fb6f4102d60 2024-05-10T10:43:33.260 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:43:33.260 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:43:15.242856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24451,24469,24446,24445,24428]},"id":1}]} 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6dc0b2880 msgr2=0x7fb6dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6dc0b2880 0x7fb6dc0b4d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009be0 tx=0x7fb6e4009260 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 msgr2=0x7fb6f41045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f41045c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9a0 tx=0x7fb6e800ecb0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 shutdown_connections 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f4108410 0x7fb6f4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb6dc0b2880 0x7fb6dc0b4d40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6f41023a0 0x7fb6f4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 --2- 172.21.15.112:0/164223148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f4102d60 0x7fb6f41045c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 >> 172.21.15.112:0/164223148 conn(0x7fb6f4079350 msgr2=0x7fb6f41016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 shutdown_connections 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:33.261+0000 7fb6f96a1700 1 -- 172.21.15.112:0/164223148 wait complete. 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-05-10T10:43:33.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:33 smithi148 ceph-mon[33568]: pgmap v529: 129 pgs: 129 active+clean; 40 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 510 op/s 2024-05-10T10:43:33.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:33 smithi195 ceph-mon[33773]: pgmap v529: 129 pgs: 129 active+clean; 40 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 510 op/s 2024-05-10T10:43:34.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:43:34.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:34 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/164223148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:43:34.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:34 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/164223148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:43:34.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:34 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/164223148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:43:34.358 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:35.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:35 smithi148 ceph-mon[33568]: pgmap v530: 129 pgs: 129 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 727 op/s 2024-05-10T10:43:35.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:35 smithi195 ceph-mon[33773]: pgmap v530: 129 pgs: 129 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 727 op/s 2024-05-10T10:43:35.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:35 smithi112 ceph-mon[30100]: pgmap v530: 129 pgs: 129 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 727 op/s 2024-05-10T10:43:36.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 -- 172.21.15.112:0/877578706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 msgr2=0x7f8480109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 --2- 172.21.15.112:0/877578706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 0x7f8480109920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8470009990 tx=0x7f8470009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 -- 172.21.15.112:0/877578706 shutdown_connections 2024-05-10T10:43:36.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 --2- 172.21.15.112:0/877578706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006c530 0x7f8480110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 --2- 172.21.15.112:0/877578706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 0x7f8480109920 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 --2- 172.21.15.112:0/877578706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848006bb70 0x7f848006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 -- 172.21.15.112:0/877578706 >> 172.21.15.112:0/877578706 conn(0x7f84800793e0 msgr2=0x7f84800797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:36.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 -- 172.21.15.112:0/877578706 shutdown_connections 2024-05-10T10:43:36.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.290+0000 7f848499c700 1 -- 172.21.15.112:0/877578706 wait complete. 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.291+0000 7f848499c700 1 Processor -- start 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.291+0000 7f848499c700 1 -- start start 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f848499c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006bb70 0x7f848019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f848499c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848006c530 0x7f848019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f848499c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 0x7f84801998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f848499c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8480113520 con 0x7f8480109540 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f848499c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f84801133a0 con 0x7f848006bb70 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f848499c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84801136a0 con 0x7f848006c530 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006bb70 0x7f848019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006bb70 0x7f848019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34076/0 (socket says 172.21.15.112:34076) 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 -- 172.21.15.112:0/2883162411 learned_addr learned my addr 172.21.15.112:0/2883162411 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847ed9d700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 0x7f84801998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847dd9b700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848006c530 0x7f848019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 -- 172.21.15.112:0/2883162411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848006c530 msgr2=0x7f848019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848006c530 0x7f848019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 -- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 msgr2=0x7f84801998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 0x7f84801998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847e59c700 1 -- 172.21.15.112:0/2883162411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848019a040 con 0x7f848006bb70 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.292+0000 7f847ed9d700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 0x7f84801998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.293+0000 7f847e59c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006bb70 0x7f848019f720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8470006010 tx=0x7f84700047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.293+0000 7f846f7fe700 1 -- 172.21.15.112:0/2883162411 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470027070 con 0x7f848006bb70 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.293+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848019a2d0 con 0x7f848006bb70 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.293+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84801a65f0 con 0x7f848006bb70 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.293+0000 7f846f7fe700 1 -- 172.21.15.112:0/2883162411 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8470022470 con 0x7f848006bb70 2024-05-10T10:43:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.293+0000 7f846f7fe700 1 -- 172.21.15.112:0/2883162411 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470017650 con 0x7f848006bb70 2024-05-10T10:43:36.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.296+0000 7f846f7fe700 1 -- 172.21.15.112:0/2883162411 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8470009d10 con 0x7f848006bb70 2024-05-10T10:43:36.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.297+0000 7f846f7fe700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f84680b2900 0x7f84680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.297+0000 7f847dd9b700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f84680b2900 0x7f84680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.297+0000 7f846f7fe700 1 -- 172.21.15.112:0/2883162411 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f84700e2f20 con 0x7f848006bb70 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.297+0000 7f848f22fb80 1 Processor -- start 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.297+0000 7f848f22fb80 1 -- start start 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.297+0000 7f847dd9b700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f84680b2900 0x7f84680b4dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8480079ac0 tx=0x7f8474009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbaa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cbaf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f848f22fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56233cafc860 con 0x56233cbaec80 2024-05-10T10:43:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f848f22fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56233ca9b7c0 con 0x56233cbaa4d0 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f848f22fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56233caf8820 con 0x56233cbaadf0 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cbaf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbaa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cbaf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57172/0 (socket says 172.21.15.112:57172) 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbaa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34082/0 (socket says 172.21.15.112:34082) 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847ed9d700 1 -- 172.21.15.112:0/3595821292 learned_addr learned my addr 172.21.15.112:0/3595821292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847dd9b700 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 -- 172.21.15.112:0/3595821292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 msgr2=0x56233cbae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 -- 172.21.15.112:0/3595821292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 msgr2=0x56233cbaf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cbaf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 -- 172.21.15.112:0/3595821292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233ca498d0 con 0x56233cbaa4d0 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.298+0000 7f847e59c700 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbaa8b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84700e06e0 tx=0x7f84700a9a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c599378a89193b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f846cff9700 1 -- 172.21.15.112:0/3595821292 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700aa240 con 0x56233cbaa4d0 2024-05-10T10:43:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f846cff9700 1 -- 172.21.15.112:0/3595821292 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84700ae5f0 con 0x56233cbaa4d0 2024-05-10T10:43:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f846cff9700 1 -- 172.21.15.112:0/3595821292 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700ae380 con 0x56233cbaa4d0 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f848f22fb80 1 -- 172.21.15.112:0/3595821292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 msgr2=0x56233cbaa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f848f22fb80 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbaa8b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84700e06e0 tx=0x7f84700a9a30 comp rx=0 tx=0).stop 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f848f22fb80 1 -- 172.21.15.112:0/3595821292 shutdown_connections 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f848f22fb80 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cbaf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f848f22fb80 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbaa8b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f848f22fb80 1 --2- 172.21.15.112:0/3595821292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.299+0000 7f848f22fb80 1 -- 172.21.15.112:0/3595821292 >> 172.21.15.112:0/3595821292 conn(0x56233cad6100 msgr2=0x56233cba4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 -- 172.21.15.112:0/3595821292 shutdown_connections 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 -- 172.21.15.112:0/3595821292 wait complete. 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 -- start start 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 Processor -- start 2024-05-10T10:43:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaadf0 0x56233cbb3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbb3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.300+0000 7f848f22fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56233cbb6c50 con 0x56233cbaadf0 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f848f22fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56233ca62c30 con 0x56233cbaa4d0 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f848f22fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56233cbb6dd0 con 0x56233cbaec80 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f848f22fb80 10 client.0 _ll_register_callbacks cb 0x56233ca94920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaadf0 0x56233cbb3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaadf0 0x56233cbb3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57178/0 (socket says 172.21.15.112:57178) 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847dd9b700 1 -- 172.21.15.112:0/1879046841 learned_addr learned my addr 172.21.15.112:0/1879046841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847ed9d700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbb3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847e59c700 1 -- 172.21.15.112:0/1879046841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 msgr2=0x56233cbb3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847e59c700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbb3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847e59c700 1 -- 172.21.15.112:0/1879046841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaadf0 msgr2=0x56233cbb3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847e59c700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaadf0 0x56233cbb3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847e59c700 1 -- 172.21.15.112:0/1879046841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233ca98690 con 0x56233cbaa4d0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847dd9b700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaadf0 0x56233cbb3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847ed9d700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbb3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f847e59c700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb6060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f84700a74b0 tx=0x7f84700ae3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926ebf49ffe88b23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.301+0000 7f8465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f8465ffb700 1 -- 172.21.15.112:0/1879046841 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700aa240 con 0x56233cbaa4d0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f8465ffb700 1 -- 172.21.15.112:0/1879046841 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f84700e0d60 con 0x56233cbaa4d0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84700ae5f0 con 0x56233cbaa4d0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f8465ffb700 1 -- 172.21.15.112:0/1879046841 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f84700a94a0 con 0x56233cbaa4d0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 10 client.25622 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56233cbb6c50 con 0x56233cbaa4d0 2024-05-10T10:43:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f8465ffb700 1 -- 172.21.15.112:0/1879046841 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84700ae5f0 con 0x56233cbaa4d0 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f8465ffb700 1 -- 172.21.15.112:0/1879046841 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f84700903f0 con 0x56233cbaa4d0 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 10 client.25622 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 10 client.25622 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 20 client.25622 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cbc3710 0x56233cbc5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 4 client.25622 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.302+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233cb76650 con 0x56233cbc3710 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.303+0000 7f847dd9b700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cbc3710 0x56233cbc5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.303+0000 7f8465ffb700 10 client.25622 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.303+0000 7f847dd9b700 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cbc3710 0x56233cbc5b00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=197d5dba8533e335 server_cookie=ebed5346a43aea04 in_seq=0 out_seq=0 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f8465ffb700 1 -- 172.21.15.112:0/1879046841 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56233cb76650 con 0x56233cbc3710 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f8465ffb700 10 client.25622 handle_command_reply: tid=0 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 client.25622 shutdown 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 msgr2=0x56233cbb6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb6060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f84700a74b0 tx=0x7f84700ae3d0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 shutdown_connections 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cbc3710 0x56233cbc5b00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbb3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f8465ffb700 0 client.25622 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb6060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 --2- 172.21.15.112:0/1879046841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaadf0 0x56233cbb3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 >> 172.21.15.112:0/1879046841 conn(0x56233cad6100 msgr2=0x56233cb9f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 shutdown_connections 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 1 -- 172.21.15.112:0/1879046841 wait complete. 2024-05-10T10:43:36.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.304+0000 7f848f22fb80 20 client.25622 trim_cache size 0 max 16384 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 Processor -- start 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 -- start start 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaa4d0 0x56233cbbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaadf0 0x56233cb9f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbcb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f846000ead0 con 0x56233cbaa4d0 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8460010790 con 0x56233cbaadf0 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f848f22fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f846000f1a0 con 0x56233cbaec80 2024-05-10T10:43:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaa4d0 0x56233cbbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaa4d0 0x56233cbbfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57182/0 (socket says 172.21.15.112:57182) 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 -- 172.21.15.112:0/862656708 learned_addr learned my addr 172.21.15.112:0/862656708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847ed9d700 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbcb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847dd9b700 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaadf0 0x56233cb9f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 -- 172.21.15.112:0/862656708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 msgr2=0x56233cbcb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbcb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 -- 172.21.15.112:0/862656708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaadf0 msgr2=0x56233cb9f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaadf0 0x56233cb9f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 -- 172.21.15.112:0/862656708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233cba5810 con 0x56233cbaa4d0 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.322+0000 7f847e59c700 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaa4d0 0x56233cbbfca0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8470005cb0 tx=0x7f847000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e84def2b0d2b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f8467fff700 1 -- 172.21.15.112:0/862656708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700aa240 con 0x56233cbaa4d0 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f8467fff700 1 -- 172.21.15.112:0/862656708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84700ae5f0 con 0x56233cbaa4d0 2024-05-10T10:43:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f8467fff700 1 -- 172.21.15.112:0/862656708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700a5ee0 con 0x56233cbaa4d0 2024-05-10T10:43:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f848f22fb80 1 -- 172.21.15.112:0/862656708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaa4d0 msgr2=0x56233cbbfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f848f22fb80 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaa4d0 0x56233cbbfca0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8470005cb0 tx=0x7f847000bfd0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f848f22fb80 1 -- 172.21.15.112:0/862656708 shutdown_connections 2024-05-10T10:43:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f848f22fb80 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaec80 0x56233cbcb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f848f22fb80 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaa4d0 0x56233cbbfca0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f848f22fb80 1 --2- 172.21.15.112:0/862656708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaadf0 0x56233cb9f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.323+0000 7f848f22fb80 1 -- 172.21.15.112:0/862656708 >> 172.21.15.112:0/862656708 conn(0x56233cad6100 msgr2=0x56233cba9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 -- 172.21.15.112:0/862656708 shutdown_connections 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 -- 172.21.15.112:0/862656708 wait complete. 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 -- start start 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 Processor -- start 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbb98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cc2bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8460010000 con 0x56233cbaec80 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f846000f930 con 0x56233cbaa4d0 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f846000ca70 con 0x56233cbaadf0 2024-05-10T10:43:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f848f22fb80 10 client.0 _ll_register_callbacks cb 0x56233ca94920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f847ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cc2bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f847e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f847dd9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbb98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f847e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34118/0 (socket says 172.21.15.112:34118) 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.324+0000 7f847e59c700 1 -- 172.21.15.112:0/4228776503 learned_addr learned my addr 172.21.15.112:0/4228776503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f847ed9d700 1 -- 172.21.15.112:0/4228776503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 msgr2=0x56233cbb98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f847ed9d700 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbb98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f847ed9d700 1 -- 172.21.15.112:0/4228776503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 msgr2=0x56233cbb93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f847ed9d700 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f847ed9d700 1 -- 172.21.15.112:0/4228776503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233cc2c4f0 con 0x56233cbaec80 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f847ed9d700 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cc2bee0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x56233cba6710 tx=0x7f8478018470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d110ae7ffa8222 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f8465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f8465ffb700 1 -- 172.21.15.112:0/4228776503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478004bb0 con 0x56233cbaec80 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f8465ffb700 1 -- 172.21.15.112:0/4228776503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8478004d50 con 0x56233cbaec80 2024-05-10T10:43:36.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84700ae5f0 con 0x56233cbaec80 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f8465ffb700 1 -- 172.21.15.112:0/4228776503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8478029050 con 0x56233cbaec80 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f848f22fb80 10 client.16239 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.325+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56233cc2d850 con 0x56233cbaec80 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f8465ffb700 1 -- 172.21.15.112:0/4228776503 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84700ae5f0 con 0x56233cbaec80 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f8465ffb700 1 -- 172.21.15.112:0/4228776503 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f84780163c0 con 0x56233cbaec80 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f848f22fb80 10 client.16239 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f848f22fb80 10 client.16239 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f848f22fb80 20 client.16239 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f848f22fb80 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cc2df90 0x56233cc303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f848f22fb80 4 client.16239 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56233cb0a240 con 0x56233cc2df90 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.326+0000 7f847e59c700 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cc2df90 0x56233cc303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f8465ffb700 10 client.16239 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:43:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f847e59c700 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cc2df90 0x56233cc303b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f32e2ddb6cd45f87 server_cookie=adbb35c504a41957 in_seq=0 out_seq=0 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f8465ffb700 1 -- 172.21.15.112:0/4228776503 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56233cb0a240 con 0x56233cc2df90 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f8465ffb700 10 client.16239 handle_command_reply: tid=0 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 client.16239 shutdown 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 msgr2=0x56233cc2bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cc2bee0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x56233cba6710 tx=0x7f8478018470 comp rx=0 tx=0).stop 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 shutdown_connections 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56233cbaec80 0x56233cc2bee0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233cbaa4d0 0x56233cbb93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x56233cc2df90 0x56233cc303b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 --2- 172.21.15.112:0/4228776503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56233cbaadf0 0x56233cbb98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f8465ffb700 0 client.16239 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 >> 172.21.15.112:0/4228776503 conn(0x56233cad6100 msgr2=0x56233cbad520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 shutdown_connections 2024-05-10T10:43:36.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 1 -- 172.21.15.112:0/4228776503 wait complete. 2024-05-10T10:43:36.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.327+0000 7f848f22fb80 20 client.16239 trim_cache size 0 max 16384 2024-05-10T10:43:36.334 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f84680b2900 msgr2=0x7f84680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f84680b2900 0x7f84680b4dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8480079ac0 tx=0x7f8474009260 comp rx=0 tx=0).stop 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006bb70 msgr2=0x7f848019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006bb70 0x7f848019f720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8470006010 tx=0x7f84700047b0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 shutdown_connections 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8480109540 0x7f84801998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f848006bb70 0x7f848019f720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f84680b2900 0x7f84680b4dc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 --2- 172.21.15.112:0/2883162411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f848006c530 0x7f848019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 >> 172.21.15.112:0/2883162411 conn(0x7f84800793e0 msgr2=0x7f8480101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 shutdown_connections 2024-05-10T10:43:36.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:36.328+0000 7f848499c700 1 -- 172.21.15.112:0/2883162411 wait complete. 2024-05-10T10:43:37.191 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:43:37.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:37 smithi148 ceph-mon[33568]: pgmap v531: 129 pgs: 129 active+clean; 41 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 479 op/s 2024-05-10T10:43:37.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:37 smithi195 ceph-mon[33773]: pgmap v531: 129 pgs: 129 active+clean; 41 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 479 op/s 2024-05-10T10:43:37.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:37 smithi112 ceph-mon[30100]: pgmap v531: 129 pgs: 129 active+clean; 41 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 479 op/s 2024-05-10T10:43:37.470 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 -- 172.21.15.112:0/3524034810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc103c20 msgr2=0x7f28bc1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 --2- 172.21.15.112:0/3524034810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc103c20 0x7f28bc1040a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009900 tx=0x7f28a402a410 comp rx=0 tx=0).stop 2024-05-10T10:43:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 -- 172.21.15.112:0/3524034810 shutdown_connections 2024-05-10T10:43:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 --2- 172.21.15.112:0/3524034810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc1045e0 0x7f28bc10ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 --2- 172.21.15.112:0/3524034810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc103c20 0x7f28bc1040a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 --2- 172.21.15.112:0/3524034810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28bc102a30 0x7f28bc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 -- 172.21.15.112:0/3524034810 >> 172.21.15.112:0/3524034810 conn(0x7f28bc0fdfa0 msgr2=0x7f28bc1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 -- 172.21.15.112:0/3524034810 shutdown_connections 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 -- 172.21.15.112:0/3524034810 wait complete. 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 Processor -- start 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.017+0000 7f28c2351640 1 -- start start 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c2351640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc102a30 0x7f28bc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c2351640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28bc103c20 0x7f28bc199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c2351640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc1045e0 0x7f28bc1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c2351640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10e410 con 0x7f28bc102a30 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c2351640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10e290 con 0x7f28bc1045e0 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c2351640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc10e590 con 0x7f28bc103c20 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc102a30 0x7f28bc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc102a30 0x7f28bc198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57208/0 (socket says 172.21.15.112:57208) 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 -- 172.21.15.112:0/1653018540 learned_addr learned my addr 172.21.15.112:0/1653018540 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28b3fff640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28bc103c20 0x7f28bc199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c08ee640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc1045e0 0x7f28bc1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 -- 172.21.15.112:0/1653018540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28bc103c20 msgr2=0x7f28bc199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28bc103c20 0x7f28bc199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 -- 172.21.15.112:0/1653018540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc1045e0 msgr2=0x7f28bc1a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc1045e0 0x7f28bc1a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 -- 172.21.15.112:0/1653018540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc1a0730 con 0x7f28bc102a30 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28c08ee640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc1045e0 0x7f28bc1a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.018+0000 7f28bbfff640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc102a30 0x7f28bc198b10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00e9c0 tx=0x7f28ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.019+0000 7f28b9ffb640 1 -- 172.21.15.112:0/1653018540 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00cd80 con 0x7f28bc102a30 2024-05-10T10:43:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.019+0000 7f28b9ffb640 1 -- 172.21.15.112:0/1653018540 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28ac0035f0 con 0x7f28bc102a30 2024-05-10T10:43:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.019+0000 7f28b9ffb640 1 -- 172.21.15.112:0/1653018540 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0044c0 con 0x7f28bc102a30 2024-05-10T10:43:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.019+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28bc074c40 con 0x7f28bc102a30 2024-05-10T10:43:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.019+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28bc075110 con 0x7f28bc102a30 2024-05-10T10:43:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.022+0000 7f28b9ffb640 1 -- 172.21.15.112:0/1653018540 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f28ac004000 con 0x7f28bc102a30 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 Processor -- start 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 -- start start 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241ff21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241ff72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a241ff9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a241f565d0 con 0x55a241ff7910 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a241f0f890 con 0x55a241ff27a0 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c506d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a241f52590 con 0x55a241ff1db0 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28bbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241ff72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a241ff9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28c08ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241ff21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a241ff9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57220/0 (socket says 172.21.15.112:57220) 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28b3fff640 1 -- 172.21.15.112:0/3957814477 learned_addr learned my addr 172.21.15.112:0/3957814477 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.023+0000 7f28b9ffb640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28a80b2920 0x7f28a80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28c08ee640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28a80b2920 0x7f28a80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b3fff640 1 -- 172.21.15.112:0/3957814477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 msgr2=0x55a241ff21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b3fff640 1 --2- 172.21.15.112:0/3957814477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241ff21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b3fff640 1 -- 172.21.15.112:0/3957814477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 msgr2=0x55a241ff72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b3fff640 1 --2- 172.21.15.112:0/3957814477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241ff72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b3fff640 1 -- 172.21.15.112:0/3957814477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a241c68210 con 0x55a241ff7910 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b9ffb640 1 -- 172.21.15.112:0/1653018540 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f28ac0e2990 con 0x7f28bc102a30 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b3fff640 1 --2- 172.21.15.112:0/3957814477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a241ff9da0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f28a402a930 tx=0x7f28a4005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3429ed568b680f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28c08ee640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28a80b2920 0x7f28a80b4de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55a241ff3000 tx=0x7f28b4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b27fc640 1 -- 172.21.15.112:0/3957814477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4049070 con 0x55a241ff7910 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b27fc640 1 -- 172.21.15.112:0/3957814477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28a4004030 con 0x55a241ff7910 2024-05-10T10:43:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.024+0000 7f28b27fc640 1 -- 172.21.15.112:0/3957814477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4049070 con 0x55a241ff7910 2024-05-10T10:43:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 -- 172.21.15.112:0/3957814477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 msgr2=0x55a241ff9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 --2- 172.21.15.112:0/3957814477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a241ff9da0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f28a402a930 tx=0x7f28a4005bf0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 -- 172.21.15.112:0/3957814477 shutdown_connections 2024-05-10T10:43:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 --2- 172.21.15.112:0/3957814477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a241ff9da0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 --2- 172.21.15.112:0/3957814477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241ff72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 --2- 172.21.15.112:0/3957814477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241ff21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 -- 172.21.15.112:0/3957814477 >> 172.21.15.112:0/3957814477 conn(0x55a241fed420 msgr2=0x55a241fed820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 -- 172.21.15.112:0/3957814477 shutdown_connections 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.025+0000 7f28c506d740 1 -- 172.21.15.112:0/3957814477 wait complete. 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 -- start start 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 Processor -- start 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff1db0 0x55a241ffbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff27a0 0x55a241ffc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff7910 0x55a242003540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a241ffa860 con 0x55a241ff27a0 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a241f13370 con 0x55a241ff1db0 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a241ffa9e0 con 0x55a241ff7910 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28c506d740 10 client.0 _ll_register_callbacks cb 0x55a241c67a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.026+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff27a0 0x55a241ffc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff1db0 0x55a241ffbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff1db0 0x55a241ffbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34164/0 (socket says 172.21.15.112:34164) 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 -- 172.21.15.112:0/3753097251 learned_addr learned my addr 172.21.15.112:0/3753097251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28c08ee640 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff7910 0x55a242003540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 -- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff7910 msgr2=0x55a242003540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff7910 0x55a242003540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 -- 172.21.15.112:0/3753097251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff27a0 msgr2=0x55a241ffc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff27a0 0x55a241ffc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 -- 172.21.15.112:0/3753097251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a241d34620 con 0x55a241ff1db0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28bbfff640 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff1db0 0x55a241ffbfc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0a9b50 tx=0x7f28ac0a9330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24755435940260a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f289b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f289b7fe640 1 -- 172.21.15.112:0/3753097251 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0042f0 con 0x55a241ff1db0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f289b7fe640 1 -- 172.21.15.112:0/3753097251 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f28ac0a5e50 con 0x55a241ff1db0 2024-05-10T10:43:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a4004030 con 0x55a241ff1db0 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f289b7fe640 1 -- 172.21.15.112:0/3753097251 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f28ac0aa2c0 con 0x55a241ff1db0 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28c506d740 10 client.25640 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.027+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a242006590 con 0x55a241ff1db0 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f289b7fe640 1 -- 172.21.15.112:0/3753097251 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28a4004030 con 0x55a241ff1db0 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f289b7fe640 1 -- 172.21.15.112:0/3753097251 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f28ac0c40e0 con 0x55a241ff1db0 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28c506d740 10 client.25640 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28c506d740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28c506d740 20 client.25640 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28c506d740 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a242007c20 0x55a24200a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28c506d740 4 client.25640 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a241f641e0 con 0x55a242007c20 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28b3fff640 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a242007c20 0x55a24200a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f289b7fe640 10 client.25640 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.028+0000 7f28b3fff640 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a242007c20 0x55a24200a0b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587e36beaa80f195 server_cookie=9afac6049fd436f6 in_seq=0 out_seq=0 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.029+0000 7f289b7fe640 1 -- 172.21.15.112:0/3753097251 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a241f641e0 con 0x55a242007c20 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.029+0000 7f289b7fe640 10 client.25640 handle_command_reply: tid=0 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.029+0000 7f28c506d740 1 client.25640 shutdown 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.029+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff1db0 msgr2=0x55a241ffbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.029+0000 7f28c506d740 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff1db0 0x55a241ffbfc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0a9b50 tx=0x7f28ac0a9330 comp rx=0 tx=0).stop 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 shutdown_connections 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff7910 0x55a242003540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff27a0 0x55a241ffc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a242007c20 0x55a24200a0b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 --2- 172.21.15.112:0/3753097251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff1db0 0x55a241ffbfc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f289b7fe640 0 client.25640 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 >> 172.21.15.112:0/3753097251 conn(0x55a241fed420 msgr2=0x55a241fec580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 shutdown_connections 2024-05-10T10:43:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 1 -- 172.21.15.112:0/3753097251 wait complete. 2024-05-10T10:43:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.030+0000 7f28c506d740 20 client.25640 trim_cache size 0 max 16384 2024-05-10T10:43:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 Processor -- start 2024-05-10T10:43:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 -- start start 2024-05-10T10:43:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241fec400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241fec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a24200e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1a91c0 con 0x55a241ff7910 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1aae80 con 0x55a241ff27a0 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c506d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1a9890 con 0x55a241ff1db0 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a24200e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a24200e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57240/0 (socket says 172.21.15.112:57240) 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241fec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 -- 172.21.15.112:0/2656584720 learned_addr learned my addr 172.21.15.112:0/2656584720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28bbfff640 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241fec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 -- 172.21.15.112:0/2656584720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 msgr2=0x55a241fec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241fec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 -- 172.21.15.112:0/2656584720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 msgr2=0x55a241fec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241fec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.047+0000 7f28c08ee640 1 -- 172.21.15.112:0/2656584720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a241fed250 con 0x55a241ff7910 2024-05-10T10:43:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.048+0000 7f28c08ee640 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a24200e9b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f28b40098d0 tx=0x7f28b4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14ec601901ac3ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.048+0000 7f28b1ffb640 1 -- 172.21.15.112:0/2656584720 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4027030 con 0x55a241ff7910 2024-05-10T10:43:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.048+0000 7f28b1ffb640 1 -- 172.21.15.112:0/2656584720 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28b40238e0 con 0x55a241ff7910 2024-05-10T10:43:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.048+0000 7f28b1ffb640 1 -- 172.21.15.112:0/2656584720 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b40104e0 con 0x55a241ff7910 2024-05-10T10:43:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 -- 172.21.15.112:0/2656584720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 msgr2=0x55a24200e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a24200e9b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f28b40098d0 tx=0x7f28b4009900 comp rx=0 tx=0).stop 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 -- 172.21.15.112:0/2656584720 shutdown_connections 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a24200e9b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a241fec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 --2- 172.21.15.112:0/2656584720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a241fec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 -- 172.21.15.112:0/2656584720 >> 172.21.15.112:0/2656584720 conn(0x55a241fed420 msgr2=0x55a241febcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 -- 172.21.15.112:0/2656584720 shutdown_connections 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 -- 172.21.15.112:0/2656584720 wait complete. 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.049+0000 7f28c506d740 1 -- start start 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 1 Processor -- start 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a242000990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a242000ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a242001e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1aa6f0 con 0x55a241ff7910 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1aa020 con 0x55a241ff27a0 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1a71a0 con 0x55a241ff1db0 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a242001e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a242001e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57248/0 (socket says 172.21.15.112:57248) 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 -- 172.21.15.112:0/2157716142 learned_addr learned my addr 172.21.15.112:0/2157716142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28bbfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a242000990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28b3fff640 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a242000ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c506d740 10 client.0 _ll_register_callbacks cb 0x7f28ac005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 -- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 msgr2=0x55a242000990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a242000990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 -- 172.21.15.112:0/2157716142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 msgr2=0x55a242000ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a242000ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28c08ee640 1 -- 172.21.15.112:0/2157716142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a242002460 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.050+0000 7f28bbfff640 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a242000990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f28c08ee640 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a242001e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f28b4014a20 tx=0x7f28b4009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65685cc437a4cceb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f289b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f289b7fe640 1 -- 172.21.15.112:0/2157716142 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4032070 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b40238e0 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f289b7fe640 1 -- 172.21.15.112:0/2157716142 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f28b4023450 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f289b7fe640 1 -- 172.21.15.112:0/2157716142 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f28b403e050 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f28c506d740 10 client.16257 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a242072bc0 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.051+0000 7f289b7fe640 1 -- 172.21.15.112:0/2157716142 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28b40238e0 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f289b7fe640 1 -- 172.21.15.112:0/2157716142 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f28b400f040 con 0x55a241ff7910 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28c506d740 10 client.16257 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28c506d740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28c506d740 20 client.16257 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28c506d740 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2420735b0 0x55a2420759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28c506d740 4 client.16257 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:43:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a241f5e050 con 0x55a2420735b0 2024-05-10T10:43:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28bbfff640 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2420735b0 0x55a2420759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f289b7fe640 10 client.16257 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.052+0000 7f28bbfff640 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2420735b0 0x55a2420759d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372e0f1185070b75 server_cookie=90cd8b8609d246cf in_seq=0 out_seq=0 2024-05-10T10:43:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f289b7fe640 1 -- 172.21.15.112:0/2157716142 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a241f5e050 con 0x55a2420735b0 2024-05-10T10:43:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f289b7fe640 10 client.16257 handle_command_reply: tid=0 2024-05-10T10:43:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 client.16257 shutdown 2024-05-10T10:43:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 msgr2=0x55a242001e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a242001e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f28b4014a20 tx=0x7f28b4009e10 comp rx=0 tx=0).stop 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 shutdown_connections 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a241ff7910 0x55a242001e70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55a2420735b0 0x55a2420759d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a241ff27a0 0x55a242000ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 --2- 172.21.15.112:0/2157716142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a241ff1db0 0x55a242000990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f289b7fe640 0 client.16257 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 >> 172.21.15.112:0/2157716142 conn(0x55a241fed420 msgr2=0x55a241febcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 shutdown_connections 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.072+0000 7f28c506d740 1 -- 172.21.15.112:0/2157716142 wait complete. 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c506d740 20 client.16257 trim_cache size 0 max 16384 2024-05-10T10:43:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:43:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:43:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:43:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:43:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28a80b2920 msgr2=0x7f28a80b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28a80b2920 0x7f28a80b4de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55a241ff3000 tx=0x7f28b4004ab0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc102a30 msgr2=0x7f28bc198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc102a30 0x7f28bc198b10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00e9c0 tx=0x7f28ac00ee80 comp rx=0 tx=0).stop 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 shutdown_connections 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f28a80b2920 0x7f28a80b4de0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc1045e0 0x7f28bc1a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28bc103c20 0x7f28bc199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 --2- 172.21.15.112:0/1653018540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28bc102a30 0x7f28bc198b10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 >> 172.21.15.112:0/1653018540 conn(0x7f28bc0fdfa0 msgr2=0x7f28bc107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 shutdown_connections 2024-05-10T10:43:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:38.073+0000 7f28c2351640 1 -- 172.21.15.112:0/1653018540 wait complete. 2024-05-10T10:43:39.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:39 smithi112 ceph-mon[30100]: pgmap v532: 129 pgs: 129 active+clean; 41 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 477 op/s 2024-05-10T10:43:39.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:39 smithi148 ceph-mon[33568]: pgmap v532: 129 pgs: 129 active+clean; 41 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 477 op/s 2024-05-10T10:43:39.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:39 smithi195 ceph-mon[33773]: pgmap v532: 129 pgs: 129 active+clean; 41 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 477 op/s 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 -- 172.21.15.112:0/2694511225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014108410 msgr2=0x7f40141087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 --2- 172.21.15.112:0/2694511225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014108410 0x7f40141087f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4004009990 tx=0x7f4004009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 -- 172.21.15.112:0/2694511225 shutdown_connections 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 --2- 172.21.15.112:0/2694511225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4014102d60 0x7f401410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 --2- 172.21.15.112:0/2694511225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014108410 0x7f40141087f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 --2- 172.21.15.112:0/2694511225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40141023a0 0x7f4014102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 -- 172.21.15.112:0/2694511225 >> 172.21.15.112:0/2694511225 conn(0x7f4014079350 msgr2=0x7f4014079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 -- 172.21.15.112:0/2694511225 shutdown_connections 2024-05-10T10:43:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.367+0000 7f4019064700 1 -- 172.21.15.112:0/2694511225 wait complete. 2024-05-10T10:43:39.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.368+0000 7f4019064700 1 Processor -- start 2024-05-10T10:43:39.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.368+0000 7f4019064700 1 -- start start 2024-05-10T10:43:39.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4019064700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40141023a0 0x7f4014104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4019064700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4014102d60 0x7f40141045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4019064700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014104b00 0x7f40141a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4019064700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4014112460 con 0x7f4014104b00 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4019064700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40141122e0 con 0x7f4014102d60 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4019064700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40141125e0 con 0x7f40141023a0 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f401359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014104b00 0x7f40141a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4012d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40141023a0 0x7f4014104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f401359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014104b00 0x7f40141a7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57272/0 (socket says 172.21.15.112:57272) 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f401359e700 1 -- 172.21.15.112:0/2174542221 learned_addr learned my addr 172.21.15.112:0/2174542221 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f401259c700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4014102d60 0x7f40141045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4012d9d700 1 -- 172.21.15.112:0/2174542221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4014102d60 msgr2=0x7f40141045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4012d9d700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4014102d60 0x7f40141045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4012d9d700 1 -- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014104b00 msgr2=0x7f40141a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4012d9d700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014104b00 0x7f40141a7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4012d9d700 1 -- 172.21.15.112:0/2174542221 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141a86f0 con 0x7f40141023a0 2024-05-10T10:43:39.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.369+0000 7f4012d9d700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40141023a0 0x7f4014104080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4004008000 tx=0x7f40040047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:39.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.370+0000 7f3ffbfff700 1 -- 172.21.15.112:0/2174542221 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004026070 con 0x7f40141023a0 2024-05-10T10:43:39.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.370+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141a8980 con 0x7f40141023a0 2024-05-10T10:43:39.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.370+0000 7f3ffbfff700 1 -- 172.21.15.112:0/2174542221 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4004021410 con 0x7f40141023a0 2024-05-10T10:43:39.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.370+0000 7f3ffbfff700 1 -- 172.21.15.112:0/2174542221 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004017650 con 0x7f40141023a0 2024-05-10T10:43:39.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.370+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a8dd0 con 0x7f40141023a0 2024-05-10T10:43:39.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.373+0000 7f3ffbfff700 1 -- 172.21.15.112:0/2174542221 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4004029070 con 0x7f40141023a0 2024-05-10T10:43:39.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f3ffbfff700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ffc0b2880 0x7f3ffc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f3ffbfff700 1 -- 172.21.15.112:0/2174542221 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f40040e3180 con 0x7f40141023a0 2024-05-10T10:43:39.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f401259c700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ffc0b2880 0x7f3ffc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 Processor -- start 2024-05-10T10:43:39.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 -- start start 2024-05-10T10:43:39.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba283590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba27d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba27e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddba1cf940 con 0x55ddba27dd50 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ddba134da0 con 0x55ddba283ad0 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.374+0000 7f40238f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ddba1cb900 con 0x55ddba2831b0 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba27e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba27e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57276/0 (socket says 172.21.15.112:57276) 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 -- 172.21.15.112:0/3018875738 learned_addr learned my addr 172.21.15.112:0/3018875738 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401259c700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ffc0b2880 0x7f3ffc0b4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4008009680 tx=0x7f4008009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401259c700 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba27d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba283590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 -- 172.21.15.112:0/3018875738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 msgr2=0x55ddba283590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba283590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 -- 172.21.15.112:0/3018875738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 msgr2=0x55ddba27d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba27d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 -- 172.21.15.112:0/3018875738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddba11c8d0 con 0x55ddba27dd50 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba283590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401259c700 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba27d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f401359e700 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba27e190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f400c00e6a0 tx=0x7f400c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff9d06f73e0a75d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.375+0000 7f3ff97fa700 1 -- 172.21.15.112:0/3018875738 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c019030 con 0x55ddba27dd50 2024-05-10T10:43:39.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f3ff97fa700 1 -- 172.21.15.112:0/3018875738 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f400c0093a0 con 0x55ddba27dd50 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f3ff97fa700 1 -- 172.21.15.112:0/3018875738 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c007500 con 0x55ddba27dd50 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 -- 172.21.15.112:0/3018875738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 msgr2=0x55ddba27e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba27e190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f400c00e6a0 tx=0x7f400c009920 comp rx=0 tx=0).stop 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 -- 172.21.15.112:0/3018875738 shutdown_connections 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba27e190 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba283590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3018875738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba27d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 -- 172.21.15.112:0/3018875738 >> 172.21.15.112:0/3018875738 conn(0x55ddba1a9b70 msgr2=0x55ddba2782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:39.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.376+0000 7f40238f7b80 1 -- 172.21.15.112:0/3018875738 shutdown_connections 2024-05-10T10:43:39.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.377+0000 7f40238f7b80 1 -- 172.21.15.112:0/3018875738 wait complete. 2024-05-10T10:43:39.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.377+0000 7f40238f7b80 1 -- start start 2024-05-10T10:43:39.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.377+0000 7f40238f7b80 1 Processor -- start 2024-05-10T10:43:39.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.377+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba27dd50 0x55ddba289700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.377+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba286aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba286fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f40238f7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddba28a450 con 0x55ddba2831b0 2024-05-10T10:43:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f40238f7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ddba188ed0 con 0x55ddba283ad0 2024-05-10T10:43:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f40238f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ddba28a5d0 con 0x55ddba27dd50 2024-05-10T10:43:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba27dd50 0x55ddba289700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f40238f7b80 10 client.0 _ll_register_callbacks cb 0x55ddba1635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba27dd50 0x55ddba289700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:43572/0 (socket says 172.21.15.112:43572) 2024-05-10T10:43:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 -- 172.21.15.112:0/3644733287 learned_addr learned my addr 172.21.15.112:0/3644733287 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f401259c700 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba286aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f401359e700 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba286fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 -- 172.21.15.112:0/3644733287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 msgr2=0x55ddba286fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba286fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 -- 172.21.15.112:0/3644733287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 msgr2=0x55ddba286aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba286aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 -- 172.21.15.112:0/3644733287 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddba186760 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba27dd50 0x55ddba289700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f400400b2e0 tx=0x7f40040a9c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7316a75ddc333978 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.378+0000 7f3ff6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/3644733287 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040aaba0 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/3644733287 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f400400fb10 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddba1dd320 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/3644733287 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f40040aa4a0 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 10 client.25681 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ddba28a4a0 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/3644733287 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ddba1dd320 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/3644733287 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f40040b2de0 con 0x55ddba27dd50 2024-05-10T10:43:39.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 10 client.25681 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 10 client.25681 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 20 client.25681 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba297220 0x55ddba299650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 4 client.25681 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.379+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddba1d73c0 con 0x55ddba297220 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.380+0000 7f401259c700 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba297220 0x55ddba299650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.381+0000 7f3ff6ff5700 10 client.25681 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.381+0000 7f401259c700 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba297220 0x55ddba299650 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d697f6884b188991 server_cookie=24c3984737770288 in_seq=0 out_seq=0 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.381+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/3644733287 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ddba1d73c0 con 0x55ddba297220 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.381+0000 7f3ff6ff5700 10 client.25681 handle_command_reply: tid=0 2024-05-10T10:43:39.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.381+0000 7f40238f7b80 1 client.25681 shutdown 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba27dd50 msgr2=0x55ddba289700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba27dd50 0x55ddba289700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f400400b2e0 tx=0x7f40040a9c90 comp rx=0 tx=0).stop 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 shutdown_connections 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba297220 0x55ddba299650 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba286fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba27dd50 0x55ddba289700 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3644733287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba286aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f3ff6ff5700 0 client.25681 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 >> 172.21.15.112:0/3644733287 conn(0x55ddba1a9b70 msgr2=0x55ddba272b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 shutdown_connections 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 1 -- 172.21.15.112:0/3644733287 wait complete. 2024-05-10T10:43:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.382+0000 7f40238f7b80 20 client.25681 trim_cache size 0 max 16384 2024-05-10T10:43:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 Processor -- start 2024-05-10T10:43:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 -- start start 2024-05-10T10:43:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba27dd50 0x55ddba27d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba293540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba283ad0 0x55ddba272c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b1900 con 0x55ddba2831b0 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b35c0 con 0x55ddba27dd50 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.398+0000 7f40238f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b1fd0 con 0x55ddba283ad0 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f401259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba293540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba27dd50 0x55ddba27d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba27dd50 0x55ddba27d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34228/0 (socket says 172.21.15.112:34228) 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f401259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba293540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57300/0 (socket says 172.21.15.112:57300) 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 -- 172.21.15.112:0/3628805499 learned_addr learned my addr 172.21.15.112:0/3628805499 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f401359e700 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba283ad0 0x55ddba272c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 -- 172.21.15.112:0/3628805499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba283ad0 msgr2=0x55ddba272c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba283ad0 0x55ddba272c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 -- 172.21.15.112:0/3628805499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 msgr2=0x55ddba293540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba293540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 -- 172.21.15.112:0/3628805499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddba299cd0 con 0x55ddba27dd50 2024-05-10T10:43:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f401259c700 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba293540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f4012d9d700 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba27dd50 0x55ddba27d810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f40040a64d0 tx=0x7f40040096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9526281ba3bcadd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:39.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f3ff8ff9700 1 -- 172.21.15.112:0/3628805499 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040aaba0 con 0x55ddba27dd50 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f3ff8ff9700 1 -- 172.21.15.112:0/3628805499 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40040ae850 con 0x55ddba27dd50 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.399+0000 7f3ff8ff9700 1 -- 172.21.15.112:0/3628805499 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400400fb10 con 0x55ddba27dd50 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 -- 172.21.15.112:0/3628805499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba27dd50 msgr2=0x55ddba27d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba27dd50 0x55ddba27d810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f40040a64d0 tx=0x7f40040096c0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 -- 172.21.15.112:0/3628805499 shutdown_connections 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba283ad0 0x55ddba272c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba27dd50 0x55ddba27d810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 --2- 172.21.15.112:0/3628805499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba2831b0 0x55ddba293540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 -- 172.21.15.112:0/3628805499 >> 172.21.15.112:0/3628805499 conn(0x55ddba1a9b70 msgr2=0x55ddba285390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 -- 172.21.15.112:0/3628805499 shutdown_connections 2024-05-10T10:43:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 -- 172.21.15.112:0/3628805499 wait complete. 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.400+0000 7f40238f7b80 1 -- start start 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 1 Processor -- start 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba28fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba28b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba28b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b2e30 con 0x55ddba27dd50 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b2760 con 0x55ddba283ad0 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40141af8a0 con 0x55ddba2831b0 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba28fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f40238f7b80 10 client.0 _ll_register_callbacks cb 0x55ddba1635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:39.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba28fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57310/0 (socket says 172.21.15.112:57310) 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 -- 172.21.15.112:0/2550651073 learned_addr learned my addr 172.21.15.112:0/2550651073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f401259c700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba28b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f401359e700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba28b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 -- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 msgr2=0x55ddba28b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba28b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 -- 172.21.15.112:0/2550651073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 msgr2=0x55ddba28b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba28b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 -- 172.21.15.112:0/2550651073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddba28bd30 con 0x55ddba27dd50 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f401259c700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba28b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.401+0000 7f4012d9d700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba28fd10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4004009390 tx=0x7f4004009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e16cfbabf1f436 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f3ff6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/2550651073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040c8020 con 0x55ddba27dd50 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/2550651073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f40040a9700 con 0x55ddba27dd50 2024-05-10T10:43:39.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40040ae850 con 0x55ddba27dd50 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/2550651073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f40040b8050 con 0x55ddba27dd50 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f40238f7b80 10 client.16275 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ddba28bd30 con 0x55ddba27dd50 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.402+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/2550651073 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40040aaba0 con 0x55ddba27dd50 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/2550651073 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f40040cb300 con 0x55ddba27dd50 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f40238f7b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f40238f7b80 10 client.16275 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f40238f7b80 20 client.16275 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f40238f7b80 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba301cf0 0x55ddba3040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f40238f7b80 4 client.16275 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f40040ae850 con 0x55ddba301cf0 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f401259c700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba301cf0 0x55ddba3040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f3ff6ff5700 10 client.16275 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:43:39.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f401259c700 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba301cf0 0x55ddba3040e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f31fcee38e1403c8 server_cookie=2ea3308cfd04e2e5 in_seq=0 out_seq=0 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f3ff6ff5700 1 -- 172.21.15.112:0/2550651073 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f40040ae850 con 0x55ddba301cf0 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f3ff6ff5700 10 client.16275 handle_command_reply: tid=0 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.403+0000 7f40238f7b80 1 client.16275 shutdown 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 msgr2=0x55ddba28fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba28fd10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4004009390 tx=0x7f4004009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 shutdown_connections 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55ddba301cf0 0x55ddba3040e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ddba283ad0 0x55ddba28b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddba27dd50 0x55ddba28fd10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 --2- 172.21.15.112:0/2550651073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ddba2831b0 0x55ddba28b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 >> 172.21.15.112:0/2550651073 conn(0x55ddba1a9b70 msgr2=0x55ddba288930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:39.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f3ff6ff5700 0 client.16275 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:43:39.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 shutdown_connections 2024-05-10T10:43:39.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 1 -- 172.21.15.112:0/2550651073 wait complete. 2024-05-10T10:43:39.412 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.404+0000 7f40238f7b80 20 client.16275 trim_cache size 0 max 16384 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ffc0b2880 msgr2=0x7f3ffc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ffc0b2880 0x7f3ffc0b4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4008009680 tx=0x7f4008009260 comp rx=0 tx=0).stop 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40141023a0 msgr2=0x7f4014104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40141023a0 0x7f4014104080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4004008000 tx=0x7f40040047b0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 shutdown_connections 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4014104b00 0x7f40141a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ffc0b2880 0x7f3ffc0b4d40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40141023a0 0x7f4014104080 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 --2- 172.21.15.112:0/2174542221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4014102d60 0x7f40141045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 >> 172.21.15.112:0/2174542221 conn(0x7f4014079350 msgr2=0x7f401406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 shutdown_connections 2024-05-10T10:43:39.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:39.405+0000 7f4019064700 1 -- 172.21.15.112:0/2174542221 wait complete. 2024-05-10T10:43:39.475 INFO:tasks.workunit.client.0.smithi112.stdout:20000 punch from 0x9116f to 0x978af, (0x6740 bytes) 2024-05-10T10:43:40.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:43:40.560 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:41.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:41 smithi148 ceph-mon[33568]: pgmap v533: 129 pgs: 129 active+clean; 42 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.1 MiB/s wr, 688 op/s 2024-05-10T10:43:41.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:41 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:43:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:41 smithi195 ceph-mon[33773]: pgmap v533: 129 pgs: 129 active+clean; 42 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.1 MiB/s wr, 688 op/s 2024-05-10T10:43:41.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:41 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:43:41.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:41 smithi112 ceph-mon[30100]: pgmap v533: 129 pgs: 129 active+clean; 42 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.1 MiB/s wr, 688 op/s 2024-05-10T10:43:41.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:41 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:43:42.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:43:42.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:43:42.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:43:42.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:43:42.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:43:42.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:43:42.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:43:42.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:43:42.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 -- 172.21.15.112:0/3498714010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8781023a0 msgr2=0x7fd878102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 --2- 172.21.15.112:0/3498714010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8781023a0 0x7fd878102820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009990 tx=0x7fd86c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 -- 172.21.15.112:0/3498714010 shutdown_connections 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 --2- 172.21.15.112:0/3498714010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 0x7fd87810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 --2- 172.21.15.112:0/3498714010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd878108410 0x7fd8781087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 --2- 172.21.15.112:0/3498714010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8781023a0 0x7fd878102820 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 -- 172.21.15.112:0/3498714010 >> 172.21.15.112:0/3498714010 conn(0x7fd878079350 msgr2=0x7fd878079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 -- 172.21.15.112:0/3498714010 shutdown_connections 2024-05-10T10:43:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.489+0000 7fd87e38f700 1 -- 172.21.15.112:0/3498714010 wait complete. 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.490+0000 7fd87e38f700 1 Processor -- start 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.490+0000 7fd87e38f700 1 -- start start 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.490+0000 7fd87e38f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8781023a0 0x7fd87819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.490+0000 7fd87e38f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 0x7fd87819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.490+0000 7fd87e38f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd878108410 0x7fd8781a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd87e38f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd878112410 con 0x7fd878108410 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd87e38f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd878112290 con 0x7fd8781023a0 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd87e38f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd878112590 con 0x7fd878102d60 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 0x7fd87819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd87c92c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd878108410 0x7fd8781a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 0x7fd87819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:43604/0 (socket says 172.21.15.112:43604) 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 -- 172.21.15.112:0/2659172058 learned_addr learned my addr 172.21.15.112:0/2659172058 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd877fff700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8781023a0 0x7fd87819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 -- 172.21.15.112:0/2659172058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8781023a0 msgr2=0x7fd87819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8781023a0 0x7fd87819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 -- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd878108410 msgr2=0x7fd8781a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd878108410 0x7fd8781a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 -- 172.21.15.112:0/2659172058 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8781986c0 con 0x7fd878102d60 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd87c92c700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd878108410 0x7fd8781a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 0x7fd87819ae10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd86c006010 tx=0x7fd86c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8757fa700 1 -- 172.21.15.112:0/2659172058 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c027070 con 0x7fd878102d60 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd878198950 con 0x7fd878102d60 2024-05-10T10:43:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8757fa700 1 -- 172.21.15.112:0/2659172058 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c022470 con 0x7fd878102d60 2024-05-10T10:43:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.491+0000 7fd8757fa700 1 -- 172.21.15.112:0/2659172058 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c017650 con 0x7fd878102d60 2024-05-10T10:43:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.492+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd878198e80 con 0x7fd878102d60 2024-05-10T10:43:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.494+0000 7fd8757fa700 1 -- 172.21.15.112:0/2659172058 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd86c009d10 con 0x7fd878102d60 2024-05-10T10:43:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.494+0000 7fd8757fa700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8600b2900 0x7fd8600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.495+0000 7fd8757fa700 1 -- 172.21.15.112:0/2659172058 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd86c0e2ff0 con 0x7fd878102d60 2024-05-10T10:43:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.495+0000 7fd877fff700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8600b2900 0x7fd8600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.495+0000 7fd877fff700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8600b2900 0x7fd8600b4dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd868009be0 tx=0x7fd868009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.495+0000 7fd888c22b80 1 Processor -- start 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.495+0000 7fd888c22b80 1 -- start start 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca654d0 0x559c6ca658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca65df0 0x559c6ca696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca6a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd888c22b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559c6c9b7860 con 0x559c6ca65df0 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd888c22b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c6c9567c0 con 0x559c6ca654d0 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd888c22b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559c6c9b3820 con 0x559c6ca69c80 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca65df0 0x559c6ca696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca65df0 0x559c6ca696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57318/0 (socket says 172.21.15.112:57318) 2024-05-10T10:43:42.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 -- 172.21.15.112:0/2292869322 learned_addr learned my addr 172.21.15.112:0/2292869322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd87c92c700 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca6a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd877fff700 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca654d0 0x559c6ca658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 -- 172.21.15.112:0/2292869322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 msgr2=0x559c6ca6a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca6a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 -- 172.21.15.112:0/2292869322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca654d0 msgr2=0x559c6ca658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca654d0 0x559c6ca658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 -- 172.21.15.112:0/2292869322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6c9048d0 con 0x559c6ca65df0 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca65df0 0x559c6ca696f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0aa240 tx=0x7fd86c0a9aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fcf5313698a50c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd85e7fc700 1 -- 172.21.15.112:0/2292869322 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0aa9b0 con 0x559c6ca65df0 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd85e7fc700 1 -- 172.21.15.112:0/2292869322 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c0aa2b0 con 0x559c6ca65df0 2024-05-10T10:43:42.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.496+0000 7fd85e7fc700 1 -- 172.21.15.112:0/2292869322 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0b2bf0 con 0x559c6ca65df0 2024-05-10T10:43:42.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 -- 172.21.15.112:0/2292869322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca65df0 msgr2=0x559c6ca696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca65df0 0x559c6ca696f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0aa240 tx=0x7fd86c0a9aa0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 -- 172.21.15.112:0/2292869322 shutdown_connections 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca6a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca654d0 0x559c6ca658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2292869322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca65df0 0x559c6ca696f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 -- 172.21.15.112:0/2292869322 >> 172.21.15.112:0/2292869322 conn(0x559c6c991100 msgr2=0x559c6ca5fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 -- 172.21.15.112:0/2292869322 shutdown_connections 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.497+0000 7fd888c22b80 1 -- 172.21.15.112:0/2292869322 wait complete. 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 -- start start 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 Processor -- start 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca71120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca65df0 0x559c6ca6e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca69c80 0x559c6ca6ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559c6ca71c50 con 0x559c6ca654d0 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c6c91dc30 con 0x559c6ca69c80 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559c6ca71dd0 con 0x559c6ca65df0 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd877fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca71120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd877fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca71120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57328/0 (socket says 172.21.15.112:57328) 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd877fff700 1 -- 172.21.15.112:0/2673325441 learned_addr learned my addr 172.21.15.112:0/2673325441 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.498+0000 7fd888c22b80 10 client.0 _ll_register_callbacks cb 0x559c6c94f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd87c92c700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca69c80 0x559c6ca6ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca65df0 0x559c6ca6e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd877fff700 1 -- 172.21.15.112:0/2673325441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca65df0 msgr2=0x559c6ca6e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd877fff700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca65df0 0x559c6ca6e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd877fff700 1 -- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca69c80 msgr2=0x559c6ca6ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd877fff700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca69c80 0x559c6ca6ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd877fff700 1 -- 172.21.15.112:0/2673325441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6c953690 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca65df0 0x559c6ca6e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd877fff700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca71120 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd86801f900 tx=0x7fd8680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867275186f03b809 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd8577fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd8577fe700 1 -- 172.21.15.112:0/2673325441 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86802c060 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd8577fe700 1 -- 172.21.15.112:0/2673325441 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd868009040 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.499+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86c0aa2b0 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd8577fe700 1 -- 172.21.15.112:0/2673325441 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd868033050 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd888c22b80 10 client.16287 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559c6ca7c4e0 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd8577fe700 1 -- 172.21.15.112:0/2673325441 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c0aa2b0 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd8577fe700 1 -- 172.21.15.112:0/2673325441 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fd8680277f0 con 0x559c6ca654d0 2024-05-10T10:43:42.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd888c22b80 10 client.16287 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd888c22b80 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.500+0000 7fd888c22b80 20 client.16287 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.501+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6ca7eb20 0x559c6ca80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.501+0000 7fd888c22b80 4 client.16287 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.501+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c6ca31650 con 0x559c6ca7eb20 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.501+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6ca7eb20 0x559c6ca80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.501+0000 7fd8577fe700 10 client.16287 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.501+0000 7fd8777fe700 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6ca7eb20 0x559c6ca80f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26cc7346d61710f9 server_cookie=a40c77d24cedce2a in_seq=0 out_seq=0 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd8577fe700 1 -- 172.21.15.112:0/2673325441 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559c6ca31650 con 0x559c6ca7eb20 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd8577fe700 10 client.16287 handle_command_reply: tid=0 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 client.16287 shutdown 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 msgr2=0x559c6ca71120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca71120 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd86801f900 tx=0x7fd8680047c0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 shutdown_connections 2024-05-10T10:43:42.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca69c80 0x559c6ca6ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6ca7eb20 0x559c6ca80f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca71120 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 --2- 172.21.15.112:0/2673325441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca65df0 0x559c6ca6e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 >> 172.21.15.112:0/2673325441 conn(0x559c6c991100 msgr2=0x559c6ca5a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd8577fe700 0 client.16287 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 shutdown_connections 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 1 -- 172.21.15.112:0/2673325441 wait complete. 2024-05-10T10:43:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.502+0000 7fd888c22b80 20 client.16287 trim_cache size 0 max 16384 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 Processor -- start 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 -- start start 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca654d0 0x559c6ca7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca5a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca69c80 0x559c6ca868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781b1370 con 0x559c6ca69c80 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781b3030 con 0x559c6ca65df0 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd888c22b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781b1a40 con 0x559c6ca654d0 2024-05-10T10:43:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca5a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca5a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34284/0 (socket says 172.21.15.112:34284) 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.520+0000 7fd8777fe700 1 -- 172.21.15.112:0/4106350212 learned_addr learned my addr 172.21.15.112:0/4106350212 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd87c92c700 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca69c80 0x559c6ca868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd877fff700 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca654d0 0x559c6ca7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd8777fe700 1 -- 172.21.15.112:0/4106350212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca654d0 msgr2=0x559c6ca7aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd8777fe700 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca654d0 0x559c6ca7aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd8777fe700 1 -- 172.21.15.112:0/4106350212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca69c80 msgr2=0x559c6ca868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd8777fe700 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca69c80 0x559c6ca868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd8777fe700 1 -- 172.21.15.112:0/4106350212 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6ca60740 con 0x559c6ca65df0 2024-05-10T10:43:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd8777fe700 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca5a5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0a9fb0 tx=0x7fd86c0a9b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b01c84cd81bd60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:42.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd85dffb700 1 -- 172.21.15.112:0/4106350212 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0c4070 con 0x559c6ca65df0 2024-05-10T10:43:42.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd85dffb700 1 -- 172.21.15.112:0/4106350212 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c022920 con 0x559c6ca65df0 2024-05-10T10:43:42.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.521+0000 7fd85dffb700 1 -- 172.21.15.112:0/4106350212 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0b2e50 con 0x559c6ca65df0 2024-05-10T10:43:42.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 -- 172.21.15.112:0/4106350212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 msgr2=0x559c6ca5a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca5a5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0a9fb0 tx=0x7fd86c0a9b50 comp rx=0 tx=0).stop 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 -- 172.21.15.112:0/4106350212 shutdown_connections 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca69c80 0x559c6ca868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca654d0 0x559c6ca7aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 --2- 172.21.15.112:0/4106350212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca5a5a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 -- 172.21.15.112:0/4106350212 >> 172.21.15.112:0/4106350212 conn(0x559c6c991100 msgr2=0x559c6ca59e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.522+0000 7fd888c22b80 1 -- 172.21.15.112:0/4106350212 shutdown_connections 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.523+0000 7fd888c22b80 1 -- 172.21.15.112:0/4106350212 wait complete. 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.523+0000 7fd888c22b80 1 -- start start 2024-05-10T10:43:42.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.523+0000 7fd888c22b80 1 Processor -- start 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.523+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca76ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.523+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca77400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd888c22b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca72340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd888c22b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781b28a0 con 0x559c6ca654d0 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd888c22b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781b21d0 con 0x559c6ca65df0 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd888c22b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781af310 con 0x559c6ca69c80 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd888c22b80 10 client.0 _ll_register_callbacks cb 0x559c6c94f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca76ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca77400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca76ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57358/0 (socket says 172.21.15.112:57358) 2024-05-10T10:43:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 -- 172.21.15.112:0/1491783337 learned_addr learned my addr 172.21.15.112:0/1491783337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd87c92c700 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca72340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 -- 172.21.15.112:0/1491783337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 msgr2=0x559c6ca72340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca72340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 -- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 msgr2=0x559c6ca77400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca77400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 -- 172.21.15.112:0/1491783337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6ca72ae0 con 0x559c6ca654d0 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd8777fe700 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca77400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd877fff700 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca76ec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd868005950 tx=0x7fd868011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2975b567b7db143e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd8577fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.524+0000 7fd8577fe700 1 -- 172.21.15.112:0/1491783337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86803c060 con 0x559c6ca654d0 2024-05-10T10:43:42.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd8577fe700 1 -- 172.21.15.112:0/1491783337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd868004310 con 0x559c6ca654d0 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86c022920 con 0x559c6ca654d0 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd8577fe700 1 -- 172.21.15.112:0/1491783337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd868043050 con 0x559c6ca654d0 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 10 client.16293 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559c6cae8f20 con 0x559c6ca654d0 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd8577fe700 1 -- 172.21.15.112:0/1491783337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c022920 con 0x559c6ca654d0 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd8577fe700 1 -- 172.21.15.112:0/1491783337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fd86801d970 con 0x559c6ca654d0 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 10 client.16293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 20 client.16293 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6cae9690 0x559c6caebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:42.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 4 client.16293 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.525+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c6c9c5240 con 0x559c6cae9690 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd8777fe700 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6cae9690 0x559c6caebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd8577fe700 10 client.16293 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd8777fe700 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6cae9690 0x559c6caebab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77874577c5a83f8a server_cookie=95041d9cae5a4ac8 in_seq=0 out_seq=0 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd8577fe700 1 -- 172.21.15.112:0/1491783337 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c6c9c5240 con 0x559c6cae9690 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd8577fe700 10 client.16293 handle_command_reply: tid=0 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 client.16293 shutdown 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 msgr2=0x559c6ca76ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca76ec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd868005950 tx=0x7fd868011b10 comp rx=0 tx=0).stop 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 shutdown_connections 2024-05-10T10:43:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c6ca69c80 0x559c6ca72340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x559c6cae9690 0x559c6caebab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c6ca654d0 0x559c6ca76ec0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 --2- 172.21.15.112:0/1491783337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6ca65df0 0x559c6ca77400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.526+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 >> 172.21.15.112:0/1491783337 conn(0x559c6c991100 msgr2=0x559c6ca5a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd8577fe700 0 client.16293 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 shutdown_connections 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd888c22b80 1 -- 172.21.15.112:0/1491783337 wait complete. 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd888c22b80 20 client.16293 trim_cache size 0 max 16384 2024-05-10T10:43:42.535 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:43:42.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8600b2900 msgr2=0x7fd8600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd87e38f700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8600b2900 0x7fd8600b4dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd868009be0 tx=0x7fd868009260 comp rx=0 tx=0).stop 2024-05-10T10:43:42.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 msgr2=0x7fd87819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:42.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.527+0000 7fd87e38f700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 0x7fd87819ae10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd86c006010 tx=0x7fd86c0047b0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 shutdown_connections 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd878108410 0x7fd8781a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8781023a0 0x7fd87819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd8600b2900 0x7fd8600b4dc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 --2- 172.21.15.112:0/2659172058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd878102d60 0x7fd87819ae10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 >> 172.21.15.112:0/2659172058 conn(0x7fd878079350 msgr2=0x7fd878101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 shutdown_connections 2024-05-10T10:43:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:42.528+0000 7fd87e38f700 1 -- 172.21.15.112:0/2659172058 wait complete. 2024-05-10T10:43:43.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:43 smithi148 ceph-mon[33568]: pgmap v534: 129 pgs: 129 active+clean; 42 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-05-10T10:43:43.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:43 smithi195 ceph-mon[33773]: pgmap v534: 129 pgs: 129 active+clean; 42 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-05-10T10:43:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:43 smithi112 ceph-mon[30100]: pgmap v534: 129 pgs: 129 active+clean; 42 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-05-10T10:43:43.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:43:43.838 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:45.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:45 smithi148 ceph-mon[33568]: pgmap v535: 129 pgs: 129 active+clean; 39 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.8 MiB/s wr, 639 op/s 2024-05-10T10:43:45.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:45 smithi112 ceph-mon[30100]: pgmap v535: 129 pgs: 129 active+clean; 39 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.8 MiB/s wr, 639 op/s 2024-05-10T10:43:45.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:45 smithi195 ceph-mon[33773]: pgmap v535: 129 pgs: 129 active+clean; 39 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.8 MiB/s wr, 639 op/s 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.702+0000 7fbc53229700 1 -- 172.21.15.112:0/1854780174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c101a90 msgr2=0x7fbc4c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.702+0000 7fbc53229700 1 --2- 172.21.15.112:0/1854780174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c101a90 0x7fbc4c101f10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009990 tx=0x7fbc38009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.702+0000 7fbc53229700 1 -- 172.21.15.112:0/1854780174 shutdown_connections 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.702+0000 7fbc53229700 1 --2- 172.21.15.112:0/1854780174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc4c102450 0x7fbc4c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.702+0000 7fbc53229700 1 --2- 172.21.15.112:0/1854780174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc4c109520 0x7fbc4c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.702+0000 7fbc53229700 1 --2- 172.21.15.112:0/1854780174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c101a90 0x7fbc4c101f10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.702+0000 7fbc53229700 1 -- 172.21.15.112:0/1854780174 >> 172.21.15.112:0/1854780174 conn(0x7fbc4c078a10 msgr2=0x7fbc4c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.703+0000 7fbc53229700 1 -- 172.21.15.112:0/1854780174 shutdown_connections 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.703+0000 7fbc53229700 1 -- 172.21.15.112:0/1854780174 wait complete. 2024-05-10T10:43:45.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.703+0000 7fbc53229700 1 Processor -- start 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.703+0000 7fbc53229700 1 -- start start 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.703+0000 7fbc53229700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc4c101a90 0x7fbc4c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.703+0000 7fbc53229700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc4c102450 0x7fbc4c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.703+0000 7fbc53229700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c109520 0x7fbc4c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc53229700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c113550 con 0x7fbc4c109520 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc4c101a90 0x7fbc4c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc4c102450 0x7fbc4c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc53229700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1133d0 con 0x7fbc4c102450 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc4c101a90 0x7fbc4c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46210/0 (socket says 172.21.15.112:46210) 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1136d0 con 0x7fbc4c101a90 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc50fc5700 1 -- 172.21.15.112:0/998621027 learned_addr learned my addr 172.21.15.112:0/998621027 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:45.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc517c6700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c109520 0x7fbc4c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc4bfff700 1 -- 172.21.15.112:0/998621027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc4c101a90 msgr2=0x7fbc4c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc4c101a90 0x7fbc4c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc4bfff700 1 -- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c109520 msgr2=0x7fbc4c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c109520 0x7fbc4c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc4bfff700 1 -- 172.21.15.112:0/998621027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c19a0b0 con 0x7fbc4c102450 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc4c102450 0x7fbc4c19fcd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc380098e0 tx=0x7fbc380047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.704+0000 7fbc49ffb700 1 -- 172.21.15.112:0/998621027 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38026070 con 0x7fbc4c102450 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.705+0000 7fbc49ffb700 1 -- 172.21.15.112:0/998621027 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc38021410 con 0x7fbc4c102450 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.705+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c19a340 con 0x7fbc4c102450 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.705+0000 7fbc49ffb700 1 -- 172.21.15.112:0/998621027 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38017650 con 0x7fbc4c102450 2024-05-10T10:43:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.705+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c1a6660 con 0x7fbc4c102450 2024-05-10T10:43:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.708+0000 7fbc49ffb700 1 -- 172.21.15.112:0/998621027 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbc38029070 con 0x7fbc4c102450 2024-05-10T10:43:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.709+0000 7fbc49ffb700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbc3c0b2880 0x7fbc3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.709+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbc3c0b2880 0x7fbc3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.709+0000 7fbc49ffb700 1 -- 172.21.15.112:0/998621027 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbc380e32f0 con 0x7fbc4c102450 2024-05-10T10:43:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.709+0000 7fbc5dabcb80 1 Processor -- start 2024-05-10T10:43:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.709+0000 7fbc5dabcb80 1 -- start start 2024-05-10T10:43:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851f6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e92f0 0x55ec851e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec8513b860 con 0x55ec851e92f0 2024-05-10T10:43:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ec850da7c0 con 0x55ec851f6240 2024-05-10T10:43:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ec85137820 con 0x55ec851e9cb0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbc3c0b2880 0x7fbc3c0b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c19aa90 tx=0x7fbc40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46222/0 (socket says 172.21.15.112:46222) 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 -- 172.21.15.112:0/1107585224 learned_addr learned my addr 172.21.15.112:0/1107585224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e92f0 0x55ec851e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 -- 172.21.15.112:0/1107585224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 msgr2=0x55ec851f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 -- 172.21.15.112:0/1107585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e92f0 msgr2=0x55ec851e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e92f0 0x55ec851e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 -- 172.21.15.112:0/1107585224 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec850888d0 con 0x55ec851e9cb0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e92f0 0x55ec851e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.710+0000 7fbc517c6700 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fe150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc44011680 tx=0x7fbc44009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=554015b6efa449f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.711+0000 7fbc36ffd700 1 -- 172.21.15.112:0/1107585224 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44019030 con 0x55ec851e9cb0 2024-05-10T10:43:45.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.711+0000 7fbc36ffd700 1 -- 172.21.15.112:0/1107585224 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc440092e0 con 0x55ec851e9cb0 2024-05-10T10:43:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.711+0000 7fbc36ffd700 1 -- 172.21.15.112:0/1107585224 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44007500 con 0x55ec851e9cb0 2024-05-10T10:43:45.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.711+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/1107585224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 msgr2=0x55ec851fe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.711+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fe150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc44011680 tx=0x7fbc44009920 comp rx=0 tx=0).stop 2024-05-10T10:43:45.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/1107585224 shutdown_connections 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fe150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/1107585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e92f0 0x55ec851e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/1107585224 >> 172.21.15.112:0/1107585224 conn(0x55ec85115100 msgr2=0x55ec851e3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/1107585224 shutdown_connections 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/1107585224 wait complete. 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 -- start start 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.712+0000 7fbc5dabcb80 1 Processor -- start 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec851fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec851fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec851fe6f0 con 0x55ec851e9cb0 2024-05-10T10:43:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ec850a1c30 con 0x55ec851f6240 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ec851fe870 con 0x55ec851e92f0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc5dabcb80 10 client.0 _ll_register_callbacks cb 0x55ec850d3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec851fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc517c6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec851fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46236/0 (socket says 172.21.15.112:46236) 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 -- 172.21.15.112:0/2659723249 learned_addr learned my addr 172.21.15.112:0/2659723249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec851fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 -- 172.21.15.112:0/2659723249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 msgr2=0x55ec851fb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851fb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 -- 172.21.15.112:0/2659723249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 msgr2=0x55ec851fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec851fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 -- 172.21.15.112:0/2659723249 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec850d7690 con 0x55ec851e92f0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec851fdcf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc4001fa90 tx=0x7fbc400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2935caf4f068b764 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:43:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.713+0000 7fbc2bfff700 1 -- 172.21.15.112:0/2659723249 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4001da10 con 0x55ec851e92f0 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc2bfff700 1 -- 172.21.15.112:0/2659723249 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbc4001dbb0 con 0x55ec851e92f0 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc440092e0 con 0x55ec851e92f0 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc2bfff700 1 -- 172.21.15.112:0/2659723249 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbc4001f320 con 0x55ec851e92f0 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 10 client.25717 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ec85208ef0 con 0x55ec851e92f0 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc2bfff700 1 -- 172.21.15.112:0/2659723249 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc40031070 con 0x55ec851e92f0 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc2bfff700 1 -- 172.21.15.112:0/2659723249 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fbc40026020 con 0x55ec851e92f0 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 10 client.25717 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 10 client.25717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 20 client.25717 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec8520b530 0x55ec8520d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 4 client.25717 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:45.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.714+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec851b5650 con 0x55ec8520b530 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.715+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec8520b530 0x55ec8520d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.715+0000 7fbc2bfff700 10 client.25717 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.715+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec8520b530 0x55ec8520d990 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4b607ef42b154d2 server_cookie=1e47a1cbe88089ae in_seq=0 out_seq=0 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc2bfff700 1 -- 172.21.15.112:0/2659723249 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec851b5650 con 0x55ec8520b530 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc2bfff700 10 client.25717 handle_command_reply: tid=0 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 client.25717 shutdown 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 msgr2=0x55ec851fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec851fdcf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc4001fa90 tx=0x7fbc400047c0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 shutdown_connections 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec8520b530 0x55ec8520d990 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc2bfff700 0 client.25717 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851fb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec851fdcf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/2659723249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec851fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.716+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 >> 172.21.15.112:0/2659723249 conn(0x55ec85115100 msgr2=0x55ec851de720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.717+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 shutdown_connections 2024-05-10T10:43:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.717+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/2659723249 wait complete. 2024-05-10T10:43:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.717+0000 7fbc5dabcb80 20 client.25717 trim_cache size 0 max 16384 2024-05-10T10:43:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 Processor -- start 2024-05-10T10:43:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 -- start start 2024-05-10T10:43:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851e92f0 0x55ec85207780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851f6240 0x55ec851de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1acf00 con 0x55ec851f6240 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1aebc0 con 0x55ec851e92f0 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1ad5d0 con 0x55ec851e9cb0 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc517c6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851f6240 0x55ec851de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851e92f0 0x55ec85207780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc517c6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851f6240 0x55ec851de5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38618/0 (socket says 172.21.15.112:38618) 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46250/0 (socket says 172.21.15.112:46250) 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc517c6700 1 -- 172.21.15.112:0/3146971958 learned_addr learned my addr 172.21.15.112:0/3146971958 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc50fc5700 1 -- 172.21.15.112:0/3146971958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 msgr2=0x55ec851fa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc50fc5700 1 -- 172.21.15.112:0/3146971958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851f6240 msgr2=0x55ec851de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851f6240 0x55ec851de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.734+0000 7fbc50fc5700 1 -- 172.21.15.112:0/3146971958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec851e56b0 con 0x55ec851e92f0 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.735+0000 7fbc517c6700 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851f6240 0x55ec851de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.735+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851e92f0 0x55ec85207780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009ba0 tx=0x7fbc40009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a60583e2d3f037 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.735+0000 7fbc367fc700 1 -- 172.21.15.112:0/3146971958 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4001ecd0 con 0x55ec851e92f0 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.735+0000 7fbc367fc700 1 -- 172.21.15.112:0/3146971958 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc4003a070 con 0x55ec851e92f0 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.735+0000 7fbc367fc700 1 -- 172.21.15.112:0/3146971958 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc400356c0 con 0x55ec851e92f0 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3146971958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851e92f0 msgr2=0x55ec85207780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851e92f0 0x55ec85207780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009ba0 tx=0x7fbc40009f00 comp rx=0 tx=0).stop 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3146971958 shutdown_connections 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851f6240 0x55ec851de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851e92f0 0x55ec85207780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3146971958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e9cb0 0x55ec851fa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3146971958 >> 172.21.15.112:0/3146971958 conn(0x55ec85115100 msgr2=0x55ec851dde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3146971958 shutdown_connections 2024-05-10T10:43:45.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3146971958 wait complete. 2024-05-10T10:43:45.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 -- start start 2024-05-10T10:43:45.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.736+0000 7fbc5dabcb80 1 Processor -- start 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec8526e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec8526e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc5dabcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1ae430 con 0x55ec851e9cb0 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1add60 con 0x55ec851f6240 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc5dabcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1aaea0 con 0x55ec851e92f0 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec8526e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec8526e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec8526e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46262/0 (socket says 172.21.15.112:46262) 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 -- 172.21.15.112:0/3666337323 learned_addr learned my addr 172.21.15.112:0/3666337323 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc5dabcb80 10 client.0 _ll_register_callbacks cb 0x55ec850d3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc517c6700 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 -- 172.21.15.112:0/3666337323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 msgr2=0x55ec851feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 -- 172.21.15.112:0/3666337323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 msgr2=0x55ec8526e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec8526e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc50fc5700 1 -- 172.21.15.112:0/3666337323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec851ff5c0 con 0x55ec851e92f0 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.737+0000 7fbc517c6700 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc50fc5700 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec8526e100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc40000c00 tx=0x7fbc4001efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5259b0b1b3e1f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc2bfff700 1 -- 172.21.15.112:0/3666337323 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40042070 con 0x55ec851e92f0 2024-05-10T10:43:45.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc2bfff700 1 -- 172.21.15.112:0/3666337323 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbc4001db30 con 0x55ec851e92f0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc4003a070 con 0x55ec851e92f0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc2bfff700 1 -- 172.21.15.112:0/3666337323 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbc4004b050 con 0x55ec851e92f0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc5dabcb80 10 client.25723 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ec85275880 con 0x55ec851e92f0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.738+0000 7fbc2bfff700 1 -- 172.21.15.112:0/3666337323 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc4003a070 con 0x55ec851e92f0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.739+0000 7fbc2bfff700 1 -- 172.21.15.112:0/3666337323 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fbc40045020 con 0x55ec851e92f0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.739+0000 7fbc5dabcb80 10 client.25723 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.739+0000 7fbc5dabcb80 10 client.25723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.739+0000 7fbc5dabcb80 20 client.25723 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.739+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec85275e60 0x55ec85278250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.739+0000 7fbc5dabcb80 4 client.25723 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.739+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec85149240 con 0x55ec85275e60 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.740+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec85275e60 0x55ec85278250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.740+0000 7fbc4bfff700 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec85275e60 0x55ec85278250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f6fb820c1858508 server_cookie=3b7e38fd49a6f21a in_seq=0 out_seq=0 2024-05-10T10:43:45.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.740+0000 7fbc2bfff700 10 client.25723 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.740+0000 7fbc2bfff700 1 -- 172.21.15.112:0/3666337323 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec85149240 con 0x55ec85275e60 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.740+0000 7fbc2bfff700 10 client.25723 handle_command_reply: tid=0 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.740+0000 7fbc5dabcb80 1 client.25723 shutdown 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 msgr2=0x55ec8526e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec8526e100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc40000c00 tx=0x7fbc4001efa0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 shutdown_connections 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ec85275e60 0x55ec85278250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec851f6240 0x55ec851feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ec851e92f0 0x55ec8526e100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 --2- 172.21.15.112:0/3666337323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec851e9cb0 0x55ec8526e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 >> 172.21.15.112:0/3666337323 conn(0x55ec85115100 msgr2=0x55ec851e82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc2bfff700 0 client.25723 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 shutdown_connections 2024-05-10T10:43:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 1 -- 172.21.15.112:0/3666337323 wait complete. 2024-05-10T10:43:45.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.741+0000 7fbc5dabcb80 20 client.25723 trim_cache size 0 max 16384 2024-05-10T10:43:45.747 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:43:45.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbc3c0b2880 msgr2=0x7fbc3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbc3c0b2880 0x7fbc3c0b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c19aa90 tx=0x7fbc40009260 comp rx=0 tx=0).stop 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc4c102450 msgr2=0x7fbc4c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc4c102450 0x7fbc4c19fcd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc380098e0 tx=0x7fbc380047b0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 shutdown_connections 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc4c109520 0x7fbc4c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbc3c0b2880 0x7fbc3c0b4d40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc4c101a90 0x7fbc4c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 --2- 172.21.15.112:0/998621027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc4c102450 0x7fbc4c19fcd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 >> 172.21.15.112:0/998621027 conn(0x7fbc4c078a10 msgr2=0x7fbc4c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 shutdown_connections 2024-05-10T10:43:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:45.742+0000 7fbc53229700 1 -- 172.21.15.112:0/998621027 wait complete. 2024-05-10T10:43:46.639 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:43:46.909 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:47 smithi195 ceph-mon[33773]: pgmap v536: 129 pgs: 129 active+clean; 39 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 417 op/s 2024-05-10T10:43:47.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:47 smithi112 ceph-mon[30100]: pgmap v536: 129 pgs: 129 active+clean; 39 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 417 op/s 2024-05-10T10:43:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:47 smithi148 ceph-mon[33568]: pgmap v536: 129 pgs: 129 active+clean; 39 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 417 op/s 2024-05-10T10:43:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:49 smithi195 ceph-mon[33773]: pgmap v537: 129 pgs: 129 active+clean; 39 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-10T10:43:49.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:49 smithi112 ceph-mon[30100]: pgmap v537: 129 pgs: 129 active+clean; 39 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-10T10:43:49.345 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:49 smithi148 ceph-mon[33568]: pgmap v537: 129 pgs: 129 active+clean; 39 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 -- 172.21.15.112:0/3422543436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 msgr2=0x7fa234102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 --2- 172.21.15.112:0/3422543436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234102270 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa224009900 tx=0x7fa22402a410 comp rx=0 tx=0).stop 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 -- 172.21.15.112:0/3422543436 shutdown_connections 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 --2- 172.21.15.112:0/3422543436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa234103520 0x7fa234109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 --2- 172.21.15.112:0/3422543436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa234102b60 0x7fa234102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 --2- 172.21.15.112:0/3422543436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234102270 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 -- 172.21.15.112:0/3422543436 >> 172.21.15.112:0/3422543436 conn(0x7fa2340fc620 msgr2=0x7fa2340fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 -- 172.21.15.112:0/3422543436 shutdown_connections 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.170+0000 7fa2398c4640 1 -- 172.21.15.112:0/3422543436 wait complete. 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 Processor -- start 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 -- start start 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa234102b60 0x7fa234199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa234103520 0x7fa2341a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410e470 con 0x7fa234101e50 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410e2f0 con 0x7fa234102b60 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2398c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410e5f0 con 0x7fa234103520 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa232ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa232ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38644/0 (socket says 172.21.15.112:38644) 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa232ffd640 1 -- 172.21.15.112:0/1279021786 learned_addr learned my addr 172.21.15.112:0/1279021786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa22bfff640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa234102b60 0x7fa234199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa2337fe640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa234103520 0x7fa2341a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa232ffd640 1 -- 172.21.15.112:0/1279021786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa234103520 msgr2=0x7fa2341a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa232ffd640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa234103520 0x7fa2341a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.171+0000 7fa232ffd640 1 -- 172.21.15.112:0/1279021786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa234102b60 msgr2=0x7fa234199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa232ffd640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa234102b60 0x7fa234199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa232ffd640 1 -- 172.21.15.112:0/1279021786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2341a0750 con 0x7fa234101e50 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa22bfff640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa234102b60 0x7fa234199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa232ffd640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234198b30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa224005ab0 tx=0x7fa2240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa2388c2640 1 -- 172.21.15.112:0/1279021786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22403f030 con 0x7fa234101e50 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2341a09e0 con 0x7fa234101e50 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa2388c2640 1 -- 172.21.15.112:0/1279021786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa224032e10 con 0x7fa234101e50 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2341a0f20 con 0x7fa234101e50 2024-05-10T10:43:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.172+0000 7fa2388c2640 1 -- 172.21.15.112:0/1279021786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224035070 con 0x7fa234101e50 2024-05-10T10:43:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa2388c2640 1 -- 172.21.15.112:0/1279021786 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa224042030 con 0x7fa234101e50 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 Processor -- start 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 -- start start 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb629025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb629076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb6290a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb628625d0 con 0x55fb62902bb0 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fb6281b890 con 0x55fb62907d20 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa23c5da740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb6285e590 con 0x55fb629021c0 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa232ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb629076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa2337fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb629025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa2337fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb629025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46298/0 (socket says 172.21.15.112:46298) 2024-05-10T10:43:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa2337fe640 1 -- 172.21.15.112:0/2074971414 learned_addr learned my addr 172.21.15.112:0/2074971414 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa232ffd640 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb629076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38660/0 (socket says 172.21.15.112:38660) 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.175+0000 7fa22bfff640 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb6290a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2337fe640 1 -- 172.21.15.112:0/2074971414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 msgr2=0x55fb6290a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2337fe640 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb6290a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2337fe640 1 -- 172.21.15.112:0/2074971414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 msgr2=0x55fb629076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2337fe640 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb629076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2337fe640 1 -- 172.21.15.112:0/2074971414 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb62574210 con 0x55fb629021c0 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2388c2640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa2200b2920 0x7fa2200b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa232ffd640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa2200b2920 0x7fa2200b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2337fe640 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb629025e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009fd0 tx=0x7fa22c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25eccbf52309622 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa2388c2640 1 -- 172.21.15.112:0/1279021786 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa224102950 con 0x7fa234101e50 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa229ffb640 1 -- 172.21.15.112:0/2074971414 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c004d20 con 0x55fb629021c0 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa229ffb640 1 -- 172.21.15.112:0/2074971414 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c004050 con 0x55fb629021c0 2024-05-10T10:43:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.176+0000 7fa229ffb640 1 -- 172.21.15.112:0/2074971414 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c005260 con 0x55fb629021c0 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa232ffd640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa2200b2920 0x7fa2200b4de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa224100430 tx=0x7fa224108550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 -- 172.21.15.112:0/2074971414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 msgr2=0x55fb629025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb629025e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009fd0 tx=0x7fa22c0118d0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 -- 172.21.15.112:0/2074971414 shutdown_connections 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb6290a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb629076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 --2- 172.21.15.112:0/2074971414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb629025e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 -- 172.21.15.112:0/2074971414 >> 172.21.15.112:0/2074971414 conn(0x55fb628f9420 msgr2=0x55fb628f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 -- 172.21.15.112:0/2074971414 shutdown_connections 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 -- 172.21.15.112:0/2074971414 wait complete. 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.177+0000 7fa23c5da740 1 -- start start 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 1 Processor -- start 2024-05-10T10:43:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb6290c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb6290c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb62913920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb6290acd0 con 0x55fb62902bb0 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fb6281f370 con 0x55fb62907d20 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb6290ae50 con 0x55fb629021c0 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb6290c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb6290c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38664/0 (socket says 172.21.15.112:38664) 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa23c5da740 10 client.0 _ll_register_callbacks cb 0x55fb62573a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 -- 172.21.15.112:0/617018758 learned_addr learned my addr 172.21.15.112:0/617018758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa2337fe640 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb6290c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa2337fe640 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb62913920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 -- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 msgr2=0x55fb6290c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb6290c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 -- 172.21.15.112:0/617018758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 msgr2=0x55fb62913920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb62913920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.178+0000 7fa22bfff640 1 -- 172.21.15.112:0/617018758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb62640620 con 0x55fb62902bb0 2024-05-10T10:43:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.179+0000 7fa22bfff640 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb6290c8e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa21c012d00 tx=0x7fa21c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d77f765887f4ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.179+0000 7fa212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa212ffd640 1 -- 172.21.15.112:0/617018758 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c012110 con 0x55fb62902bb0 2024-05-10T10:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa212ffd640 1 -- 172.21.15.112:0/617018758 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa21c004050 con 0x55fb62902bb0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22c004050 con 0x55fb62902bb0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa212ffd640 1 -- 172.21.15.112:0/617018758 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa21c025050 con 0x55fb62902bb0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 10 client.16323 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fb629169d0 con 0x55fb62902bb0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa212ffd640 1 -- 172.21.15.112:0/617018758 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c004050 con 0x55fb62902bb0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa212ffd640 1 -- 172.21.15.112:0/617018758 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fa21c003ba0 con 0x55fb62902bb0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 10 client.16323 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 20 client.16323 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb629181d0 0x55fb6291a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 4 client.16323 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb628701e0 con 0x55fb629181d0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.181+0000 7fa2337fe640 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb629181d0 0x55fb6291a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.182+0000 7fa212ffd640 10 client.16323 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.182+0000 7fa2337fe640 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb629181d0 0x55fb6291a610 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d48a24fc228f6b server_cookie=d4b01b9619092603 in_seq=0 out_seq=0 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa212ffd640 1 -- 172.21.15.112:0/617018758 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb628701e0 con 0x55fb629181d0 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa212ffd640 10 client.16323 handle_command_reply: tid=0 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 client.16323 shutdown 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 msgr2=0x55fb6290c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb6290c8e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa21c012d00 tx=0x7fa21c00f790 comp rx=0 tx=0).stop 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 shutdown_connections 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb629181d0 0x55fb6291a610 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62907d20 0x55fb62913920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb62902bb0 0x55fb6290c8e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa212ffd640 0 client.16323 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 --2- 172.21.15.112:0/617018758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb629021c0 0x55fb6290c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 >> 172.21.15.112:0/617018758 conn(0x55fb628f9420 msgr2=0x55fb628f8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 shutdown_connections 2024-05-10T10:43:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 1 -- 172.21.15.112:0/617018758 wait complete. 2024-05-10T10:43:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.183+0000 7fa23c5da740 20 client.16323 trim_cache size 0 max 16384 2024-05-10T10:43:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 Processor -- start 2024-05-10T10:43:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 -- start start 2024-05-10T10:43:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb628fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb629138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 0x55fb628f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341a9180 con 0x55fb629021c0 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341aae40 con 0x55fb62902bb0 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa23c5da740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341a9850 con 0x55fb62907d20 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa2337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb628fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa2337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb628fc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38680/0 (socket says 172.21.15.112:38680) 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.198+0000 7fa2337fe640 1 -- 172.21.15.112:0/321546980 learned_addr learned my addr 172.21.15.112:0/321546980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa22bfff640 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb629138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2337fe640 1 -- 172.21.15.112:0/321546980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 msgr2=0x55fb628f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2337fe640 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 0x55fb628f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2337fe640 1 -- 172.21.15.112:0/321546980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 msgr2=0x55fb629138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2337fe640 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb629138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2337fe640 1 -- 172.21.15.112:0/321546980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb628f9250 con 0x55fb629021c0 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa22bfff640 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb629138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2337fe640 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb628fc810 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa22c011dd0 tx=0x7fa22c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe0c3c93b291719 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2297fa640 1 -- 172.21.15.112:0/321546980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c004640 con 0x55fb629021c0 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2297fa640 1 -- 172.21.15.112:0/321546980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c003a30 con 0x55fb629021c0 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.199+0000 7fa2297fa640 1 -- 172.21.15.112:0/321546980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00f720 con 0x55fb629021c0 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 -- 172.21.15.112:0/321546980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 msgr2=0x55fb628fc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb628fc810 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa22c011dd0 tx=0x7fa22c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 -- 172.21.15.112:0/321546980 shutdown_connections 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 0x55fb628f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb629138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 --2- 172.21.15.112:0/321546980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb628fc810 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 -- 172.21.15.112:0/321546980 >> 172.21.15.112:0/321546980 conn(0x55fb628f9420 msgr2=0x55fb628fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 -- 172.21.15.112:0/321546980 shutdown_connections 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 -- 172.21.15.112:0/321546980 wait complete. 2024-05-10T10:43:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 -- start start 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 Processor -- start 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.200+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb62911350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb62911890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 0x55fb629128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341aa6b0 con 0x55fb629021c0 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341a9fe0 con 0x55fb62902bb0 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341a7160 con 0x55fb62907d20 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 10 client.0 _ll_register_callbacks cb 0x7fa21c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb62911350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb62911350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38686/0 (socket says 172.21.15.112:38686) 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 -- 172.21.15.112:0/1434808101 learned_addr learned my addr 172.21.15.112:0/1434808101 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 0x55fb629128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa22bfff640 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb62911890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 -- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 msgr2=0x55fb629128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 0x55fb629128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 -- 172.21.15.112:0/1434808101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 msgr2=0x55fb62911890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb62911890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 -- 172.21.15.112:0/1434808101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb62912e90 con 0x55fb629021c0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa2337fe640 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb62911350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa22c011e90 tx=0x7fa22c011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3a6a8e576c2583 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa212ffd640 1 -- 172.21.15.112:0/1434808101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c036040 con 0x55fb629021c0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa212ffd640 1 -- 172.21.15.112:0/1434808101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa22c004cc0 con 0x55fb629021c0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22c003a30 con 0x55fb629021c0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa212ffd640 1 -- 172.21.15.112:0/1434808101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa22c003f00 con 0x55fb629021c0 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 10 client.16329 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.201+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fb62982dd0 con 0x55fb629021c0 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa212ffd640 1 -- 172.21.15.112:0/1434808101 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22c003a30 con 0x55fb629021c0 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa212ffd640 1 -- 172.21.15.112:0/1434808101 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fa22c03c070 con 0x55fb629021c0 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa23c5da740 10 client.16329 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa23c5da740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa23c5da740 20 client.16329 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa23c5da740 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb62983640 0x55fb62985a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa23c5da740 4 client.16329 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb6286a050 con 0x55fb62983640 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa22bfff640 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb62983640 0x55fb62985a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa212ffd640 10 client.16329 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.202+0000 7fa22bfff640 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb62983640 0x55fb62985a60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f01abb6715ab9c5 server_cookie=cb24fcb12602ff07 in_seq=0 out_seq=0 2024-05-10T10:43:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.250+0000 7fa212ffd640 1 -- 172.21.15.112:0/1434808101 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb6286a050 con 0x55fb62983640 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.250+0000 7fa212ffd640 10 client.16329 handle_command_reply: tid=0 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.250+0000 7fa23c5da740 1 client.16329 shutdown 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 msgr2=0x55fb62911350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb62911350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa22c011e90 tx=0x7fa22c011e00 comp rx=0 tx=0).stop 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 shutdown_connections 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb62907d20 0x55fb629128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55fb62983640 0x55fb62985a60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb62902bb0 0x55fb62911890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa212ffd640 0 client.16329 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 --2- 172.21.15.112:0/1434808101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb629021c0 0x55fb62911350 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 >> 172.21.15.112:0/1434808101 conn(0x55fb628f9420 msgr2=0x55fb6290b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 shutdown_connections 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 1 -- 172.21.15.112:0/1434808101 wait complete. 2024-05-10T10:43:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa23c5da740 20 client.16329 trim_cache size 0 max 16384 2024-05-10T10:43:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:43:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:43:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:43:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa2200b2920 msgr2=0x7fa2200b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa2200b2920 0x7fa2200b4de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa224100430 tx=0x7fa224108550 comp rx=0 tx=0).stop 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 msgr2=0x7fa234198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234198b30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa224005ab0 tx=0x7fa2240042f0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 shutdown_connections 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa2200b2920 0x7fa2200b4de0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa234103520 0x7fa2341a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa234102b60 0x7fa234199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 --2- 172.21.15.112:0/1279021786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa234101e50 0x7fa234198b30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.251+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 >> 172.21.15.112:0/1279021786 conn(0x7fa2340fc620 msgr2=0x7fa234105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.252+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 shutdown_connections 2024-05-10T10:43:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:43:51.252+0000 7fa2398c4640 1 -- 172.21.15.112:0/1279021786 wait complete. 2024-05-10T10:43:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:51 smithi195 ceph-mon[33773]: pgmap v538: 129 pgs: 129 active+clean; 41 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 604 op/s 2024-05-10T10:43:51.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:51 smithi112 ceph-mon[30100]: pgmap v538: 129 pgs: 129 active+clean; 41 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 604 op/s 2024-05-10T10:43:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:51 smithi148 ceph-mon[33568]: pgmap v538: 129 pgs: 129 active+clean; 41 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 604 op/s 2024-05-10T10:43:51.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.839+0000 7f7bb992c700 1 -- 172.21.15.112:0/3373192771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4101a90 msgr2=0x7f7bb4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.839+0000 7f7bb992c700 1 --2- 172.21.15.112:0/3373192771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4101a90 0x7f7bb4101f10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009990 tx=0x7f7ba8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.839+0000 7f7bb992c700 1 -- 172.21.15.112:0/3373192771 shutdown_connections 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.839+0000 7f7bb992c700 1 --2- 172.21.15.112:0/3373192771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bb4102450 0x7f7bb4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.839+0000 7f7bb992c700 1 --2- 172.21.15.112:0/3373192771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb4109520 0x7f7bb4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.839+0000 7f7bb992c700 1 --2- 172.21.15.112:0/3373192771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4101a90 0x7f7bb4101f10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.839+0000 7f7bb992c700 1 -- 172.21.15.112:0/3373192771 >> 172.21.15.112:0/3373192771 conn(0x7f7bb4078a10 msgr2=0x7f7bb40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.840+0000 7f7bb992c700 1 -- 172.21.15.112:0/3373192771 shutdown_connections 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.840+0000 7f7bb992c700 1 -- 172.21.15.112:0/3373192771 wait complete. 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.840+0000 7f7bb992c700 1 Processor -- start 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb992c700 1 -- start start 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb992c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bb4101a90 0x7f7bb41a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb992c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4102450 0x7f7bb41a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb992c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb4109520 0x7f7bb419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb992c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4113500 con 0x7f7bb4102450 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb992c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4113380 con 0x7f7bb4109520 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb992c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4113680 con 0x7f7bb4101a90 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4102450 0x7f7bb41a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4102450 0x7f7bb41a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38704/0 (socket says 172.21.15.112:38704) 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2675235048 learned_addr learned my addr 172.21.15.112:0/2675235048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bb4101a90 0x7f7bb41a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb37fe700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb4109520 0x7f7bb419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2675235048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bb4101a90 msgr2=0x7f7bb41a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bb4101a90 0x7f7bb41a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2675235048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb4109520 msgr2=0x7f7bb419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb4109520 0x7f7bb419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2675235048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb419c1c0 con 0x7f7bb4102450 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.841+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4102450 0x7f7bb41a1de0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8006010 tx=0x7f7ba80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.842+0000 7f7bb892a700 1 -- 172.21.15.112:0/2675235048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8027070 con 0x7f7bb4102450 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.842+0000 7f7bb892a700 1 -- 172.21.15.112:0/2675235048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ba8022470 con 0x7f7bb4102450 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.842+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb419c450 con 0x7f7bb4102450 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.842+0000 7f7bb892a700 1 -- 172.21.15.112:0/2675235048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8017650 con 0x7f7bb4102450 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.842+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb41a8770 con 0x7f7bb4102450 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.844+0000 7f7bb892a700 1 -- 172.21.15.112:0/2675235048 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7ba8009d10 con 0x7f7bb4102450 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.845+0000 7f7bb892a700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b9c0b2900 0x7f7b9c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.845+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b9c0b2900 0x7f7b9c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.845+0000 7f7bb892a700 1 -- 172.21.15.112:0/2675235048 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7ba80e2fb0 con 0x7f7bb4102450 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b9c0b2900 0x7f7b9c0b4dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7bb419cba0 tx=0x7f7ba4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 Processor -- start 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 -- start start 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cfc80 0x559b2a4d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b2a41d860 con 0x559b2a4cfc80 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559b2a3bc7c0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559b2a419820 con 0x559b2a4cb4d0 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bb37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cfc80 0x559b2a4d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.846+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cfc80 0x559b2a4d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38708/0 (socket says 172.21.15.112:38708) 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb37fe700 1 -- 172.21.15.112:0/1113542511 learned_addr learned my addr 172.21.15.112:0/1113542511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb2ffd700 1 -- 172.21.15.112:0/1113542511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 msgr2=0x559b2a4cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb2ffd700 1 -- 172.21.15.112:0/1113542511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cfc80 msgr2=0x559b2a4d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cfc80 0x559b2a4d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb2ffd700 1 -- 172.21.15.112:0/1113542511 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2a36a8d0 con 0x559b2a4cb4d0 2024-05-10T10:43:51.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:51.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4cb8b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4011c70 tx=0x7f7ba4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584b6a451d2f681c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7b99ffb700 1 -- 172.21.15.112:0/1113542511 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4022070 con 0x559b2a4cb4d0 2024-05-10T10:43:51.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7b99ffb700 1 -- 172.21.15.112:0/1113542511 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ba40044e0 con 0x559b2a4cb4d0 2024-05-10T10:43:51.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.847+0000 7f7b99ffb700 1 -- 172.21.15.112:0/1113542511 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40038e0 con 0x559b2a4cb4d0 2024-05-10T10:43:51.899 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1113542511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 msgr2=0x559b2a4cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4cb8b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4011c70 tx=0x7f7ba4014000 comp rx=0 tx=0).stop 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1113542511 shutdown_connections 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cfc80 0x559b2a4d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4cb8b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1113542511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1113542511 >> 172.21.15.112:0/1113542511 conn(0x559b2a3f7100 msgr2=0x559b2a4c5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1113542511 shutdown_connections 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.848+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1113542511 wait complete. 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.849+0000 7f7bc41bfb80 1 -- start start 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.849+0000 7f7bc41bfb80 1 Processor -- start 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.849+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.849+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4d4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b2a4d7e90 con 0x559b2a4cbdf0 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559b2a383c30 con 0x559b2a4cfc80 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559b2a4d8010 con 0x559b2a4cb4d0 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bc41bfb80 10 client.0 _ll_register_callbacks cb 0x559b2a3b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38724/0 (socket says 172.21.15.112:38724) 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2707860029 learned_addr learned my addr 172.21.15.112:0/2707860029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb37fe700 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4d4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2707860029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 msgr2=0x559b2a4d70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4d70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2707860029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 msgr2=0x559b2a4d4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4d4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2707860029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2a3b9690 con 0x559b2a4cbdf0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d4450 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80e0720 tx=0x7f7ba8022c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09d518a6e36e86c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.850+0000 7f7b92ffd700 1 -- 172.21.15.112:0/2707860029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80aa9d0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7b92ffd700 1 -- 172.21.15.112:0/2707860029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7ba80e0df0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba40044e0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7b92ffd700 1 -- 172.21.15.112:0/2707860029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ba80aa2d0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 10 client.16341 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559b2a4e26d0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7b92ffd700 1 -- 172.21.15.112:0/2707860029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ba40044e0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7b92ffd700 1 -- 172.21.15.112:0/2707860029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f7ba80b2ca0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 10 client.16341 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 20 client.16341 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a4e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 4 client.16341 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b2a497650 con 0x559b2a4e4d70 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.851+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a4e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.852+0000 7f7b92ffd700 10 client.16341 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.852+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a4e71d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886ab7268a93fc1a server_cookie=8bbdbcfe7c9137f3 in_seq=0 out_seq=0 2024-05-10T10:43:51.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.852+0000 7f7b92ffd700 1 -- 172.21.15.112:0/2707860029 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b2a497650 con 0x559b2a4e4d70 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.852+0000 7f7b92ffd700 10 client.16341 handle_command_reply: tid=0 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.852+0000 7f7bc41bfb80 1 client.16341 shutdown 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 msgr2=0x559b2a4d4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d4450 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80e0720 tx=0x7f7ba8022c00 comp rx=0 tx=0).stop 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 shutdown_connections 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a4e71d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7b92ffd700 0 client.16341 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4d70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2707860029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d4450 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 >> 172.21.15.112:0/2707860029 conn(0x559b2a3f7100 msgr2=0x559b2a4c0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 shutdown_connections 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2707860029 wait complete. 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.853+0000 7f7bc41bfb80 20 client.16341 trim_cache size 0 max 16384 2024-05-10T10:43:51.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 Processor -- start 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 -- start start 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b1260 con 0x559b2a4cbdf0 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b2f20 con 0x559b2a4cfc80 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b1930 con 0x559b2a4cb4d0 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bb27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4ecaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38738/0 (socket says 172.21.15.112:38738) 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2347462711 learned_addr learned my addr 172.21.15.112:0/2347462711 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.870+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7bb37fe700 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2347462711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 msgr2=0x559b2a4e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4e0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2347462711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 msgr2=0x559b2a4ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7bb27fc700 1 -- 172.21.15.112:0/2347462711 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2a4d7d70 con 0x559b2a4cbdf0 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4ecaa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80a6350 tx=0x7f7ba8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ce15aa2ca8ec3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7b997fa700 1 -- 172.21.15.112:0/2347462711 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80ae680 con 0x559b2a4cbdf0 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7b997fa700 1 -- 172.21.15.112:0/2347462711 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ba80b30a0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.871+0000 7f7b997fa700 1 -- 172.21.15.112:0/2347462711 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80a9530 con 0x559b2a4cbdf0 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2347462711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 msgr2=0x559b2a4ecaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4ecaa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80a6350 tx=0x7f7ba8009710 comp rx=0 tx=0).stop 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2347462711 shutdown_connections 2024-05-10T10:43:51.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cfc80 0x559b2a4ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cb4d0 0x559b2a4e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/2347462711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cbdf0 0x559b2a4ecaa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2347462711 >> 172.21.15.112:0/2347462711 conn(0x559b2a3f7100 msgr2=0x559b2a4d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2347462711 shutdown_connections 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/2347462711 wait complete. 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 -- start start 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.872+0000 7f7bc41bfb80 1 Processor -- start 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cb4d0 0x559b2a4dd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bc41bfb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cfc80 0x559b2a4d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b2790 con 0x559b2a4cb4d0 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b20c0 con 0x559b2a4cbdf0 2024-05-10T10:43:51.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bc41bfb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41af200 con 0x559b2a4cfc80 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bc41bfb80 10 client.0 _ll_register_callbacks cb 0x559b2a3b5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cb4d0 0x559b2a4dd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cfc80 0x559b2a4d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cfc80 0x559b2a4d8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46398/0 (socket says 172.21.15.112:46398) 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cb4d0 0x559b2a4dd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38746/0 (socket says 172.21.15.112:38746) 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb37fe700 1 -- 172.21.15.112:0/1234310315 learned_addr learned my addr 172.21.15.112:0/1234310315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 -- 172.21.15.112:0/1234310315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cfc80 msgr2=0x559b2a4d8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cfc80 0x559b2a4d8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 -- 172.21.15.112:0/1234310315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 msgr2=0x559b2a4d8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 -- 172.21.15.112:0/1234310315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2a4d96e0 con 0x559b2a4cb4d0 2024-05-10T10:43:51.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7bb2ffd700 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cb4d0 0x559b2a4dd6d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x559b2a4e7a10 tx=0x7f7ba4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11785fe1a616e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.873+0000 7f7b92ffd700 1 -- 172.21.15.112:0/1234310315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4022040 con 0x559b2a4cb4d0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7b92ffd700 1 -- 172.21.15.112:0/1234310315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7ba40055f0 con 0x559b2a4cb4d0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba80b30a0 con 0x559b2a4cb4d0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7b92ffd700 1 -- 172.21.15.112:0/1234310315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ba403f050 con 0x559b2a4cb4d0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 10 client.16347 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559b2a54f560 con 0x559b2a4cb4d0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7b92ffd700 1 -- 172.21.15.112:0/1234310315 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ba80b30a0 con 0x559b2a4cb4d0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7b92ffd700 1 -- 172.21.15.112:0/1234310315 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f7ba400ea70 con 0x559b2a4cb4d0 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 10 client.16347 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:51.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 20 client.16347 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a551c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 4 client.16347 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b2a42b240 con 0x559b2a4e4d70 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.874+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a551c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7b92ffd700 10 client.16347 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bb27fc700 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a551c90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ce813c79d3a902 server_cookie=d84b7c60267ac3c0 in_seq=0 out_seq=0 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7b92ffd700 1 -- 172.21.15.112:0/1234310315 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b2a42b240 con 0x559b2a4e4d70 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7b92ffd700 10 client.16347 handle_command_reply: tid=0 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 client.16347 shutdown 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cb4d0 msgr2=0x559b2a4dd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cb4d0 0x559b2a4dd6d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x559b2a4e7a10 tx=0x7f7ba4005040 comp rx=0 tx=0).stop 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 shutdown_connections 2024-05-10T10:43:51.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x559b2a4e4d70 0x559b2a551c90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559b2a4cfc80 0x559b2a4d8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b2a4cb4d0 0x559b2a4dd6d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 --2- 172.21.15.112:0/1234310315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b2a4cbdf0 0x559b2a4d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 >> 172.21.15.112:0/1234310315 conn(0x559b2a3f7100 msgr2=0x559b2a4bfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7b92ffd700 0 client.16347 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 shutdown_connections 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 1 -- 172.21.15.112:0/1234310315 wait complete. 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.875+0000 7f7bc41bfb80 20 client.16347 trim_cache size 0 max 16384 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b9c0b2900 msgr2=0x7f7b9c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b9c0b2900 0x7f7b9c0b4dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7bb419cba0 tx=0x7f7ba4009260 comp rx=0 tx=0).stop 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4102450 msgr2=0x7f7bb41a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4102450 0x7f7bb41a1de0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8006010 tx=0x7f7ba80047b0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 shutdown_connections 2024-05-10T10:43:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb4109520 0x7f7bb419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7b9c0b2900 0x7f7b9c0b4dc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bb4101a90 0x7f7bb41a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 --2- 172.21.15.112:0/2675235048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb4102450 0x7f7bb41a1de0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 >> 172.21.15.112:0/2675235048 conn(0x7f7bb4078a10 msgr2=0x7f7bb41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 shutdown_connections 2024-05-10T10:43:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:51.876+0000 7f7bb992c700 1 -- 172.21.15.112:0/2675235048 wait complete. 2024-05-10T10:43:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:53 smithi195 ceph-mon[33773]: pgmap v539: 129 pgs: 129 active+clean; 41 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 413 op/s 2024-05-10T10:43:53.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:53 smithi112 ceph-mon[30100]: pgmap v539: 129 pgs: 129 active+clean; 41 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 413 op/s 2024-05-10T10:43:53.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:53 smithi148 ceph-mon[33568]: pgmap v539: 129 pgs: 129 active+clean; 41 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 413 op/s 2024-05-10T10:43:53.736 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:43:53.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:43:54.011 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:55.312 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:55 smithi112 ceph-mon[30100]: pgmap v540: 129 pgs: 129 active+clean; 37 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.5 MiB/s wr, 555 op/s 2024-05-10T10:43:55.315 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:55 smithi195 ceph-mon[33773]: pgmap v540: 129 pgs: 129 active+clean; 37 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.5 MiB/s wr, 555 op/s 2024-05-10T10:43:55.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:55 smithi148 ceph-mon[33568]: pgmap v540: 129 pgs: 129 active+clean; 37 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.5 MiB/s wr, 555 op/s 2024-05-10T10:43:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 -- 172.21.15.112:0/2853827833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4101a90 msgr2=0x7f97e4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 --2- 172.21.15.112:0/2853827833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4101a90 0x7f97e4101f10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009990 tx=0x7f97d8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 -- 172.21.15.112:0/2853827833 shutdown_connections 2024-05-10T10:43:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 --2- 172.21.15.112:0/2853827833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97e4102450 0x7f97e4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 --2- 172.21.15.112:0/2853827833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4109520 0x7f97e4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 --2- 172.21.15.112:0/2853827833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4101a90 0x7f97e4101f10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 -- 172.21.15.112:0/2853827833 >> 172.21.15.112:0/2853827833 conn(0x7f97e4078a10 msgr2=0x7f97e40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 -- 172.21.15.112:0/2853827833 shutdown_connections 2024-05-10T10:43:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.904+0000 7f97e9079700 1 -- 172.21.15.112:0/2853827833 wait complete. 2024-05-10T10:43:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 Processor -- start 2024-05-10T10:43:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 -- start start 2024-05-10T10:43:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97e4101a90 0x7f97e419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4102450 0x7f97e419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4109520 0x7f97e41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4113500 con 0x7f97e4109520 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4113380 con 0x7f97e4101a90 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.905+0000 7f97e9079700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4113680 con 0x7f97e4102450 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4109520 0x7f97e41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4102450 0x7f97e419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4109520 0x7f97e41a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53402/0 (socket says 172.21.15.112:53402) 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4102450 0x7f97e419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46192/0 (socket says 172.21.15.112:46192) 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e359e700 1 -- 172.21.15.112:0/2621427259 learned_addr learned my addr 172.21.15.112:0/2621427259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97e4101a90 0x7f97e419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 -- 172.21.15.112:0/2621427259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97e4101a90 msgr2=0x7f97e419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97e4101a90 0x7f97e419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 -- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4109520 msgr2=0x7f97e41a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4109520 0x7f97e41a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 -- 172.21.15.112:0/2621427259 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e41a0d20 con 0x7f97e4102450 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97e4101a90 0x7f97e419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.906+0000 7f97e259c700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4102450 0x7f97e419c250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c00 tx=0x7f97d80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.907+0000 7f97cbfff700 1 -- 172.21.15.112:0/2621427259 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8026070 con 0x7f97e4102450 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.907+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e41a0fb0 con 0x7f97e4102450 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.907+0000 7f97cbfff700 1 -- 172.21.15.112:0/2621427259 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d8021410 con 0x7f97e4102450 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.907+0000 7f97cbfff700 1 -- 172.21.15.112:0/2621427259 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8017650 con 0x7f97e4102450 2024-05-10T10:43:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.907+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e410a280 con 0x7f97e4102450 2024-05-10T10:43:55.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.909+0000 7f97cbfff700 1 -- 172.21.15.112:0/2621427259 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f97d8029070 con 0x7f97e4102450 2024-05-10T10:43:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.910+0000 7f97cbfff700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f97cc0bb110 0x7f97cc0bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.910+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f97cc0bb110 0x7f97cc0bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.910+0000 7f97cbfff700 1 -- 172.21.15.112:0/2621427259 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f97d80e3150 con 0x7f97e4102450 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.910+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f97cc0bb110 0x7f97cc0bd5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009fd0 tx=0x7f97d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 Processor -- start 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 -- start start 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d021068550 0x55d021068930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068e70 0x55d02106c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02106cd00 0x55d02106d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d020fba8e0 con 0x55d02106cd00 2024-05-10T10:43:55.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d020f59840 con 0x55d021068e70 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97f390cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d020fb68a0 con 0x55d021068550 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02106cd00 0x55d02106d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e2d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d021068550 0x55d021068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02106cd00 0x55d02106d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53404/0 (socket says 172.21.15.112:53404) 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e2d9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d021068550 0x55d021068930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46204/0 (socket says 172.21.15.112:46204) 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e359e700 1 -- 172.21.15.112:0/3565619216 learned_addr learned my addr 172.21.15.112:0/3565619216 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e259c700 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068e70 0x55d02106c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e259c700 1 -- 172.21.15.112:0/3565619216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d021068550 msgr2=0x55d021068930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e259c700 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d021068550 0x55d021068930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e259c700 1 -- 172.21.15.112:0/3565619216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02106cd00 msgr2=0x55d02106d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e259c700 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02106cd00 0x55d02106d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e259c700 1 -- 172.21.15.112:0/3565619216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d020f07950 con 0x55d021068e70 2024-05-10T10:43:55.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.911+0000 7f97e359e700 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02106cd00 0x55d02106d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.912+0000 7f97e259c700 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068e70 0x55d02106c770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97d80e0910 tx=0x7f97d80a9c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d628519d096e5a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.912+0000 7f97c97fa700 1 -- 172.21.15.112:0/3565619216 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80aab70 con 0x55d021068e70 2024-05-10T10:43:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.912+0000 7f97c97fa700 1 -- 172.21.15.112:0/3565619216 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d80aa470 con 0x55d021068e70 2024-05-10T10:43:55.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.912+0000 7f97c97fa700 1 -- 172.21.15.112:0/3565619216 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80a4df0 con 0x55d021068e70 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- 172.21.15.112:0/3565619216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068e70 msgr2=0x55d02106c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068e70 0x55d02106c770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97d80e0910 tx=0x7f97d80a9c60 comp rx=0 tx=0).stop 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- 172.21.15.112:0/3565619216 shutdown_connections 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d02106cd00 0x55d02106d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d021068550 0x55d021068930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 --2- 172.21.15.112:0/3565619216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068e70 0x55d02106c770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- 172.21.15.112:0/3565619216 >> 172.21.15.112:0/3565619216 conn(0x55d020f94180 msgr2=0x55d021062cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- 172.21.15.112:0/3565619216 shutdown_connections 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- 172.21.15.112:0/3565619216 wait complete. 2024-05-10T10:43:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- start start 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 Processor -- start 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d0210740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068e70 0x55d0210714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d0210719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d021074cd0 con 0x55d021068e70 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d020f20cb0 con 0x55d021068550 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.913+0000 7f97f390cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d021074e50 con 0x55d02106cd00 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97f390cb80 10 client.0 _ll_register_callbacks cb 0x55d020f529a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068e70 0x55d0210714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d0210740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068e70 0x55d0210714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53408/0 (socket says 172.21.15.112:53408) 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 -- 172.21.15.112:0/2569253607 learned_addr learned my addr 172.21.15.112:0/2569253607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e359e700 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d0210719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 -- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 msgr2=0x55d0210719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d0210719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 -- 172.21.15.112:0/2569253607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 msgr2=0x55d0210740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d0210740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 -- 172.21.15.112:0/2569253607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d020f56710 con 0x55d021068e70 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d0210740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97e259c700 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068e70 0x55d0210714a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f97d80a64f0 tx=0x7f97d800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5afc73ab7fd97d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2569253607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80aab70 con 0x55d021068e70 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2569253607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f97d80a4da0 con 0x55d021068e70 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d80aa470 con 0x55d021068e70 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.914+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2569253607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f97d80b3240 con 0x55d021068e70 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 10 client.16359 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d021074cd0 con 0x55d021068e70 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2569253607 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d80aa470 con 0x55d021068e70 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2569253607 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f97d80a5de0 con 0x55d021068e70 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 10 client.16359 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 20 client.16359 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210818c0 0x55d021083d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 4 client.16359 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0210346d0 con 0x55d0210818c0 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.915+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210818c0 0x55d021083d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.916+0000 7f97c6ff5700 10 client.16359 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.916+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210818c0 0x55d021083d20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91d55f4857ac7d7 server_cookie=ef8e214f7a251757 in_seq=0 out_seq=0 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.916+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2569253607 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0210346d0 con 0x55d0210818c0 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.916+0000 7f97c6ff5700 10 client.16359 handle_command_reply: tid=0 2024-05-10T10:43:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 client.16359 shutdown 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068e70 msgr2=0x55d0210714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068e70 0x55d0210714a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f97d80a64f0 tx=0x7f97d800b490 comp rx=0 tx=0).stop 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 shutdown_connections 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210818c0 0x55d021083d20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d0210719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d0210740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2569253607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068e70 0x55d0210714a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 >> 172.21.15.112:0/2569253607 conn(0x55d020f94180 msgr2=0x55d02105d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97c6ff5700 0 client.16359 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 shutdown_connections 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 1 -- 172.21.15.112:0/2569253607 wait complete. 2024-05-10T10:43:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.917+0000 7f97f390cb80 20 client.16359 trim_cache size 0 max 16384 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.941+0000 7f97f390cb80 1 Processor -- start 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.941+0000 7f97f390cb80 1 -- start start 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068550 0x55d021070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d021073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021073750 0x55d02106a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97f390cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97d000ea70 con 0x55d021068550 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97f390cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0010730 con 0x55d021073750 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97f390cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f97d000f140 con 0x55d02106cd00 2024-05-10T10:43:55.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068550 0x55d021070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068550 0x55d021070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53410/0 (socket says 172.21.15.112:53410) 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 -- 172.21.15.112:0/2162034209 learned_addr learned my addr 172.21.15.112:0/2162034209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e259c700 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d021073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e359e700 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021073750 0x55d02106a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 -- 172.21.15.112:0/2162034209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 msgr2=0x55d021073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d021073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 -- 172.21.15.112:0/2162034209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021073750 msgr2=0x55d02106a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021073750 0x55d02106a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 -- 172.21.15.112:0/2162034209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0210749f0 con 0x55d021068550 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e259c700 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d021073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:55.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068550 0x55d021070da0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f97d40041d0 tx=0x7f97d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac89d144e5ef8b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:55.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.942+0000 7f97c8ff9700 1 -- 172.21.15.112:0/2162034209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4030070 con 0x55d021068550 2024-05-10T10:43:55.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97c8ff9700 1 -- 172.21.15.112:0/2162034209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d401e940 con 0x55d021068550 2024-05-10T10:43:55.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97c8ff9700 1 -- 172.21.15.112:0/2162034209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d402a590 con 0x55d021068550 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 -- 172.21.15.112:0/2162034209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068550 msgr2=0x55d021070da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068550 0x55d021070da0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f97d40041d0 tx=0x7f97d4014000 comp rx=0 tx=0).stop 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 -- 172.21.15.112:0/2162034209 shutdown_connections 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021073750 0x55d02106a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021068550 0x55d021070da0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2162034209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d021073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 -- 172.21.15.112:0/2162034209 >> 172.21.15.112:0/2162034209 conn(0x55d020f94180 msgr2=0x55d021072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 -- 172.21.15.112:0/2162034209 shutdown_connections 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.943+0000 7f97f390cb80 1 -- 172.21.15.112:0/2162034209 wait complete. 2024-05-10T10:43:55.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 -- start start 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 Processor -- start 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d021079ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d02107a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021073750 0x55d021076390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97d000ffa0 con 0x55d021073750 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97d000f8d0 con 0x55d021068550 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f97d000ca10 con 0x55d02106cd00 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97f390cb80 10 client.0 _ll_register_callbacks cb 0x55d020f529a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d02107a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d02107a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46212/0 (socket says 172.21.15.112:46212) 2024-05-10T10:43:55.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e259c700 1 -- 172.21.15.112:0/2407829056 learned_addr learned my addr 172.21.15.112:0/2407829056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021073750 0x55d021076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d021079ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e259c700 1 -- 172.21.15.112:0/2407829056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 msgr2=0x55d021079ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e259c700 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d021079ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e259c700 1 -- 172.21.15.112:0/2407829056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021073750 msgr2=0x55d021076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.944+0000 7f97e259c700 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021073750 0x55d021076390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97e259c700 1 -- 172.21.15.112:0/2407829056 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d021076bd0 con 0x55d02106cd00 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97e259c700 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d02107a420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97d80e0910 tx=0x7f97d80a92a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e2e5b80523d5da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2407829056 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80aab70 con 0x55d02106cd00 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2407829056 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f97d800fb10 con 0x55d02106cd00 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d401e940 con 0x55d02106cd00 2024-05-10T10:43:55.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2407829056 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f97d80aa470 con 0x55d02106cd00 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97f390cb80 10 client.25774 fetch_fsmap learned FSMap version 36 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d021076bd0 con 0x55d02106cd00 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2407829056 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97d401e940 con 0x55d02106cd00 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.945+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2407829056 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f97d8084b80 con 0x55d02106cd00 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97f390cb80 10 client.25774 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97f390cb80 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97f390cb80 20 client.25774 populate_metadata read hostname 'smithi112' 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210ecfd0 0x55d0210ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97f390cb80 4 client.25774 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d020fc82c0 con 0x55d0210ecfd0 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210ecfd0 0x55d0210ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:55.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97c6ff5700 10 client.25774 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.946+0000 7f97e2d9d700 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210ecfd0 0x55d0210ef3c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1610e26f04fd601 server_cookie=36a4b6bd952086a0 in_seq=0 out_seq=0 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97c6ff5700 1 -- 172.21.15.112:0/2407829056 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d020fc82c0 con 0x55d0210ecfd0 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97c6ff5700 10 client.25774 handle_command_reply: tid=0 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 client.25774 shutdown 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 msgr2=0x55d02107a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d02107a420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97d80e0910 tx=0x7f97d80a92a0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 shutdown_connections 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55d0210ecfd0 0x55d0210ef3c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d021073750 0x55d021076390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d021068550 0x55d021079ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 --2- 172.21.15.112:0/2407829056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d02106cd00 0x55d02107a420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97c6ff5700 0 client.25774 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 >> 172.21.15.112:0/2407829056 conn(0x55d020f94180 msgr2=0x55d021067360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 shutdown_connections 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 1 -- 172.21.15.112:0/2407829056 wait complete. 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.947+0000 7f97f390cb80 20 client.25774 trim_cache size 0 max 16384 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "eab31fcb-6ff2-4ba5-b185-3bff2ab06be3", 2024-05-10T10:43:55.955 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:43:55.956 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:43:55.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f97cc0bb110 msgr2=0x7f97cc0bd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f97cc0bb110 0x7f97cc0bd5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009fd0 tx=0x7f97d4009260 comp rx=0 tx=0).stop 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4102450 msgr2=0x7f97e419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4102450 0x7f97e419c250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c00 tx=0x7f97d80047b0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 shutdown_connections 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e4109520 0x7f97e41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97e4101a90 0x7f97e419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f97cc0bb110 0x7f97cc0bd5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 --2- 172.21.15.112:0/2621427259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97e4102450 0x7f97e419c250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 >> 172.21.15.112:0/2621427259 conn(0x7f97e4078a10 msgr2=0x7f97e40fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 shutdown_connections 2024-05-10T10:43:55.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:43:55.948+0000 7f97e9079700 1 -- 172.21.15.112:0/2621427259 wait complete. 2024-05-10T10:43:56.316 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:43:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: 2024-05-10T10:43:55.974+0000 7ff3584cc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002bb [...2,head] /volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp/ auth{0=1,3=1,4=1} v5735 ap=2 f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1) n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2) (inest sync->mix g=0,3,4) caps={15294=pAsLsXsFs/-@1} | lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x5572a8cae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000002bb:[<0x10000000007/ltp v436>,<0x10000000006/xfstests-dev v72>,<0x10000000005/tmp v10>,<0x10000000003/client.0 v12>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v28>,<0x10000000001/sv_0 v52>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x100000002bb:[<0x10000000007/ltp v5735>,<0x10000000006/xfstests-dev v12369>,<0x10000000005/tmp v6802>,<0x10000000003/client.0 v231>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v237>,<0x10000000001/sv_0 v253>,<0x10000000000/qa v243>,<0x1/volumes v235>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T10:42:55.038905+0000 15=14+1)","ondisk_value.rstat":"n(v0 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","memory_value.dirstat":"f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1)","memory_value.rstat":"n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: pgmap v541: 129 pgs: 129 active+clean; 37 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:43:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:57 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: pgmap v541: 129 pgs: 129 active+clean; 37 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:43:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:57 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: pgmap v541: 129 pgs: 129 active+clean; 37 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:43:57.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:43:57.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:43:57.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:57 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:43:57.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:43:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:43:59.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:43:59 smithi195 ceph-mon[33773]: pgmap v542: 129 pgs: 129 active+clean; 38 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 429 op/s 2024-05-10T10:43:59.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:43:59 smithi112 ceph-mon[30100]: pgmap v542: 129 pgs: 129 active+clean; 38 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 429 op/s 2024-05-10T10:43:59.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:43:59 smithi148 ceph-mon[33568]: pgmap v542: 129 pgs: 129 active+clean; 38 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 429 op/s 2024-05-10T10:44:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.378+0000 7f257d3e6700 1 -- 172.21.15.112:0/489320435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f257806c530 msgr2=0x7f2578110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.378+0000 7f257d3e6700 1 --2- 172.21.15.112:0/489320435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f257806c530 0x7f2578110350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2570009f90 tx=0x7f257000b330 comp rx=0 tx=0).stop 2024-05-10T10:44:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.378+0000 7f257d3e6700 1 -- 172.21.15.112:0/489320435 shutdown_connections 2024-05-10T10:44:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.378+0000 7f257d3e6700 1 --2- 172.21.15.112:0/489320435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f257806c530 0x7f2578110350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.378+0000 7f257d3e6700 1 --2- 172.21.15.112:0/489320435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2578109530 0x7f2578109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.378+0000 7f257d3e6700 1 --2- 172.21.15.112:0/489320435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f257806bb70 0x7f257806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.378+0000 7f257d3e6700 1 -- 172.21.15.112:0/489320435 >> 172.21.15.112:0/489320435 conn(0x7f2578079350 msgr2=0x7f2578079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.379+0000 7f257d3e6700 1 -- 172.21.15.112:0/489320435 shutdown_connections 2024-05-10T10:44:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.379+0000 7f257d3e6700 1 -- 172.21.15.112:0/489320435 wait complete. 2024-05-10T10:44:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.379+0000 7f257d3e6700 1 Processor -- start 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f257d3e6700 1 -- start start 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f257d3e6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257806bb70 0x7f257810e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f257d3e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f257806c530 0x7f257810eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f257d3e6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2578109530 0x7f25781a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f257d3e6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2578113560 con 0x7f257806c530 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f257d3e6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25781133e0 con 0x7f2578109530 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f257d3e6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f25781136e0 con 0x7f257806bb70 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257806bb70 0x7f257810e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257806bb70 0x7f257810e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:46242/0 (socket says 172.21.15.112:46242) 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 -- 172.21.15.112:0/882952683 learned_addr learned my addr 172.21.15.112:0/882952683 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f25767fc700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f257806c530 0x7f257810eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f25777fe700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2578109530 0x7f25781a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 -- 172.21.15.112:0/882952683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2578109530 msgr2=0x7f25781a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2578109530 0x7f25781a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 -- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f257806c530 msgr2=0x7f257810eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f257806c530 0x7f257810eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.380+0000 7f2576ffd700 1 -- 172.21.15.112:0/882952683 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25781a5130 con 0x7f257806bb70 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.381+0000 7f2576ffd700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257806bb70 0x7f257810e5e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f256800e9c0 tx=0x7f256800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.381+0000 7f255ffff700 1 -- 172.21.15.112:0/882952683 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256800cbd0 con 0x7f257806bb70 2024-05-10T10:44:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.381+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25781a5420 con 0x7f257806bb70 2024-05-10T10:44:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.381+0000 7f255ffff700 1 -- 172.21.15.112:0/882952683 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f256800ee60 con 0x7f257806bb70 2024-05-10T10:44:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.381+0000 7f255ffff700 1 -- 172.21.15.112:0/882952683 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680054b0 con 0x7f257806bb70 2024-05-10T10:44:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.381+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25781acce0 con 0x7f257806bb70 2024-05-10T10:44:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.384+0000 7f255ffff700 1 -- 172.21.15.112:0/882952683 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2568021440 con 0x7f257806bb70 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.384+0000 7f255ffff700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25600b2880 0x7f25600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.384+0000 7f25767fc700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25600b2880 0x7f25600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.384+0000 7f255ffff700 1 -- 172.21.15.112:0/882952683 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f25680e2730 con 0x7f257806bb70 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 Processor -- start 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f25767fc700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25600b2880 0x7f25600b4d40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f256c009750 tx=0x7f256c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 -- start start 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b06810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ad62a58940 con 0x55ad62b0cad0 2024-05-10T10:44:00.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ad629bdda0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f2587c79b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ad62a54900 con 0x55ad62b06d50 2024-05-10T10:44:00.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.385+0000 7f25767fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b06810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f2576ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f25767fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b06810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53448/0 (socket says 172.21.15.112:53448) 2024-05-10T10:44:00.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f25767fc700 1 -- 172.21.15.112:0/80129232 learned_addr learned my addr 172.21.15.112:0/80129232 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:00.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f25777fe700 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f2576ffd700 1 -- 172.21.15.112:0/80129232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 msgr2=0x55ad62b07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f2576ffd700 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b07190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f2576ffd700 1 -- 172.21.15.112:0/80129232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 msgr2=0x55ad62b06810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f2576ffd700 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b06810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f2576ffd700 1 -- 172.21.15.112:0/80129232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad629a58d0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f25767fc700 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b06810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f2576ffd700 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0c590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25680e0210 tx=0x7f25680a5220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fe58a420e49800 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:00.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f255d7fa700 1 -- 172.21.15.112:0/80129232 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680b2170 con 0x55ad62b0c1b0 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f255d7fa700 1 -- 172.21.15.112:0/80129232 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25680a99c0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.386+0000 7f255d7fa700 1 -- 172.21.15.112:0/80129232 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680cc0e0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 -- 172.21.15.112:0/80129232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 msgr2=0x55ad62b0c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0c590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25680e0210 tx=0x7f25680a5220 comp rx=0 tx=0).stop 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 -- 172.21.15.112:0/80129232 shutdown_connections 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0c590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 --2- 172.21.15.112:0/80129232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b06810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 -- 172.21.15.112:0/80129232 >> 172.21.15.112:0/80129232 conn(0x55ad62a32b70 msgr2=0x55ad62b012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 -- 172.21.15.112:0/80129232 shutdown_connections 2024-05-10T10:44:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.387+0000 7f2587c79b80 1 -- 172.21.15.112:0/80129232 wait complete. 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 -- start start 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 Processor -- start 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ad62b13450 con 0x55ad62b0cad0 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ad62a11ed0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ad62b135d0 con 0x55ad62b06d50 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2587c79b80 10 client.0 _ll_register_callbacks cb 0x55ad629ec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f25777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f25777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53456/0 (socket says 172.21.15.112:53456) 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f25777fe700 1 -- 172.21.15.112:0/3926718377 learned_addr learned my addr 172.21.15.112:0/3926718377 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f25767fc700 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f2576ffd700 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.388+0000 7f25767fc700 1 -- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 msgr2=0x55ad62b126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f25767fc700 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f25767fc700 1 -- 172.21.15.112:0/3926718377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 msgr2=0x55ad62b0ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f25767fc700 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f25767fc700 1 -- 172.21.15.112:0/3926718377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad62a0f760 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f25767fc700 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0fa80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ad62b03450 tx=0x7f256c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d80393bc4782d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f255aff5700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f255aff5700 1 -- 172.21.15.112:0/3926718377 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c02b060 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f255aff5700 1 -- 172.21.15.112:0/3926718377 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f256c01da00 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25680a99c0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f255aff5700 1 -- 172.21.15.112:0/3926718377 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f256c032050 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f2587c79b80 10 client.25760 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ad62b1dd40 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.389+0000 7f255aff5700 1 -- 172.21.15.112:0/3926718377 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25680a99c0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f255aff5700 1 -- 172.21.15.112:0/3926718377 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f256c026bc0 con 0x55ad62b0c1b0 2024-05-10T10:44:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2587c79b80 10 client.25760 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2587c79b80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2587c79b80 20 client.25760 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2587c79b80 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b203d0 0x55ad62b227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2587c79b80 4 client.25760 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad62a66320 con 0x55ad62b203d0 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2576ffd700 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b203d0 0x55ad62b227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f255aff5700 10 client.25760 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.390+0000 7f2576ffd700 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b203d0 0x55ad62b227c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88a94c2322e842d7 server_cookie=afda34b1fe0d1306 in_seq=0 out_seq=0 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f255aff5700 1 -- 172.21.15.112:0/3926718377 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ad62a66320 con 0x55ad62b203d0 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f255aff5700 10 client.25760 handle_command_reply: tid=0 2024-05-10T10:44:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 client.25760 shutdown 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 msgr2=0x55ad62b0fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0fa80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ad62b03450 tx=0x7f256c0047c0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 shutdown_connections 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b203d0 0x55ad62b227c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f255aff5700 0 client.25760 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 --2- 172.21.15.112:0/3926718377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b0fa80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 >> 172.21.15.112:0/3926718377 conn(0x55ad62a32b70 msgr2=0x55ad62afbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 shutdown_connections 2024-05-10T10:44:00.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.391+0000 7f2587c79b80 1 -- 172.21.15.112:0/3926718377 wait complete. 2024-05-10T10:44:00.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.392+0000 7f2587c79b80 20 client.25760 trim_cache size 0 max 16384 2024-05-10T10:44:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 Processor -- start 2024-05-10T10:44:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 -- start start 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b06810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b1c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f256400eac0 con 0x55ad62b0cad0 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2564010780 con 0x55ad62b0c1b0 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2587c79b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f256400f190 con 0x55ad62b06d50 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f25777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f25767fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b1c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f25777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53468/0 (socket says 172.21.15.112:53468) 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f25777fe700 1 -- 172.21.15.112:0/333776921 learned_addr learned my addr 172.21.15.112:0/333776921 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2576ffd700 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b06810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2576ffd700 1 -- 172.21.15.112:0/333776921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 msgr2=0x55ad62b1c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2576ffd700 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b1c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.415+0000 7f2576ffd700 1 -- 172.21.15.112:0/333776921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 msgr2=0x55ad62b0ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.416+0000 7f2576ffd700 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.416+0000 7f2576ffd700 1 -- 172.21.15.112:0/333776921 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad62afdc80 con 0x55ad62b06d50 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.416+0000 7f25777fe700 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.416+0000 7f2576ffd700 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b06810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f25680dfea0 tx=0x7f25680cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befa14192c96f54 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.416+0000 7f255cff9700 1 -- 172.21.15.112:0/333776921 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680a9790 con 0x55ad62b06d50 2024-05-10T10:44:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.416+0000 7f255cff9700 1 -- 172.21.15.112:0/333776921 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25680a99c0 con 0x55ad62b06d50 2024-05-10T10:44:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.416+0000 7f255cff9700 1 -- 172.21.15.112:0/333776921 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256800cd70 con 0x55ad62b06d50 2024-05-10T10:44:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 -- 172.21.15.112:0/333776921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 msgr2=0x55ad62b06810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b06810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f25680dfea0 tx=0x7f25680cc040 comp rx=0 tx=0).stop 2024-05-10T10:44:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 -- 172.21.15.112:0/333776921 shutdown_connections 2024-05-10T10:44:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b0ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b06810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 --2- 172.21.15.112:0/333776921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b1c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 -- 172.21.15.112:0/333776921 >> 172.21.15.112:0/333776921 conn(0x55ad62a32b70 msgr2=0x55ad62b0d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 -- 172.21.15.112:0/333776921 shutdown_connections 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 -- 172.21.15.112:0/333776921 wait complete. 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 -- start start 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.417+0000 7f2587c79b80 1 Processor -- start 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b18e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2587c79b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b15890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2587c79b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f256400fff0 con 0x55ad62b0cad0 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2587c79b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f256400f920 con 0x55ad62b0c1b0 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2587c79b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f256400ca60 con 0x55ad62b06d50 2024-05-10T10:44:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2587c79b80 10 client.0 _ll_register_callbacks cb 0x55ad629ec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b15890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25767fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b18e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b15890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53474/0 (socket says 172.21.15.112:53474) 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 -- 172.21.15.112:0/2384999570 learned_addr learned my addr 172.21.15.112:0/2384999570 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25767fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b18e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:44002/0 (socket says 172.21.15.112:44002) 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2576ffd700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 -- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 msgr2=0x55ad62b188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 -- 172.21.15.112:0/2384999570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 msgr2=0x55ad62b18e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b18e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 -- 172.21.15.112:0/2384999570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad62b15dd0 con 0x55ad62b0cad0 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f2576ffd700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25767fc700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b18e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f25777fe700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b15890 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2570003fd0 tx=0x7f257000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73de42a03a5922fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.418+0000 7f255aff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.419+0000 7f255aff5700 1 -- 172.21.15.112:0/2384999570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257002b060 con 0x55ad62b0cad0 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.419+0000 7f255aff5700 1 -- 172.21.15.112:0/2384999570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2570007630 con 0x55ad62b0cad0 2024-05-10T10:44:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.419+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25680a99c0 con 0x55ad62b0cad0 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.419+0000 7f255aff5700 1 -- 172.21.15.112:0/2384999570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2570032050 con 0x55ad62b0cad0 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.419+0000 7f2587c79b80 10 client.16383 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.419+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ad62b8b400 con 0x55ad62b0cad0 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.419+0000 7f255aff5700 1 -- 172.21.15.112:0/2384999570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25680a99c0 con 0x55ad62b0cad0 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f255aff5700 1 -- 172.21.15.112:0/2384999570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f2570011070 con 0x55ad62b0cad0 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2587c79b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2587c79b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2587c79b80 20 client.16383 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2587c79b80 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b8bbd0 0x55ad62b8dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2587c79b80 4 client.16383 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad62a603c0 con 0x55ad62b8bbd0 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2576ffd700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b8bbd0 0x55ad62b8dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f255aff5700 10 client.16383 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.420+0000 7f2576ffd700 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b8bbd0 0x55ad62b8dff0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f40201299274e1 server_cookie=1e5af53d51c9bd11 in_seq=0 out_seq=0 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f255aff5700 1 -- 172.21.15.112:0/2384999570 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ad62a603c0 con 0x55ad62b8bbd0 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f255aff5700 10 client.16383 handle_command_reply: tid=0 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 client.16383 shutdown 2024-05-10T10:44:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 msgr2=0x55ad62b15890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b15890 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2570003fd0 tx=0x7f257000bc20 comp rx=0 tx=0).stop 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 shutdown_connections 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ad62b8bbd0 0x55ad62b8dff0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f255aff5700 0 client.16383 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad62b0cad0 0x55ad62b15890 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ad62b06d50 0x55ad62b188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 --2- 172.21.15.112:0/2384999570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad62b0c1b0 0x55ad62b18e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 >> 172.21.15.112:0/2384999570 conn(0x55ad62a32b70 msgr2=0x55ad62b11080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 shutdown_connections 2024-05-10T10:44:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.421+0000 7f2587c79b80 1 -- 172.21.15.112:0/2384999570 wait complete. 2024-05-10T10:44:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f2587c79b80 20 client.16383 trim_cache size 0 max 16384 2024-05-10T10:44:00.433 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:44:00.433 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:44:00.433 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-10T10:44:00.433 INFO:teuthology.orchestra.run.smithi112.stdout: "eab31fcb-6ff2-4ba5-b185-3bff2ab06be3": { 2024-05-10T10:44:00.433 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-10T10:44:00.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "eab31fcb-6ff2-4ba5-b185-3bff2ab06be3", 2024-05-10T10:44:00.434 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-10T10:44:00.434 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:44:00.434 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:44:00.434 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:44:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25600b2880 msgr2=0x7f25600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f257d3e6700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25600b2880 0x7f25600b4d40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f256c009750 tx=0x7f256c009310 comp rx=0 tx=0).stop 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257806bb70 msgr2=0x7f257810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f257d3e6700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257806bb70 0x7f257810e5e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f256800e9c0 tx=0x7f256800ecd0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 shutdown_connections 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f257d3e6700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2578109530 0x7f25781a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.422+0000 7f257d3e6700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f25600b2880 0x7f25600b4d40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.423+0000 7f257d3e6700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257806bb70 0x7f257810e5e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.423+0000 7f257d3e6700 1 --2- 172.21.15.112:0/882952683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f257806c530 0x7f257810eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.423+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 >> 172.21.15.112:0/882952683 conn(0x7f2578079350 msgr2=0x7f2578103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.423+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 shutdown_connections 2024-05-10T10:44:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:00.423+0000 7f257d3e6700 1 -- 172.21.15.112:0/882952683 wait complete. 2024-05-10T10:44:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:01 smithi195 ceph-mon[33773]: pgmap v543: 129 pgs: 129 active+clean; 38 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 669 op/s 2024-05-10T10:44:01.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:01 smithi112 ceph-mon[30100]: pgmap v543: 129 pgs: 129 active+clean; 38 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 669 op/s 2024-05-10T10:44:01.340 INFO:tasks.cephfs.filesystem:scrub status for tag:eab31fcb-6ff2-4ba5-b185-3bff2ab06be3 - {'path': '/', 'tag': 'eab31fcb-6ff2-4ba5-b185-3bff2ab06be3', 'options': 'recursive,force'} 2024-05-10T10:44:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:01 smithi148 ceph-mon[33568]: pgmap v543: 129 pgs: 129 active+clean; 38 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 669 op/s 2024-05-10T10:44:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:03 smithi195 ceph-mon[33773]: pgmap v544: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 757 op/s 2024-05-10T10:44:03.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:03 smithi112 ceph-mon[30100]: pgmap v544: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 757 op/s 2024-05-10T10:44:03.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:03 smithi148 ceph-mon[33568]: pgmap v544: 129 pgs: 129 active+clean; 39 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 757 op/s 2024-05-10T10:44:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 -- 172.21.15.112:0/3437386455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200108790 msgr2=0x7fa20010ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 --2- 172.21.15.112:0/3437386455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200108790 0x7fa20010ab80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009ca0 tx=0x7fa1f802a8c0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 -- 172.21.15.112:0/3437386455 shutdown_connections 2024-05-10T10:44:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 --2- 172.21.15.112:0/3437386455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200108790 0x7fa20010ab80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 --2- 172.21.15.112:0/3437386455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa200105e60 0x7fa200108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 --2- 172.21.15.112:0/3437386455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa200103530 0x7fa200105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 -- 172.21.15.112:0/3437386455 >> 172.21.15.112:0/3437386455 conn(0x7fa2000fcec0 msgr2=0x7fa2000ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 -- 172.21.15.112:0/3437386455 shutdown_connections 2024-05-10T10:44:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.350+0000 7fa207719640 1 -- 172.21.15.112:0/3437386455 wait complete. 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 Processor -- start 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 -- start start 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa200103530 0x7fa200111a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200105e60 0x7fa200111fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa200108790 0x7fa200112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010c150 con 0x7fa200105e60 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010bfd0 con 0x7fa200103530 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa207719640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010c2d0 con 0x7fa200108790 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa204cb4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200105e60 0x7fa200111fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa204cb4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200105e60 0x7fa200111fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53494/0 (socket says 172.21.15.112:53494) 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa2054b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa200103530 0x7fa200111a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa204cb4640 1 -- 172.21.15.112:0/1036859348 learned_addr learned my addr 172.21.15.112:0/1036859348 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.351+0000 7fa205cb6640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa200108790 0x7fa200112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa2054b5640 1 -- 172.21.15.112:0/1036859348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa200108790 msgr2=0x7fa200112fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa2054b5640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa200108790 0x7fa200112fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa2054b5640 1 -- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200105e60 msgr2=0x7fa200111fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa2054b5640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200105e60 0x7fa200111fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa2054b5640 1 -- 172.21.15.112:0/1036859348 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa200113620 con 0x7fa200103530 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa204cb4640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200105e60 0x7fa200111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa2054b5640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa200103530 0x7fa200111a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00e9f0 tx=0x7fa1fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa1f67fc640 1 -- 172.21.15.112:0/1036859348 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00ccc0 con 0x7fa200103530 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa200113910 con 0x7fa200103530 2024-05-10T10:44:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa1f67fc640 1 -- 172.21.15.112:0/1036859348 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1fc0108c0 con 0x7fa200103530 2024-05-10T10:44:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa1f67fc640 1 -- 172.21.15.112:0/1036859348 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0187c0 con 0x7fa200103530 2024-05-10T10:44:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.352+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa200113eb0 con 0x7fa200103530 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.355+0000 7fa1f67fc640 1 -- 172.21.15.112:0/1036859348 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa1fc010430 con 0x7fa200103530 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.355+0000 7fa20a43a740 1 Processor -- start 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.355+0000 7fa20a43a740 1 -- start start 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69adb0 0x555c8d69b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69b7a0 0x555c8d6a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa20a43a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c8d5ff5d0 con 0x555c8d69b7a0 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa20a43a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555c8d5b8890 con 0x555c8d69adb0 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa20a43a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555c8d5fb590 con 0x555c8d6a0910 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa1f67fc640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa1ec0b28a0 0x7fa1ec0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa2054b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69b7a0 0x555c8d6a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa2054b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69b7a0 0x555c8d6a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53496/0 (socket says 172.21.15.112:53496) 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa2054b5640 1 -- 172.21.15.112:0/4011943960 learned_addr learned my addr 172.21.15.112:0/4011943960 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa205cb6640 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69adb0 0x555c8d69b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa204cb4640 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa205cb6640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa1ec0b28a0 0x7fa1ec0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa1f67fc640 1 -- 172.21.15.112:0/1036859348 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa1fc0e2870 con 0x7fa200103530 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa204cb4640 1 -- 172.21.15.112:0/4011943960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69adb0 msgr2=0x555c8d69b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa204cb4640 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69adb0 0x555c8d69b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa204cb4640 1 -- 172.21.15.112:0/4011943960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69b7a0 msgr2=0x555c8d6a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa204cb4640 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69b7a0 0x555c8d6a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa204cb4640 1 -- 172.21.15.112:0/4011943960 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8d311210 con 0x555c8d6a0910 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.356+0000 7fa2054b5640 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69b7a0 0x555c8d6a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa204cb4640 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6a2da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa200112f40 tx=0x7fa1f00119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a13d89e71967830 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa205cb6640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa1ec0b28a0 0x7fa1ec0b4d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8006010 tx=0x7fa1f80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa1eb7fe640 1 -- 172.21.15.112:0/4011943960 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f001e070 con 0x555c8d6a0910 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa1eb7fe640 1 -- 172.21.15.112:0/4011943960 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1f00044c0 con 0x555c8d6a0910 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa1eb7fe640 1 -- 172.21.15.112:0/4011943960 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000f7c0 con 0x555c8d6a0910 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa20a43a740 1 -- 172.21.15.112:0/4011943960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 msgr2=0x555c8d6a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa20a43a740 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6a2da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa200112f40 tx=0x7fa1f00119a0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa20a43a740 1 -- 172.21.15.112:0/4011943960 shutdown_connections 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa20a43a740 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6a2da0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.357+0000 7fa20a43a740 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69b7a0 0x555c8d6a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 --2- 172.21.15.112:0/4011943960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69adb0 0x555c8d69b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 -- 172.21.15.112:0/4011943960 >> 172.21.15.112:0/4011943960 conn(0x555c8d696420 msgr2=0x555c8d696820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 -- 172.21.15.112:0/4011943960 shutdown_connections 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 -- 172.21.15.112:0/4011943960 wait complete. 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 -- start start 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 Processor -- start 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d6ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c8d6a38c0 con 0x555c8d6a0910 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555c8d5bc370 con 0x555c8d69b7a0 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555c8d6a3a40 con 0x555c8d69adb0 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa20a43a740 10 client.0 _ll_register_callbacks cb 0x555c8d310a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa205cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d6ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa2054b5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa205cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d6ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53502/0 (socket says 172.21.15.112:53502) 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.358+0000 7fa205cb6640 1 -- 172.21.15.112:0/2700249229 learned_addr learned my addr 172.21.15.112:0/2700249229 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa204cb4640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa2054b5640 1 -- 172.21.15.112:0/2700249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 msgr2=0x555c8d6a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa2054b5640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa2054b5640 1 -- 172.21.15.112:0/2700249229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 msgr2=0x555c8d6ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa2054b5640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d6ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa2054b5640 1 -- 172.21.15.112:0/2700249229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8d3dd620 con 0x555c8d69adb0 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa204cb4640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa205cb6640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d6ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa2054b5640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6a4f70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x555c8d69c210 tx=0x7fa1fc010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbadc9ecee941755 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa1e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/2700249229 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0adeb0 con 0x555c8d69adb0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/2700249229 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa1fc0cc0e0 con 0x555c8d69adb0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c8d60d1e0 con 0x555c8d69adb0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/2700249229 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc0b2960 con 0x555c8d69adb0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa20a43a740 10 client.25804 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555c8d6af540 con 0x555c8d69adb0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.359+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/2700249229 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555c8d60d1e0 con 0x555c8d69adb0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/2700249229 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fa1fc0a89f0 con 0x555c8d69adb0 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa20a43a740 10 client.25804 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa20a43a740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa20a43a740 20 client.25804 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa20a43a740 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d6b0c80 0x555c8d6b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa20a43a740 4 client.25804 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c8d607050 con 0x555c8d6b0c80 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa204cb4640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d6b0c80 0x555c8d6b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa1e8ff9640 10 client.25804 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.360+0000 7fa204cb4640 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d6b0c80 0x555c8d6b3140 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=264482771e9145f3 server_cookie=7e21508181d5538e in_seq=0 out_seq=0 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/2700249229 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c8d607050 con 0x555c8d6b0c80 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa1e8ff9640 10 client.25804 handle_command_reply: tid=0 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 client.25804 shutdown 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 msgr2=0x555c8d6a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6a4f70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x555c8d69c210 tx=0x7fa1fc010720 comp rx=0 tx=0).stop 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 shutdown_connections 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d6ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d6b0c80 0x555c8d6b3140 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 --2- 172.21.15.112:0/2700249229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6a4f70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 >> 172.21.15.112:0/2700249229 conn(0x555c8d696420 msgr2=0x555c8d695580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa1e8ff9640 0 client.25804 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 shutdown_connections 2024-05-10T10:44:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 1 -- 172.21.15.112:0/2700249229 wait complete. 2024-05-10T10:44:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.361+0000 7fa20a43a740 20 client.25804 trim_cache size 0 max 16384 2024-05-10T10:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 Processor -- start 2024-05-10T10:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 -- start start 2024-05-10T10:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d699330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001ba1f0 con 0x555c8d6a0910 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001bbeb0 con 0x555c8d69b7a0 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa20a43a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001ba8c0 con 0x555c8d69adb0 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d699330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa2054b5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d699330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53516/0 (socket says 172.21.15.112:53516) 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 -- 172.21.15.112:0/898977658 learned_addr learned my addr 172.21.15.112:0/898977658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa204cb4640 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 -- 172.21.15.112:0/898977658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 msgr2=0x555c8d6b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 -- 172.21.15.112:0/898977658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 msgr2=0x555c8d6b8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6b8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.378+0000 7fa205cb6640 1 -- 172.21.15.112:0/898977658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8d695cd0 con 0x555c8d6a0910 2024-05-10T10:44:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa205cb6640 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d699330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009a60 tx=0x7fa1f803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c4a94faf7f53a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa1eaffd640 1 -- 172.21.15.112:0/898977658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8004040 con 0x555c8d6a0910 2024-05-10T10:44:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa1eaffd640 1 -- 172.21.15.112:0/898977658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1f8053070 con 0x555c8d6a0910 2024-05-10T10:44:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa1eaffd640 1 -- 172.21.15.112:0/898977658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8030a20 con 0x555c8d6a0910 2024-05-10T10:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa20a43a740 1 -- 172.21.15.112:0/898977658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 msgr2=0x555c8d699330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa20a43a740 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d699330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009a60 tx=0x7fa1f803b000 comp rx=0 tx=0).stop 2024-05-10T10:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa20a43a740 1 -- 172.21.15.112:0/898977658 shutdown_connections 2024-05-10T10:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa20a43a740 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d6a0910 0x555c8d699330 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa20a43a740 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6b8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa20a43a740 1 --2- 172.21.15.112:0/898977658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d69adb0 0x555c8d6b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.379+0000 7fa20a43a740 1 -- 172.21.15.112:0/898977658 >> 172.21.15.112:0/898977658 conn(0x555c8d696420 msgr2=0x555c8d69c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 -- 172.21.15.112:0/898977658 shutdown_connections 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 -- 172.21.15.112:0/898977658 wait complete. 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 -- start start 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 Processor -- start 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69adb0 0x555c8d6a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6aa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001bb720 con 0x555c8d69adb0 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001bb050 con 0x555c8d69b7a0 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001b81d0 con 0x555c8d6a0910 2024-05-10T10:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa20a43a740 10 client.0 _ll_register_callbacks cb 0x7fa1fc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa2054b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69adb0 0x555c8d6a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa204cb4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa205cb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa204cb4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:44062/0 (socket says 172.21.15.112:44062) 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.380+0000 7fa204cb4640 1 -- 172.21.15.112:0/3023690799 learned_addr learned my addr 172.21.15.112:0/3023690799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa204cb4640 1 -- 172.21.15.112:0/3023690799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 msgr2=0x555c8d6aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa204cb4640 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa204cb4640 1 -- 172.21.15.112:0/3023690799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69adb0 msgr2=0x555c8d6a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa204cb4640 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69adb0 0x555c8d6a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa204cb4640 1 -- 172.21.15.112:0/3023690799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8d6aaf60 con 0x555c8d69b7a0 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa2054b5640 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69adb0 0x555c8d6a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa204cb4640 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a9a20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005fd0 tx=0x7fa1f0003ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6321b294662108b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa1e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/3023690799 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f001e040 con 0x555c8d69b7a0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f8053070 con 0x555c8d69b7a0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/3023690799 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa1f0005040 con 0x555c8d69b7a0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/3023690799 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa1f0039050 con 0x555c8d69b7a0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa20a43a740 10 client.25784 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.381+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555c8d71b780 con 0x555c8d69b7a0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/3023690799 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1f8053070 con 0x555c8d69b7a0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/3023690799 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fa1f0003fb0 con 0x555c8d69b7a0 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa20a43a740 10 client.25784 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa20a43a740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa20a43a740 20 client.25784 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa20a43a740 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d71c060 0x555c8d71e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa20a43a740 4 client.25784 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:44:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c8d5825f0 con 0x555c8d71c060 2024-05-10T10:44:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa2054b5640 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d71c060 0x555c8d71e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.382+0000 7fa1e8ff9640 10 client.25784 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.383+0000 7fa2054b5640 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d71c060 0x555c8d71e480 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b31bac2ef0b134 server_cookie=f9ffea33b8d6219b in_seq=0 out_seq=0 2024-05-10T10:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa1e8ff9640 1 -- 172.21.15.112:0/3023690799 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c8d5825f0 con 0x555c8d71c060 2024-05-10T10:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa1e8ff9640 10 client.25784 handle_command_reply: tid=0 2024-05-10T10:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 client.25784 shutdown 2024-05-10T10:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 msgr2=0x555c8d6a9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a9a20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005fd0 tx=0x7fa1f0003ad0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 shutdown_connections 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c8d6a0910 0x555c8d6aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555c8d71c060 0x555c8d71e480 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c8d69b7a0 0x555c8d6a9a20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 --2- 172.21.15.112:0/3023690799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d69adb0 0x555c8d6a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa1e8ff9640 0 client.25784 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 >> 172.21.15.112:0/3023690799 conn(0x555c8d696420 msgr2=0x555c8d699330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 shutdown_connections 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.396+0000 7fa20a43a740 1 -- 172.21.15.112:0/3023690799 wait complete. 2024-05-10T10:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa20a43a740 20 client.25784 trim_cache size 0 max 16384 2024-05-10T10:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa1ec0b28a0 msgr2=0x7fa1ec0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa1ec0b28a0 0x7fa1ec0b4d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8006010 tx=0x7fa1f80095c0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa200103530 msgr2=0x7fa200111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa200103530 0x7fa200111a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00e9f0 tx=0x7fa1fc00eeb0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 shutdown_connections 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fa1ec0b28a0 0x7fa1ec0b4d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa200108790 0x7fa200112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa200105e60 0x7fa200111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 --2- 172.21.15.112:0/1036859348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa200103530 0x7fa200111a80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 >> 172.21.15.112:0/1036859348 conn(0x7fa2000fcec0 msgr2=0x7fa200103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 shutdown_connections 2024-05-10T10:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:04.397+0000 7fa207719640 1 -- 172.21.15.112:0/1036859348 wait complete. 2024-05-10T10:44:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:05 smithi195 ceph-mon[33773]: pgmap v545: 129 pgs: 129 active+clean; 40 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 856 op/s 2024-05-10T10:44:05.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:05 smithi112 ceph-mon[30100]: pgmap v545: 129 pgs: 129 active+clean; 40 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 856 op/s 2024-05-10T10:44:05.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:05 smithi148 ceph-mon[33568]: pgmap v545: 129 pgs: 129 active+clean; 40 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 856 op/s 2024-05-10T10:44:07.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:07 smithi112 ceph-mon[30100]: pgmap v546: 129 pgs: 129 active+clean; 40 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 713 op/s 2024-05-10T10:44:07.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:07 smithi148 ceph-mon[33568]: pgmap v546: 129 pgs: 129 active+clean; 40 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 713 op/s 2024-05-10T10:44:07.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:07 smithi195 ceph-mon[33773]: pgmap v546: 129 pgs: 129 active+clean; 40 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 713 op/s 2024-05-10T10:44:09.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:09 smithi112 ceph-mon[30100]: pgmap v547: 129 pgs: 129 active+clean; 41 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 751 op/s 2024-05-10T10:44:09.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:09 smithi148 ceph-mon[33568]: pgmap v547: 129 pgs: 129 active+clean; 41 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 751 op/s 2024-05-10T10:44:09.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:09 smithi195 ceph-mon[33773]: pgmap v547: 129 pgs: 129 active+clean; 41 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 751 op/s 2024-05-10T10:44:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:44:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:44:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:44:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:44:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:44:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:44:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:44:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:44:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:44:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:44:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:44:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:44:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:11 smithi112 ceph-mon[30100]: pgmap v548: 129 pgs: 129 active+clean; 41 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 814 op/s 2024-05-10T10:44:11.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:11 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:44:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:11 smithi148 ceph-mon[33568]: pgmap v548: 129 pgs: 129 active+clean; 41 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 814 op/s 2024-05-10T10:44:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:11 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:44:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:11 smithi195 ceph-mon[33773]: pgmap v548: 129 pgs: 129 active+clean; 41 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 814 op/s 2024-05-10T10:44:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:11 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:44:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:13 smithi112 ceph-mon[30100]: pgmap v549: 129 pgs: 129 active+clean; 41 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 610 op/s 2024-05-10T10:44:13.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:13 smithi148 ceph-mon[33568]: pgmap v549: 129 pgs: 129 active+clean; 41 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 610 op/s 2024-05-10T10:44:13.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:13 smithi195 ceph-mon[33773]: pgmap v549: 129 pgs: 129 active+clean; 41 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 610 op/s 2024-05-10T10:44:15.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:15 smithi112 ceph-mon[30100]: pgmap v550: 129 pgs: 129 active+clean; 41 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 457 op/s 2024-05-10T10:44:15.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:15 smithi195 ceph-mon[33773]: pgmap v550: 129 pgs: 129 active+clean; 41 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 457 op/s 2024-05-10T10:44:15.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:15 smithi148 ceph-mon[33568]: pgmap v550: 129 pgs: 129 active+clean; 41 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 457 op/s 2024-05-10T10:44:17.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:17 smithi112 ceph-mon[30100]: pgmap v551: 129 pgs: 129 active+clean; 41 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 338 op/s 2024-05-10T10:44:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 -- 172.21.15.112:0/1797251503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 msgr2=0x7fb0b01026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/1797251503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b01026f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb09800aa10 tx=0x7fb09802b720 comp rx=0 tx=0).stop 2024-05-10T10:44:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 -- 172.21.15.112:0/1797251503 shutdown_connections 2024-05-10T10:44:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/1797251503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0102c30 0x7fb0b01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/1797251503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b01026f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/1797251503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b0101080 0x7fb0b01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 -- 172.21.15.112:0/1797251503 >> 172.21.15.112:0/1797251503 conn(0x7fb0b00fc610 msgr2=0x7fb0b00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 -- 172.21.15.112:0/1797251503 shutdown_connections 2024-05-10T10:44:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.497+0000 7fb0b4cda640 1 -- 172.21.15.112:0/1797251503 wait complete. 2024-05-10T10:44:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 Processor -- start 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 -- start start 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b0101080 0x7fb0b01968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b0196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0102c30 0x7fb0b019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010c1f0 con 0x7fb0b0102270 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010c070 con 0x7fb0b0102c30 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0b4cda640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010c370 con 0x7fb0b0101080 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.498+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b0196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b0101080 0x7fb0b01968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b0196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40012/0 (socket says 172.21.15.112:40012) 2024-05-10T10:44:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0102c30 0x7fb0b019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0a7fff640 1 -- 172.21.15.112:0/3078996568 learned_addr learned my addr 172.21.15.112:0/3078996568 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0ae59c640 1 -- 172.21.15.112:0/3078996568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0102c30 msgr2=0x7fb0b019de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0102c30 0x7fb0b019de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0ae59c640 1 -- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 msgr2=0x7fb0b0196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b0196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0ae59c640 1 -- 172.21.15.112:0/3078996568 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b019e4b0 con 0x7fb0b0101080 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b0196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b0101080 0x7fb0b01968e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000e9c0 tx=0x7fb0a000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0a77fe640 1 -- 172.21.15.112:0/3078996568 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a000cd80 con 0x7fb0b0101080 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0a77fe640 1 -- 172.21.15.112:0/3078996568 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a00108c0 con 0x7fb0b0101080 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b019e7a0 con 0x7fb0b0101080 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0a77fe640 1 -- 172.21.15.112:0/3078996568 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00187c0 con 0x7fb0b0101080 2024-05-10T10:44:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.499+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b019ed40 con 0x7fb0b0101080 2024-05-10T10:44:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.502+0000 7fb0a77fe640 1 -- 172.21.15.112:0/3078996568 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb0a001f020 con 0x7fb0b0101080 2024-05-10T10:44:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.502+0000 7fb0b79fb740 1 Processor -- start 2024-05-10T10:44:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.502+0000 7fb0b79fb740 1 -- start start 2024-05-10T10:44:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x55775036aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x55775036f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036fd20 0x5577503721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0a77fe640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb09c0b2920 0x7fb09c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5577502ca5d0 con 0x55775036a620 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557750283890 con 0x55775036b010 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5577502c6590 con 0x55775036fd20 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x55775036f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x55775036aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x55775036f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55192/0 (socket says 172.21.15.112:55192) 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 -- 172.21.15.112:0/4081634323 learned_addr learned my addr 172.21.15.112:0/4081634323 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036fd20 0x5577503721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0aed9d640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb09c0b2920 0x7fb09c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 -- 172.21.15.112:0/4081634323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036fd20 msgr2=0x5577503721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036fd20 0x5577503721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 -- 172.21.15.112:0/4081634323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 msgr2=0x55775036aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x55775036aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0a77fe640 1 -- 172.21.15.112:0/3078996568 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb0a00e25f0 con 0x7fb0b0101080 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.503+0000 7fb0ae59c640 1 -- 172.21.15.112:0/4081634323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774ffdc210 con 0x55775036b010 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.504+0000 7fb0aed9d640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb09c0b2920 0x7fb09c0b4de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005c40 tx=0x7fb0a8009bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.504+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x55775036f6b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a5900 tx=0x7fb0a000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7db6f7a6f7db949 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.504+0000 7fb0a4ff9640 1 -- 172.21.15.112:0/4081634323 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00a8ff0 con 0x55775036b010 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.504+0000 7fb0a4ff9640 1 -- 172.21.15.112:0/4081634323 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a0009460 con 0x55775036b010 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.504+0000 7fb0a4ff9640 1 -- 172.21.15.112:0/4081634323 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00b26e0 con 0x55775036b010 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 -- 172.21.15.112:0/4081634323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 msgr2=0x55775036f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x55775036f6b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a5900 tx=0x7fb0a000cf20 comp rx=0 tx=0).stop 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 -- 172.21.15.112:0/4081634323 shutdown_connections 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036fd20 0x5577503721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x55775036f6b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/4081634323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x55775036aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 -- 172.21.15.112:0/4081634323 >> 172.21.15.112:0/4081634323 conn(0x557750362960 msgr2=0x557750360f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 -- 172.21.15.112:0/4081634323 shutdown_connections 2024-05-10T10:44:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 -- 172.21.15.112:0/4081634323 wait complete. 2024-05-10T10:44:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 -- start start 2024-05-10T10:44:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.505+0000 7fb0b79fb740 1 Processor -- start 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x5577503742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x557750374830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036fd20 0x55775037b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557750372c10 con 0x55775036a620 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557750287370 con 0x55775036fd20 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557750372d90 con 0x55775036b010 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0b79fb740 10 client.0 _ll_register_callbacks cb 0x55774ffdba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x5577503742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x5577503742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40030/0 (socket says 172.21.15.112:40030) 2024-05-10T10:44:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2960610059 learned_addr learned my addr 172.21.15.112:0/2960610059 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x557750374830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0aed9d640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036fd20 0x55775037b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 msgr2=0x557750374830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x557750374830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2960610059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036fd20 msgr2=0x55775037b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036fd20 0x55775037b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2960610059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577500a8620 con 0x55775036a620 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x557750374830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x5577503742f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000eff0 tx=0x7fb0a00a8ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a7624fd5b2b788 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.506+0000 7fb095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb095ffb640 1 -- 172.21.15.112:0/2960610059 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00b26e0 con 0x55775036a620 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb095ffb640 1 -- 172.21.15.112:0/2960610059 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb0a001ad30 con 0x55775036a620 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5577502d81e0 con 0x55775036a620 2024-05-10T10:44:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb095ffb640 1 -- 172.21.15.112:0/2960610059 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb0a00e2300 con 0x55775036a620 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 10 client.16413 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55775037e850 con 0x55775036a620 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb095ffb640 1 -- 172.21.15.112:0/2960610059 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5577502d81e0 con 0x55775036a620 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb095ffb640 1 -- 172.21.15.112:0/2960610059 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fb0a00b2250 con 0x55775036a620 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 10 client.16413 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 20 client.16413 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x557750381f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 4 client.16413 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577502d2050 con 0x55775037fba0 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.507+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x557750381f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.508+0000 7fb095ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.508+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x557750381f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f97fe8b8932b189 server_cookie=974d59ddd15afaea in_seq=0 out_seq=0 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb095ffb640 1 -- 172.21.15.112:0/2960610059 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577502d2050 con 0x55775037fba0 2024-05-10T10:44:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb095ffb640 10 client.16413 handle_command_reply: tid=0 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 client.16413 shutdown 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 msgr2=0x5577503742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x5577503742f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000eff0 tx=0x7fb0a00a8ff0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 shutdown_connections 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x557750381f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb095ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036fd20 0x55775037b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x557750374830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2960610059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036a620 0x5577503742f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 >> 172.21.15.112:0/2960610059 conn(0x557750362960 msgr2=0x557750360580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 shutdown_connections 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2960610059 wait complete. 2024-05-10T10:44:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.509+0000 7fb0b79fb740 20 client.16413 trim_cache size 0 max 16384 2024-05-10T10:44:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 Processor -- start 2024-05-10T10:44:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 -- start start 2024-05-10T10:44:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036a620 0x557750387840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x55775037b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x557750387d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01a91d0 con 0x55775036fd20 2024-05-10T10:44:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01aae90 con 0x55775036a620 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01a98a0 con 0x55775036b010 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036a620 0x557750387840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x557750387d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x55775037b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036a620 0x557750387840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55220/0 (socket says 172.21.15.112:55220) 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x557750387d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40042/0 (socket says 172.21.15.112:40042) 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x55775037b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:38830/0 (socket says 172.21.15.112:38830) 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.526+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2079833869 learned_addr learned my addr 172.21.15.112:0/2079833869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2079833869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 msgr2=0x55775037b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/2079833869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x55775037b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2079833869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 msgr2=0x557750387d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/2079833869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x557750387d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb0ae59c640 1 -- 172.21.15.112:0/2079833869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557750362790 con 0x55775036a620 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/2079833869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036a620 0x557750387840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0064040 tx=0x7fb0a00a6000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcbbbc5e9face5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb097fff640 1 -- 172.21.15.112:0/2079833869 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00b5150 con 0x55775036a620 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb097fff640 1 -- 172.21.15.112:0/2079833869 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a00adc30 con 0x55775036a620 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.527+0000 7fb097fff640 1 -- 172.21.15.112:0/2079833869 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00cc1c0 con 0x55775036a620 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2079833869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036a620 msgr2=0x557750387840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2079833869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036a620 0x557750387840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0064040 tx=0x7fb0a00a6000 comp rx=0 tx=0).stop 2024-05-10T10:44:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2079833869 shutdown_connections 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2079833869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x557750387d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2079833869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036b010 0x55775037b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/2079833869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036a620 0x557750387840 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2079833869 >> 172.21.15.112:0/2079833869 conn(0x557750362960 msgr2=0x55775036bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2079833869 shutdown_connections 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 -- 172.21.15.112:0/2079833869 wait complete. 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 -- start start 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 Processor -- start 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036a620 0x557750378e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.528+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x5577503793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0b79fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x55775037a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01aa700 con 0x55775036fd20 2024-05-10T10:44:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01aa030 con 0x55775036b010 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0b79fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01a71b0 con 0x55775036a620 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0b79fb740 10 client.0 _ll_register_callbacks cb 0x7fb0a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x5577503793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x5577503793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55230/0 (socket says 172.21.15.112:55230) 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0a7fff640 1 -- 172.21.15.112:0/3590345491 learned_addr learned my addr 172.21.15.112:0/3590345491 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036a620 0x557750378e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0ae59c640 1 -- 172.21.15.112:0/3590345491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 msgr2=0x5577503793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x5577503793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0ae59c640 1 -- 172.21.15.112:0/3590345491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 msgr2=0x55775037a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.529+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x55775037a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x55775037a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb0ae59c640 1 -- 172.21.15.112:0/3590345491 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775037a9d0 con 0x55775036a620 2024-05-10T10:44:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb0aed9d640 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x55775037a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:44:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb0ae59c640 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036a620 0x557750378e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a5900 tx=0x7fb0a0010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ca5d7ea8587e01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb095ffb640 1 -- 172.21.15.112:0/3590345491 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00a9650 con 0x55775036a620 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a00adc30 con 0x55775036a620 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb095ffb640 1 -- 172.21.15.112:0/3590345491 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb0a00aabb0 con 0x55775036a620 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb095ffb640 1 -- 172.21.15.112:0/3590345491 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb0a00a8ff0 con 0x55775036a620 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb0b79fb740 10 client.25828 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55775037a9d0 con 0x55775036a620 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.531+0000 7fb095ffb640 1 -- 172.21.15.112:0/3590345491 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a00adc30 con 0x55775036a620 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb095ffb640 1 -- 172.21.15.112:0/3590345491 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fb0a00cac30 con 0x55775036a620 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0b79fb740 10 client.25828 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0b79fb740 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0b79fb740 20 client.25828 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x5577503eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0b79fb740 4 client.25828 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:44:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55775024d5f0 con 0x55775037fba0 2024-05-10T10:44:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x5577503eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb095ffb640 10 client.25828 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.532+0000 7fb0a7fff640 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x5577503eca10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6356e3b6319ce1 server_cookie=3b7a5274c98e7521 in_seq=0 out_seq=0 2024-05-10T10:44:17.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:17 smithi148 ceph-mon[33568]: pgmap v551: 129 pgs: 129 active+clean; 41 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 338 op/s 2024-05-10T10:44:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.544+0000 7fb095ffb640 1 -- 172.21.15.112:0/3590345491 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55775024d5f0 con 0x55775037fba0 2024-05-10T10:44:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb095ffb640 10 client.25828 handle_command_reply: tid=0 2024-05-10T10:44:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 client.25828 shutdown 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036a620 msgr2=0x557750378e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036a620 0x557750378e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a5900 tx=0x7fb0a0010ca0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 shutdown_connections 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775036fd20 0x55775037a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55775037fba0 0x5577503eca10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55775036b010 0x5577503793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb095ffb640 0 client.25828 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 --2- 172.21.15.112:0/3590345491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55775036a620 0x557750378e90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 >> 172.21.15.112:0/3590345491 conn(0x557750362960 msgr2=0x557750373f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 shutdown_connections 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 1 -- 172.21.15.112:0/3590345491 wait complete. 2024-05-10T10:44:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.545+0000 7fb0b79fb740 20 client.25828 trim_cache size 0 max 16384 2024-05-10T10:44:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:44:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:44:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:44:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb09c0b2920 msgr2=0x7fb09c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb09c0b2920 0x7fb09c0b4de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005c40 tx=0x7fb0a8009bb0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b0101080 msgr2=0x7fb0b01968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b0101080 0x7fb0b01968e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000e9c0 tx=0x7fb0a000ee80 comp rx=0 tx=0).stop 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 shutdown_connections 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fb09c0b2920 0x7fb09c0b4de0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0102c30 0x7fb0b019de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b0102270 0x7fb0b0196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 --2- 172.21.15.112:0/3078996568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0b0101080 0x7fb0b01968e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 >> 172.21.15.112:0/3078996568 conn(0x7fb0b00fc610 msgr2=0x7fb0b0105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 shutdown_connections 2024-05-10T10:44:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:17.546+0000 7fb0b4cda640 1 -- 172.21.15.112:0/3078996568 wait complete. 2024-05-10T10:44:17.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:17 smithi195 ceph-mon[33773]: pgmap v551: 129 pgs: 129 active+clean; 41 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 338 op/s 2024-05-10T10:44:19.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:19 smithi148 ceph-mon[33568]: pgmap v552: 129 pgs: 129 active+clean; 42 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 378 op/s 2024-05-10T10:44:19.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:19 smithi195 ceph-mon[33773]: pgmap v552: 129 pgs: 129 active+clean; 42 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 378 op/s 2024-05-10T10:44:19.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:19 smithi112 ceph-mon[30100]: pgmap v552: 129 pgs: 129 active+clean; 42 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 378 op/s 2024-05-10T10:44:21.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:21 smithi148 ceph-mon[33568]: pgmap v553: 129 pgs: 129 active+clean; 42 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-05-10T10:44:21.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:21 smithi195 ceph-mon[33773]: pgmap v553: 129 pgs: 129 active+clean; 42 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-05-10T10:44:21.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:21 smithi112 ceph-mon[30100]: pgmap v553: 129 pgs: 129 active+clean; 42 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-05-10T10:44:23.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:23 smithi148 ceph-mon[33568]: pgmap v554: 129 pgs: 129 active+clean; 39 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 375 op/s 2024-05-10T10:44:23.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:23 smithi195 ceph-mon[33773]: pgmap v554: 129 pgs: 129 active+clean; 39 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 375 op/s 2024-05-10T10:44:23.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:23 smithi112 ceph-mon[30100]: pgmap v554: 129 pgs: 129 active+clean; 39 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 375 op/s 2024-05-10T10:44:25.381 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:25 smithi112 ceph-mon[30100]: pgmap v555: 129 pgs: 129 active+clean; 39 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 460 op/s 2024-05-10T10:44:25.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:25 smithi148 ceph-mon[33568]: pgmap v555: 129 pgs: 129 active+clean; 39 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 460 op/s 2024-05-10T10:44:25.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:25 smithi195 ceph-mon[33773]: pgmap v555: 129 pgs: 129 active+clean; 39 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 460 op/s 2024-05-10T10:44:27.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:27 smithi148 ceph-mon[33568]: pgmap v556: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-05-10T10:44:27.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:27 smithi195 ceph-mon[33773]: pgmap v556: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-05-10T10:44:27.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:27 smithi112 ceph-mon[30100]: pgmap v556: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-05-10T10:44:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:29 smithi148 ceph-mon[33568]: pgmap v557: 129 pgs: 129 active+clean; 40 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 374 op/s 2024-05-10T10:44:29.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:29 smithi195 ceph-mon[33773]: pgmap v557: 129 pgs: 129 active+clean; 40 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 374 op/s 2024-05-10T10:44:29.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:29 smithi112 ceph-mon[30100]: pgmap v557: 129 pgs: 129 active+clean; 40 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 374 op/s 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 -- 172.21.15.112:0/2305112888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc102a20 msgr2=0x7f41fc102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 --2- 172.21.15.112:0/2305112888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc102a20 0x7f41fc102e40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009900 tx=0x7f41ec02a410 comp rx=0 tx=0).stop 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 -- 172.21.15.112:0/2305112888 shutdown_connections 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 --2- 172.21.15.112:0/2305112888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc1045d0 0x7f41fc10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 --2- 172.21.15.112:0/2305112888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41fc103c10 0x7f41fc104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 --2- 172.21.15.112:0/2305112888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc102a20 0x7f41fc102e40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 -- 172.21.15.112:0/2305112888 >> 172.21.15.112:0/2305112888 conn(0x7f41fc0fdf90 msgr2=0x7f41fc1003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 -- 172.21.15.112:0/2305112888 shutdown_connections 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 -- 172.21.15.112:0/2305112888 wait complete. 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.647+0000 7f4201578640 1 Processor -- start 2024-05-10T10:44:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f4201578640 1 -- start start 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f4201578640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc102a20 0x7f41fc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f4201578640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc103c10 0x7f41fc199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f4201578640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41fc1045d0 0x7f41fc1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f4201578640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10e3d0 con 0x7f41fc103c10 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f4201578640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10e250 con 0x7f41fc102a20 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f4201578640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10e550 con 0x7f41fc1045d0 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc103c10 0x7f41fc199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc103c10 0x7f41fc199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48348/0 (socket says 172.21.15.112:48348) 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41faffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc102a20 0x7f41fc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41fa7fc640 1 -- 172.21.15.112:0/1979878785 learned_addr learned my addr 172.21.15.112:0/1979878785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41fc1045d0 0x7f41fc1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41faffd640 1 -- 172.21.15.112:0/1979878785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41fc1045d0 msgr2=0x7f41fc1a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41faffd640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41fc1045d0 0x7f41fc1a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41faffd640 1 -- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc103c10 msgr2=0x7f41fc199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41faffd640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc103c10 0x7f41fc199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41faffd640 1 -- 172.21.15.112:0/1979878785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc1a0700 con 0x7f41fc102a20 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc103c10 0x7f41fc199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.648+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41fc1045d0 0x7f41fc1a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.649+0000 7f41faffd640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc102a20 0x7f41fc198b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009230 tx=0x7f41ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.649+0000 7f41e3fff640 1 -- 172.21.15.112:0/1979878785 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec035050 con 0x7f41fc102a20 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.649+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc074be0 con 0x7f41fc102a20 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.649+0000 7f41e3fff640 1 -- 172.21.15.112:0/1979878785 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41ec032e10 con 0x7f41fc102a20 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.649+0000 7f41e3fff640 1 -- 172.21.15.112:0/1979878785 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec003a80 con 0x7f41fc102a20 2024-05-10T10:44:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.649+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc0750f0 con 0x7f41fc102a20 2024-05-10T10:44:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.651+0000 7f420436d740 1 Processor -- start 2024-05-10T10:44:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.651+0000 7f420436d740 1 -- start start 2024-05-10T10:44:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.651+0000 7f41e3fff640 1 -- 172.21.15.112:0/1979878785 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f41ec03ac70 con 0x7f41fc102a20 2024-05-10T10:44:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f420436d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f420436d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d6b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f420436d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d6dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f420436d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555748cca5d0 con 0x555748d65db0 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f420436d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555748c83890 con 0x555748d6b910 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f420436d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555748cc6590 con 0x555748d667a0 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48364/0 (socket says 172.21.15.112:48364) 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 -- 172.21.15.112:0/2084811535 learned_addr learned my addr 172.21.15.112:0/2084811535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d6dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41faffd640 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d6b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 -- 172.21.15.112:0/2084811535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 msgr2=0x555748d6b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d6b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 -- 172.21.15.112:0/2084811535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 msgr2=0x555748d6dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d6dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fb7fe640 1 -- 172.21.15.112:0/2084811535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557489dc210 con 0x555748d65db0 2024-05-10T10:44:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.652+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d6dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.653+0000 7f41e3fff640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41e40b28a0 0x7f41e40b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.653+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d661d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f41f40117e0 tx=0x7f41f4011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d0a57274fc2f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.653+0000 7f41faffd640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41e40b28a0 0x7f41e40b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.653+0000 7f41e3fff640 1 -- 172.21.15.112:0/1979878785 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f41ec106600 con 0x7f41fc102a20 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.653+0000 7f41e17fa640 1 -- 172.21.15.112:0/2084811535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011310 con 0x555748d65db0 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.653+0000 7f41e17fa640 1 -- 172.21.15.112:0/2084811535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41f4003e00 con 0x555748d65db0 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.653+0000 7f41e17fa640 1 -- 172.21.15.112:0/2084811535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4004430 con 0x555748d65db0 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 -- 172.21.15.112:0/2084811535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 msgr2=0x555748d661d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f41faffd640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41e40b28a0 0x7f41e40b4d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f41ec1040e0 tx=0x7f41ec10c550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d661d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f41f40117e0 tx=0x7f41f4011ca0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 -- 172.21.15.112:0/2084811535 shutdown_connections 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d6dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d6b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 --2- 172.21.15.112:0/2084811535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d661d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 -- 172.21.15.112:0/2084811535 >> 172.21.15.112:0/2084811535 conn(0x555748d61420 msgr2=0x555748d61820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 -- 172.21.15.112:0/2084811535 shutdown_connections 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.654+0000 7f420436d740 1 -- 172.21.15.112:0/2084811535 wait complete. 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 -- start start 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 Processor -- start 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d65db0 0x555748d70020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d70560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d6b910 0x555748d775a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555748d6e8c0 con 0x555748d6b910 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555748c87370 con 0x555748d65db0 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555748d6ea40 con 0x555748d667a0 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d70560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d65db0 0x555748d70020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d70560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:53978/0 (socket says 172.21.15.112:53978) 2024-05-10T10:44:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 -- 172.21.15.112:0/757591063 learned_addr learned my addr 172.21.15.112:0/757591063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f420436d740 10 client.0 _ll_register_callbacks cb 0x5557489dba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d6b910 0x555748d775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 -- 172.21.15.112:0/757591063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d65db0 msgr2=0x555748d70020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d65db0 0x555748d70020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 -- 172.21.15.112:0/757591063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d6b910 msgr2=0x555748d775a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d6b910 0x555748d775a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.655+0000 7f41fa7fc640 1 -- 172.21.15.112:0/757591063 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555748aa8620 con 0x555748d667a0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d70560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f41f0012d20 tx=0x7f41f000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6066b5bdecb22c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f41deff5640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f41deff5640 1 -- 172.21.15.112:0/757591063 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00120d0 con 0x555748d667a0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f4003e00 con 0x555748d667a0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f41deff5640 1 -- 172.21.15.112:0/757591063 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f41f0004030 con 0x555748d667a0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f41deff5640 1 -- 172.21.15.112:0/757591063 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f41f0012270 con 0x555748d667a0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f420436d740 10 client.25840 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555748d7a680 con 0x555748d667a0 2024-05-10T10:44:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f41deff5640 1 -- 172.21.15.112:0/757591063 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41f4003e00 con 0x555748d667a0 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f41deff5640 1 -- 172.21.15.112:0/757591063 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f41f00047a0 con 0x555748d667a0 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f420436d740 10 client.25840 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f420436d740 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f420436d740 20 client.25840 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.656+0000 7f420436d740 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748d7be30 0x555748d7e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.657+0000 7f420436d740 4 client.25840 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.657+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555748cd81e0 con 0x555748d7be30 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.657+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748d7be30 0x555748d7e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.657+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748d7be30 0x555748d7e270 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39630abac51f8938 server_cookie=396245105fc0266e in_seq=0 out_seq=0 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.657+0000 7f41deff5640 10 client.25840 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f41deff5640 1 -- 172.21.15.112:0/757591063 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555748cd81e0 con 0x555748d7be30 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f41deff5640 10 client.25840 handle_command_reply: tid=0 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 client.25840 shutdown 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 msgr2=0x555748d70560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d70560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f41f0012d20 tx=0x7f41f000f790 comp rx=0 tx=0).stop 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 shutdown_connections 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d6b910 0x555748d775a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d70560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748d7be30 0x555748d7e270 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 --2- 172.21.15.112:0/757591063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d65db0 0x555748d70020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f41deff5640 0 client.25840 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 >> 172.21.15.112:0/757591063 conn(0x555748d61420 msgr2=0x555748d60580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 shutdown_connections 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 1 -- 172.21.15.112:0/757591063 wait complete. 2024-05-10T10:44:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.658+0000 7f420436d740 20 client.25840 trim_cache size 0 max 16384 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.674+0000 7f420436d740 1 Processor -- start 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.674+0000 7f420436d740 1 -- start start 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f420436d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d65db0 0x555748d642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f420436d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d667a0 0x555748d77560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f420436d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d64810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f420436d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1a91b0 con 0x555748d667a0 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f420436d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1aae70 con 0x555748d6b910 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f420436d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1a9880 con 0x555748d65db0 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d65db0 0x555748d642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d667a0 0x555748d77560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d65db0 0x555748d642d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:53992/0 (socket says 172.21.15.112:53992) 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fb7fe640 1 -- 172.21.15.112:0/1490653214 learned_addr learned my addr 172.21.15.112:0/1490653214 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d667a0 0x555748d77560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48372/0 (socket says 172.21.15.112:48372) 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d64810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 -- 172.21.15.112:0/1490653214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d65db0 msgr2=0x555748d642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d65db0 0x555748d642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 -- 172.21.15.112:0/1490653214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 msgr2=0x555748d64810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d64810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 -- 172.21.15.112:0/1490653214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555748d7e8f0 con 0x555748d667a0 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d65db0 0x555748d642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.675+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d667a0 0x555748d77560 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f41f0004c60 tx=0x7f41f0012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9f1f53470936e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f41e0ff9640 1 -- 172.21.15.112:0/1490653214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f002e070 con 0x555748d667a0 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f41e0ff9640 1 -- 172.21.15.112:0/1490653214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41f000fe80 con 0x555748d667a0 2024-05-10T10:44:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f41e0ff9640 1 -- 172.21.15.112:0/1490653214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00039e0 con 0x555748d667a0 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 -- 172.21.15.112:0/1490653214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d667a0 msgr2=0x555748d77560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d667a0 0x555748d77560 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f41f0004c60 tx=0x7f41f0012910 comp rx=0 tx=0).stop 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 -- 172.21.15.112:0/1490653214 shutdown_connections 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d64810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d667a0 0x555748d77560 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 --2- 172.21.15.112:0/1490653214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d65db0 0x555748d642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 -- 172.21.15.112:0/1490653214 >> 172.21.15.112:0/1490653214 conn(0x555748d61420 msgr2=0x555748d67a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.676+0000 7f420436d740 1 -- 172.21.15.112:0/1490653214 shutdown_connections 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 -- 172.21.15.112:0/1490653214 wait complete. 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 -- start start 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 Processor -- start 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d74790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d74cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d75c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1aa6e0 con 0x555748d65db0 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1aa010 con 0x555748d6b910 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1a7190 con 0x555748d667a0 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f420436d740 10 client.0 _ll_register_callbacks cb 0x7f41f0009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d74790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d74790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48388/0 (socket says 172.21.15.112:48388) 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 -- 172.21.15.112:0/1147445981 learned_addr learned my addr 172.21.15.112:0/1147445981 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d74cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d75c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 -- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 msgr2=0x555748d74cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d74cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 -- 172.21.15.112:0/1147445981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 msgr2=0x555748d75c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d75c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.677+0000 7f41fb7fe640 1 -- 172.21.15.112:0/1147445981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555748d76260 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f41fb7fe640 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d74790 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x555748d67000 tx=0x7f41f4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce8aa4a8a689af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f41deff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f41deff5640 1 -- 172.21.15.112:0/1147445981 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4035040 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f41deff5640 1 -- 172.21.15.112:0/1147445981 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f41f400fba0 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555748cd2050 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f41deff5640 1 -- 172.21.15.112:0/1147445981 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f41f400fda0 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 10 client.16434 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555748d77570 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f41deff5640 1 -- 172.21.15.112:0/1147445981 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555748cd2050 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f41deff5640 1 -- 172.21.15.112:0/1147445981 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f41f4028040 con 0x555748d65db0 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 10 client.16434 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 20 client.16434 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748de6b40 0x555748de8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 4 client.16434 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:44:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.678+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555748c4d5f0 con 0x555748de6b40 2024-05-10T10:44:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.679+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748de6b40 0x555748de8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.679+0000 7f41fa7fc640 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748de6b40 0x555748de8f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287ce5701d834f34 server_cookie=8592d99ac1cf4ab7 in_seq=0 out_seq=0 2024-05-10T10:44:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.679+0000 7f41deff5640 10 client.16434 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f41deff5640 1 -- 172.21.15.112:0/1147445981 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555748c4d5f0 con 0x555748de6b40 2024-05-10T10:44:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f41deff5640 10 client.16434 handle_command_reply: tid=0 2024-05-10T10:44:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 client.16434 shutdown 2024-05-10T10:44:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 msgr2=0x555748d74790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d74790 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x555748d67000 tx=0x7f41f4009fd0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 shutdown_connections 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x555748de6b40 0x555748de8f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555748d6b910 0x555748d75c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555748d667a0 0x555748d74cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f41deff5640 0 client.16434 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 --2- 172.21.15.112:0/1147445981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555748d65db0 0x555748d74790 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.697+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 >> 172.21.15.112:0/1147445981 conn(0x555748d61420 msgr2=0x555748d642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 shutdown_connections 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f420436d740 1 -- 172.21.15.112:0/1147445981 wait complete. 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f420436d740 20 client.16434 trim_cache size 0 max 16384 2024-05-10T10:44:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:44:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:44:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:44:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41e40b28a0 msgr2=0x7f41e40b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41e40b28a0 0x7f41e40b4d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f41ec1040e0 tx=0x7f41ec10c550 comp rx=0 tx=0).stop 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc102a20 msgr2=0x7f41fc198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc102a20 0x7f41fc198b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009230 tx=0x7f41ec004270 comp rx=0 tx=0).stop 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 shutdown_connections 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f41e40b28a0 0x7f41e40b4d60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41fc1045d0 0x7f41fc1a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41fc103c10 0x7f41fc199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 --2- 172.21.15.112:0/1979878785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc102a20 0x7f41fc198b30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.698+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 >> 172.21.15.112:0/1979878785 conn(0x7f41fc0fdf90 msgr2=0x7f41fc107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.699+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 shutdown_connections 2024-05-10T10:44:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:30.699+0000 7f4201578640 1 -- 172.21.15.112:0/1979878785 wait complete. 2024-05-10T10:44:31.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:44:31.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:31 smithi112 ceph-mon[30100]: pgmap v558: 129 pgs: 129 active+clean; 40 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 444 op/s 2024-05-10T10:44:31.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:31 smithi148 ceph-mon[33568]: pgmap v558: 129 pgs: 129 active+clean; 40 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 444 op/s 2024-05-10T10:44:31.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:31 smithi195 ceph-mon[33773]: pgmap v558: 129 pgs: 129 active+clean; 40 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 444 op/s 2024-05-10T10:44:31.631 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:32.333 INFO:tasks.workunit.client.0.smithi112.stdout:50000 write 0x39280 thru 0x48ee3 (0xfc64 bytes) 2024-05-10T10:44:32.334 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-05-10T10:44:32.334 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-05-10T10:44:32.336 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-05-10T10:44:32.467 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-10T10:44:32.467 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-10T10:44:32.467 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-05-10T10:44:32.467 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-10T10:44:32.470 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-05-10T10:44:32.472 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x81c806 2024-05-10T10:44:32.580 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9bc582 2024-05-10T10:44:32.599 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9eecbc 2024-05-10T10:44:33.441 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:33 smithi112 ceph-mon[30100]: pgmap v559: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-05-10T10:44:33.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:33 smithi148 ceph-mon[33568]: pgmap v559: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-05-10T10:44:33.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 -- 172.21.15.112:0/1016337874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc109540 msgr2=0x7f59cc109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/1016337874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc109540 0x7f59cc109920 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009990 tx=0x7f59c8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 -- 172.21.15.112:0/1016337874 shutdown_connections 2024-05-10T10:44:33.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/1016337874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 0x7f59cc110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/1016337874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc109540 0x7f59cc109920 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/1016337874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59cc06bb70 0x7f59cc06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 -- 172.21.15.112:0/1016337874 >> 172.21.15.112:0/1016337874 conn(0x7f59cc0793e0 msgr2=0x7f59cc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:33.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 -- 172.21.15.112:0/1016337874 shutdown_connections 2024-05-10T10:44:33.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.564+0000 7f59d3f1e700 1 -- 172.21.15.112:0/1016337874 wait complete. 2024-05-10T10:44:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:33 smithi195 ceph-mon[33773]: pgmap v559: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 Processor -- start 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 -- start start 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc06bb70 0x7f59cc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 0x7f59cc19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59cc109540 0x7f59cc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc113550 con 0x7f59cc06bb70 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1133d0 con 0x7f59cc109540 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d3f1e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1136d0 con 0x7f59cc06c530 2024-05-10T10:44:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d1cba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc06bb70 0x7f59cc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d14b9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 0x7f59cc19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d14b9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 0x7f59cc19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:54020/0 (socket says 172.21.15.112:54020) 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d14b9700 1 -- 172.21.15.112:0/2071803058 learned_addr learned my addr 172.21.15.112:0/2071803058 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.565+0000 7f59d24bb700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59cc109540 0x7f59cc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d14b9700 1 -- 172.21.15.112:0/2071803058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59cc109540 msgr2=0x7f59cc1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d14b9700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59cc109540 0x7f59cc1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d14b9700 1 -- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc06bb70 msgr2=0x7f59cc19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d14b9700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc06bb70 0x7f59cc19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d14b9700 1 -- 172.21.15.112:0/2071803058 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc19a050 con 0x7f59cc06c530 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d1cba700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc06bb70 0x7f59cc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d14b9700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 0x7f59cc19fc70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00e9a0 tx=0x7f59bc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59c2ffd700 1 -- 172.21.15.112:0/2071803058 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00c9c0 con 0x7f59cc06c530 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc19a340 con 0x7f59cc06c530 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59c2ffd700 1 -- 172.21.15.112:0/2071803058 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59bc010430 con 0x7f59cc06c530 2024-05-10T10:44:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59c2ffd700 1 -- 172.21.15.112:0/2071803058 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc018680 con 0x7f59cc06c530 2024-05-10T10:44:33.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.566+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1a6600 con 0x7f59cc06c530 2024-05-10T10:44:33.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.571+0000 7f59c2ffd700 1 -- 172.21.15.112:0/2071803058 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f59bc00cb60 con 0x7f59cc06c530 2024-05-10T10:44:33.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.571+0000 7f59de7b1b80 1 Processor -- start 2024-05-10T10:44:33.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.571+0000 7f59de7b1b80 1 -- start start 2024-05-10T10:44:33.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8ed190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c57b83e940 con 0x55c57b8ecd50 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c57b7a3da0 con 0x55c57b8f2ad0 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c57b83a900 con 0x55c57b8f21b0 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:54032/0 (socket says 172.21.15.112:54032) 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 -- 172.21.15.112:0/1328789362 learned_addr learned my addr 172.21.15.112:0/1328789362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59c2ffd700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f59b80b6c90 0x7f59b80b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d14b9700 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8ed190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d1cba700 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 -- 172.21.15.112:0/1328789362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 msgr2=0x55c57b8ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8ec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 -- 172.21.15.112:0/1328789362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 msgr2=0x55c57b8ed190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8ed190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 -- 172.21.15.112:0/1328789362 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57b78b8d0 con 0x55c57b8f21b0 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d24bb700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f59b80b6c90 0x7f59b80b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59d1cba700 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.572+0000 7f59c2ffd700 1 -- 172.21.15.112:0/2071803058 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f59bc0e2710 con 0x7f59cc06c530 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.573+0000 7f59d24bb700 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f2590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59c4011740 tx=0x7f59c4011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768db997caf22fca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.573+0000 7f59b7fff700 1 -- 172.21.15.112:0/1328789362 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c401c030 con 0x55c57b8f21b0 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.573+0000 7f59b7fff700 1 -- 172.21.15.112:0/1328789362 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59c4003dd0 con 0x55c57b8f21b0 2024-05-10T10:44:33.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.573+0000 7f59b7fff700 1 -- 172.21.15.112:0/1328789362 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4018450 con 0x55c57b8f21b0 2024-05-10T10:44:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1328789362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 msgr2=0x55c57b8f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59d24bb700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f59b80b6c90 0x7f59b80b9150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f59c4020000 tx=0x7f59c4021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f2590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59c4011740 tx=0x7f59c4011a50 comp rx=0 tx=0).stop 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1328789362 shutdown_connections 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8ed190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f2590 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1328789362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1328789362 >> 172.21.15.112:0/1328789362 conn(0x55c57b818b70 msgr2=0x55c57b8e72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1328789362 shutdown_connections 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1328789362 wait complete. 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 -- start start 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.574+0000 7f59de7b1b80 1 Processor -- start 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c57b8f93f0 con 0x55c57b8ecd50 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c57b7f7ed0 con 0x55c57b8f2ad0 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c57b8f9570 con 0x55c57b8f21b0 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59de7b1b80 10 client.0 _ll_register_callbacks cb 0x55c57b7d25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d1cba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d14b9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d1cba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8f8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48418/0 (socket says 172.21.15.112:48418) 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d1cba700 1 -- 172.21.15.112:0/3632863945 learned_addr learned my addr 172.21.15.112:0/3632863945 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d14b9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:54036/0 (socket says 172.21.15.112:54036) 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d24bb700 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d14b9700 1 -- 172.21.15.112:0/3632863945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 msgr2=0x55c57b8f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d14b9700 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d14b9700 1 -- 172.21.15.112:0/3632863945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 msgr2=0x55c57b8f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d14b9700 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d14b9700 1 -- 172.21.15.112:0/3632863945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57b7f5760 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.575+0000 7f59d24bb700 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59d14b9700 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f5a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0dfe50 tx=0x7f59bc0a9c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=629680ee037cbc2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59b57fa700 1 -- 172.21.15.112:0/3632863945 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0a5860 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59b57fa700 1 -- 172.21.15.112:0/3632863945 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f59bc0e2440 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c4003dd0 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59b57fa700 1 -- 172.21.15.112:0/3632863945 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f59bc0b2140 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 10 client.25858 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c57b903c50 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59b57fa700 1 -- 172.21.15.112:0/3632863945 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59c4003dd0 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59b57fa700 1 -- 172.21.15.112:0/3632863945 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f59bc0cc0e0 con 0x55c57b8f21b0 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 10 client.25858 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 20 client.25858 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b9061e0 0x55c57b908610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 4 client.25858 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.576+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57b84c320 con 0x55c57b9061e0 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.577+0000 7f59d1cba700 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b9061e0 0x55c57b908610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.577+0000 7f59b57fa700 10 client.25858 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.577+0000 7f59d1cba700 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b9061e0 0x55c57b908610 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214d8d78cfd033ca server_cookie=98906e7382476fec in_seq=0 out_seq=0 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59b57fa700 1 -- 172.21.15.112:0/3632863945 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c57b84c320 con 0x55c57b9061e0 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59b57fa700 10 client.25858 handle_command_reply: tid=0 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 client.25858 shutdown 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 msgr2=0x55c57b8f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f5a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0dfe50 tx=0x7f59bc0a9c00 comp rx=0 tx=0).stop 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 shutdown_connections 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b9061e0 0x55c57b908610 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8ecd50 0x55c57b8f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59b57fa700 0 client.25858 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3632863945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8f21b0 0x55c57b8f5a10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 >> 172.21.15.112:0/3632863945 conn(0x55c57b818b70 msgr2=0x55c57b8e1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 shutdown_connections 2024-05-10T10:44:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3632863945 wait complete. 2024-05-10T10:44:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.578+0000 7f59de7b1b80 20 client.25858 trim_cache size 0 max 16384 2024-05-10T10:44:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 Processor -- start 2024-05-10T10:44:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 -- start start 2024-05-10T10:44:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b902480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f21b0 0x55c57b8e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1ad030 con 0x55c57b8f21b0 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1aecf0 con 0x55c57b8f2ad0 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1ad700 con 0x55c57b8ecd50 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f21b0 0x55c57b8e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f21b0 0x55c57b8e1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48420/0 (socket says 172.21.15.112:48420) 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 -- 172.21.15.112:0/3316256546 learned_addr learned my addr 172.21.15.112:0/3316256546 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d1cba700 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b902480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d24bb700 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 -- 172.21.15.112:0/3316256546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 msgr2=0x55c57b902480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b902480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 -- 172.21.15.112:0/3316256546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 msgr2=0x55c57b8f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.603+0000 7f59d14b9700 1 -- 172.21.15.112:0/3316256546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57b908c90 con 0x55c57b8f21b0 2024-05-10T10:44:33.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.604+0000 7f59d14b9700 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f21b0 0x55c57b8e1c40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0dfe80 tx=0x7f59bc0a9d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89abca24b451d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:33.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.604+0000 7f59b77fe700 1 -- 172.21.15.112:0/3316256546 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0a5860 con 0x55c57b8f21b0 2024-05-10T10:44:33.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.604+0000 7f59b77fe700 1 -- 172.21.15.112:0/3316256546 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59bc0a94b0 con 0x55c57b8f21b0 2024-05-10T10:44:33.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.604+0000 7f59b77fe700 1 -- 172.21.15.112:0/3316256546 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0e2440 con 0x55c57b8f21b0 2024-05-10T10:44:33.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3316256546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f21b0 msgr2=0x55c57b8e1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f21b0 0x55c57b8e1c40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0dfe80 tx=0x7f59bc0a9d20 comp rx=0 tx=0).stop 2024-05-10T10:44:33.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3316256546 shutdown_connections 2024-05-10T10:44:33.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f2ad0 0x55c57b8f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b902480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/3316256546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f21b0 0x55c57b8e1c40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3316256546 >> 172.21.15.112:0/3316256546 conn(0x55c57b818b70 msgr2=0x55c57b8f4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3316256546 shutdown_connections 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.605+0000 7f59de7b1b80 1 -- 172.21.15.112:0/3316256546 wait complete. 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 -- start start 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 Processor -- start 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b8fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f21b0 0x55c57b90d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f2ad0 0x55c57b90da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1ae560 con 0x55c57b8f2ad0 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1ade90 con 0x55c57b8f21b0 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1aafd0 con 0x55c57b8ecd50 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59de7b1b80 10 client.0 _ll_register_callbacks cb 0x55c57b7d25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:33.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59d24bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f2ad0 0x55c57b90da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59d14b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f21b0 0x55c57b90d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59d24bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f2ad0 0x55c57b90da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48422/0 (socket says 172.21.15.112:48422) 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.606+0000 7f59d24bb700 1 -- 172.21.15.112:0/1406391471 learned_addr learned my addr 172.21.15.112:0/1406391471 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d1cba700 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b8fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d24bb700 1 -- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 msgr2=0x55c57b8fee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d24bb700 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b8fee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d24bb700 1 -- 172.21.15.112:0/1406391471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f21b0 msgr2=0x55c57b90d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d24bb700 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f21b0 0x55c57b90d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d24bb700 1 -- 172.21.15.112:0/1406391471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57b90e2d0 con 0x55c57b8f2ad0 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d14b9700 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f21b0 0x55c57b90d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59d24bb700 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f2ad0 0x55c57b90da90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009b50 tx=0x7f59c4009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15ae672c836395b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:33.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59b57fa700 1 -- 172.21.15.112:0/1406391471 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4031070 con 0x55c57b8f2ad0 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59b57fa700 1 -- 172.21.15.112:0/1406391471 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f59c4018840 con 0x55c57b8f2ad0 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59bc0a94b0 con 0x55c57b8f2ad0 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.607+0000 7f59b57fa700 1 -- 172.21.15.112:0/1406391471 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f59c403a050 con 0x55c57b8f2ad0 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.608+0000 7f59de7b1b80 10 client.16452 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.608+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c57b970b90 con 0x55c57b8f2ad0 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.608+0000 7f59b57fa700 1 -- 172.21.15.112:0/1406391471 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59c4003dd0 con 0x55c57b8f2ad0 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.608+0000 7f59b57fa700 1 -- 172.21.15.112:0/1406391471 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f59c4034020 con 0x55c57b8f2ad0 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.608+0000 7f59de7b1b80 10 client.16452 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.608+0000 7f59de7b1b80 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.609+0000 7f59de7b1b80 20 client.16452 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.609+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b971140 0x55c57b973560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.609+0000 7f59de7b1b80 4 client.16452 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.609+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f59bc0a94b0 con 0x55c57b971140 2024-05-10T10:44:33.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.609+0000 7f59d1cba700 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b971140 0x55c57b973560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.609+0000 7f59d1cba700 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b971140 0x55c57b973560 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2bd9bf411dc29b server_cookie=8a5e261d2cba34db in_seq=0 out_seq=0 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59b57fa700 10 client.16452 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59b57fa700 1 -- 172.21.15.112:0/1406391471 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f59bc0a94b0 con 0x55c57b971140 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59b57fa700 10 client.16452 handle_command_reply: tid=0 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 client.16452 shutdown 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f2ad0 msgr2=0x55c57b90da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f2ad0 0x55c57b90da90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009b50 tx=0x7f59c4009c40 comp rx=0 tx=0).stop 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 shutdown_connections 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c57b971140 0x55c57b973560 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c57b8f2ad0 0x55c57b90da90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c57b8ecd50 0x55c57b8fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 --2- 172.21.15.112:0/1406391471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c57b8f21b0 0x55c57b90d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59b57fa700 0 client.16452 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.610+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 >> 172.21.15.112:0/1406391471 conn(0x55c57b818b70 msgr2=0x55c57b8f4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 shutdown_connections 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59de7b1b80 1 -- 172.21.15.112:0/1406391471 wait complete. 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59de7b1b80 20 client.16452 trim_cache size 0 max 16384 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:44:33.616 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:44:33.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f59b80b6c90 msgr2=0x7f59b80b9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f59b80b6c90 0x7f59b80b9150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f59c4020000 tx=0x7f59c4021000 comp rx=0 tx=0).stop 2024-05-10T10:44:33.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 msgr2=0x7f59cc19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:33.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 0x7f59cc19fc70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00e9a0 tx=0x7f59bc00ecb0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.611+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 shutdown_connections 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.612+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59cc109540 0x7f59cc1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.612+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59cc06bb70 0x7f59cc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.612+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f59b80b6c90 0x7f59b80b9150 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.612+0000 7f59d3f1e700 1 --2- 172.21.15.112:0/2071803058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59cc06c530 0x7f59cc19fc70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.612+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 >> 172.21.15.112:0/2071803058 conn(0x7f59cc0793e0 msgr2=0x7f59cc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.612+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 shutdown_connections 2024-05-10T10:44:33.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:33.612+0000 7f59d3f1e700 1 -- 172.21.15.112:0/2071803058 wait complete. 2024-05-10T10:44:34.442 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:44:34.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:44:34.759 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:34.901 INFO:tasks.workunit.client.0.smithi112.stdout:fallocating to largest ever: 0x9f2a70 2024-05-10T10:44:35.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:35 smithi112 ceph-mon[30100]: pgmap v560: 129 pgs: 129 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2024-05-10T10:44:35.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:35 smithi148 ceph-mon[33568]: pgmap v560: 129 pgs: 129 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2024-05-10T10:44:35.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:35 smithi195 ceph-mon[33773]: pgmap v560: 129 pgs: 129 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2024-05-10T10:44:36.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 -- 172.21.15.112:0/2549637372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 msgr2=0x7fab0c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:36.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 --2- 172.21.15.112:0/2549637372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 0x7fab0c108840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fab08009990 tx=0x7fab08009ca0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 -- 172.21.15.112:0/2549637372 shutdown_connections 2024-05-10T10:44:36.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 --2- 172.21.15.112:0/2549637372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab0c103ef0 0x7fab0c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 --2- 172.21.15.112:0/2549637372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 0x7fab0c108840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 --2- 172.21.15.112:0/2549637372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c103550 0x7fab0c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 -- 172.21.15.112:0/2549637372 >> 172.21.15.112:0/2549637372 conn(0x7fab0c07a600 msgr2=0x7fab0c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:36.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 -- 172.21.15.112:0/2549637372 shutdown_connections 2024-05-10T10:44:36.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.637+0000 7fab14659700 1 -- 172.21.15.112:0/2549637372 wait complete. 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.638+0000 7fab14659700 1 Processor -- start 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.638+0000 7fab14659700 1 -- start start 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab14659700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c103550 0x7fab0c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab14659700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab0c103ef0 0x7fab0c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab14659700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 0x7fab0c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab14659700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c113530 con 0x7fab0c103ef0 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab14659700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1133b0 con 0x7fab0c108460 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab11bf4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab0c103ef0 0x7fab0c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab123f5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c103550 0x7fab0c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab12bf6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 0x7fab0c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:36.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab11bf4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab0c103ef0 0x7fab0c19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58552/0 (socket says 172.21.15.112:58552) 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab123f5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c103550 0x7fab0c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:35980/0 (socket says 172.21.15.112:35980) 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab11bf4700 1 -- 172.21.15.112:0/2625993698 learned_addr learned my addr 172.21.15.112:0/2625993698 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.639+0000 7fab14659700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1136b0 con 0x7fab0c103550 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab12bf6700 1 -- 172.21.15.112:0/2625993698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c103550 msgr2=0x7fab0c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab12bf6700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c103550 0x7fab0c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab12bf6700 1 -- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab0c103ef0 msgr2=0x7fab0c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab12bf6700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab0c103ef0 0x7fab0c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab12bf6700 1 -- 172.21.15.112:0/2625993698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c199ff0 con 0x7fab0c108460 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab12bf6700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 0x7fab0c199890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab0400efc0 tx=0x7fab0400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab037fe700 1 -- 172.21.15.112:0/2625993698 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040093a0 con 0x7fab0c108460 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab037fe700 1 -- 172.21.15.112:0/2625993698 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab0400c940 con 0x7fab0c108460 2024-05-10T10:44:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.640+0000 7fab037fe700 1 -- 172.21.15.112:0/2625993698 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04004360 con 0x7fab0c108460 2024-05-10T10:44:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.641+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19a280 con 0x7fab0c108460 2024-05-10T10:44:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.641+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c0756d0 con 0x7fab0c108460 2024-05-10T10:44:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.642+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c104c10 con 0x7fab0c108460 2024-05-10T10:44:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.644+0000 7fab037fe700 1 -- 172.21.15.112:0/2625993698 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fab0401b030 con 0x7fab0c108460 2024-05-10T10:44:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.644+0000 7fab037fe700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faaf80b6de0 0x7faaf80b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:36.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.645+0000 7fab123f5700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faaf80b6de0 0x7faaf80b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:36.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.645+0000 7fab037fe700 1 -- 172.21.15.112:0/2625993698 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fab040e1f80 con 0x7fab0c108460 2024-05-10T10:44:36.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.645+0000 7fab123f5700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faaf80b6de0 0x7faaf80b92a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab0800b650 tx=0x7fab0801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.649+0000 7fab037fe700 1 -- 172.21.15.112:0/2625993698 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fab04022060 con 0x7fab0c108460 2024-05-10T10:44:36.945 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x9fdd84 2024-05-10T10:44:36.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.972+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab0c10ba90 con 0x7fab0c108460 2024-05-10T10:44:36.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.974+0000 7fab037fe700 1 -- 172.21.15.112:0/2625993698 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11967 (secure 0 0 0) 0x7fab04010040 con 0x7fab0c108460 2024-05-10T10:44:36.975 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:44:36.975 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:43:15.242856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24451,24469,24446,24445,24428]},"id":1}]} 2024-05-10T10:44:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faaf80b6de0 msgr2=0x7faaf80b92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faaf80b6de0 0x7faaf80b92a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab0800b650 tx=0x7fab0801a040 comp rx=0 tx=0).stop 2024-05-10T10:44:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 msgr2=0x7fab0c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 0x7fab0c199890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab0400efc0 tx=0x7fab0400c3a0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 shutdown_connections 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c108460 0x7fab0c199890 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faaf80b6de0 0x7faaf80b92a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c103550 0x7fab0c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 --2- 172.21.15.112:0/2625993698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab0c103ef0 0x7fab0c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.976+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 >> 172.21.15.112:0/2625993698 conn(0x7fab0c07a600 msgr2=0x7fab0c1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.977+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 shutdown_connections 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:36.977+0000 7fab14659700 1 -- 172.21.15.112:0/2625993698 wait complete. 2024-05-10T10:44:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-05-10T10:44:37.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:37 smithi112 ceph-mon[30100]: pgmap v561: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-05-10T10:44:37.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:37 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2625993698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:44:37.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:37 smithi148 ceph-mon[33568]: pgmap v561: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-05-10T10:44:37.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:37 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2625993698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:44:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:37 smithi195 ceph-mon[33773]: pgmap v561: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-05-10T10:44:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:37 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2625993698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:44:37.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:44:38.096 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:39.388 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:39 smithi112 ceph-mon[30100]: pgmap v562: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-05-10T10:44:39.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:39 smithi148 ceph-mon[33568]: pgmap v562: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-05-10T10:44:39.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:39 smithi195 ceph-mon[33773]: pgmap v562: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 -- 172.21.15.112:0/377345596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 msgr2=0x7fdaa0112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 --2- 172.21.15.112:0/377345596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa0112290 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fda9800b0a0 tx=0x7fda9800b3b0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 -- 172.21.15.112:0/377345596 shutdown_connections 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 --2- 172.21.15.112:0/377345596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa0112290 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 --2- 172.21.15.112:0/377345596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaa00756d0 0x7fdaa0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 --2- 172.21.15.112:0/377345596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa0075ff0 0x7fdaa010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 -- 172.21.15.112:0/377345596 >> 172.21.15.112:0/377345596 conn(0x7fdaa007a600 msgr2=0x7fdaa007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 -- 172.21.15.112:0/377345596 shutdown_connections 2024-05-10T10:44:40.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.030+0000 7fdaa6427700 1 -- 172.21.15.112:0/377345596 wait complete. 2024-05-10T10:44:40.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 Processor -- start 2024-05-10T10:44:40.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 -- start start 2024-05-10T10:44:40.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa00756d0 0x7fdaa01a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaa0075ff0 0x7fdaa01a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01156f0 con 0x7fdaa010fea0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0115570 con 0x7fdaa0075ff0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa6427700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0115870 con 0x7fdaa00756d0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fda97fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaa0075ff0 0x7fdaa01a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58568/0 (socket says 172.21.15.112:58568) 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fda9ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa00756d0 0x7fdaa01a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 -- 172.21.15.112:0/3911829704 learned_addr learned my addr 172.21.15.112:0/3911829704 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 -- 172.21.15.112:0/3911829704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa00756d0 msgr2=0x7fdaa01a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa00756d0 0x7fdaa01a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 -- 172.21.15.112:0/3911829704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaa0075ff0 msgr2=0x7fdaa01a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaa0075ff0 0x7fdaa01a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fdaa49c4700 1 -- 172.21.15.112:0/3911829704 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa019c1d0 con 0x7fdaa010fea0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.031+0000 7fda97fff700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaa0075ff0 0x7fdaa01a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.032+0000 7fdaa49c4700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa019ba70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fda9800b780 tx=0x7fda98004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:40.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.032+0000 7fda9dffb700 1 -- 172.21.15.112:0/3911829704 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98021070 con 0x7fdaa010fea0 2024-05-10T10:44:40.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.032+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa019c460 con 0x7fdaa010fea0 2024-05-10T10:44:40.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.032+0000 7fda9dffb700 1 -- 172.21.15.112:0/3911829704 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda98007b80 con 0x7fdaa010fea0 2024-05-10T10:44:40.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.032+0000 7fda9dffb700 1 -- 172.21.15.112:0/3911829704 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98005040 con 0x7fdaa010fea0 2024-05-10T10:44:40.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.032+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01a8780 con 0x7fdaa010fea0 2024-05-10T10:44:40.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.034+0000 7fda9dffb700 1 -- 172.21.15.112:0/3911829704 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fda98024070 con 0x7fdaa010fea0 2024-05-10T10:44:40.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.035+0000 7fda9dffb700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda8c0b2880 0x7fda8c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.035+0000 7fda9dffb700 1 -- 172.21.15.112:0/3911829704 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fda9800f020 con 0x7fdaa010fea0 2024-05-10T10:44:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.035+0000 7fda9ffff700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda8c0b2880 0x7fda8c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fda9ffff700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda8c0b2880 0x7fda8c0b4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdaa019cbb0 tx=0x7fda90009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 Processor -- start 2024-05-10T10:44:40.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 -- start start 2024-05-10T10:44:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e2e5fb860 con 0x557e2e6a9df0 2024-05-10T10:44:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557e2e59a7c0 con 0x557e2e6adc80 2024-05-10T10:44:40.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.036+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557e2e5f7820 con 0x557e2e6a94d0 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda97fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fdaa49c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda97fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58576/0 (socket says 172.21.15.112:58576) 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda97fff700 1 -- 172.21.15.112:0/2002797845 learned_addr learned my addr 172.21.15.112:0/2002797845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda9ffff700 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda9ffff700 1 -- 172.21.15.112:0/2002797845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 msgr2=0x557e2e6ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda9ffff700 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda9ffff700 1 -- 172.21.15.112:0/2002797845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 msgr2=0x557e2e6ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda9ffff700 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda9ffff700 1 -- 172.21.15.112:0/2002797845 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e2e5488d0 con 0x557e2e6a94d0 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda9ffff700 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6a98b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda90011c90 tx=0x7fda90014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a3e454f73791aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda967fc700 1 -- 172.21.15.112:0/2002797845 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90004230 con 0x557e2e6a94d0 2024-05-10T10:44:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.037+0000 7fda967fc700 1 -- 172.21.15.112:0/2002797845 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda9000ecd0 con 0x557e2e6a94d0 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fda967fc700 1 -- 172.21.15.112:0/2002797845 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90005020 con 0x557e2e6a94d0 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fdab0cbab80 1 -- 172.21.15.112:0/2002797845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 msgr2=0x557e2e6a98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6a98b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda90011c90 tx=0x7fda90014000 comp rx=0 tx=0).stop 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fdab0cbab80 1 -- 172.21.15.112:0/2002797845 shutdown_connections 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6a98b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/2002797845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.038+0000 7fdab0cbab80 1 -- 172.21.15.112:0/2002797845 >> 172.21.15.112:0/2002797845 conn(0x557e2e5d5100 msgr2=0x557e2e6a3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.039+0000 7fdab0cbab80 1 -- 172.21.15.112:0/2002797845 shutdown_connections 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.039+0000 7fdab0cbab80 1 -- 172.21.15.112:0/2002797845 wait complete. 2024-05-10T10:44:40.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.039+0000 7fdab0cbab80 1 -- start start 2024-05-10T10:44:40.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.039+0000 7fdab0cbab80 1 Processor -- start 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e2e6b5e90 con 0x557e2e6a9df0 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557e2e561c30 con 0x557e2e6adc80 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557e2e6b6010 con 0x557e2e6a94d0 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58578/0 (socket says 172.21.15.112:58578) 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 -- 172.21.15.112:0/3257635134 learned_addr learned my addr 172.21.15.112:0/3257635134 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdaa49c4700 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fdab0cbab80 10 client.0 _ll_register_callbacks cb 0x557e2e593920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 -- 172.21.15.112:0/3257635134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 msgr2=0x557e2e6b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 -- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 msgr2=0x557e2e6b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.040+0000 7fda97fff700 1 -- 172.21.15.112:0/3257635134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e2e597690 con 0x557e2e6a9df0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fda97fff700 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6b23a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fda88011d10 tx=0x7fda8800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cc0f28d6bf1a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fda7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda9000ecd0 con 0x557e2e6a9df0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fda7f7fe700 1 -- 172.21.15.112:0/3257635134 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8801f060 con 0x557e2e6a9df0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fda7f7fe700 1 -- 172.21.15.112:0/3257635134 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fda88017050 con 0x557e2e6a9df0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fda7f7fe700 1 -- 172.21.15.112:0/3257635134 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fda8800fde0 con 0x557e2e6a9df0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fdab0cbab80 10 client.16470 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=36}) -- 0x557e2e6c0610 con 0x557e2e6a9df0 2024-05-10T10:44:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fda7f7fe700 1 -- 172.21.15.112:0/3257635134 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fda88019030 con 0x557e2e6a9df0 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fdab0cbab80 10 client.16470 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fdab0cbab80 10 client.16470 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fdab0cbab80 20 client.16470 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.041+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e6c2fd0 0x557e2e6c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.042+0000 7fdab0cbab80 4 client.16470 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.042+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fda9000ecd0 con 0x557e2e6c2fd0 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.042+0000 7fda9ffff700 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e6c2fd0 0x557e2e6c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.042+0000 7fda7f7fe700 10 client.16470 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.042+0000 7fda9ffff700 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e6c2fd0 0x557e2e6c5430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6ce36f15dbc3f03 server_cookie=13d6164f494d777b in_seq=0 out_seq=0 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.043+0000 7fda7f7fe700 1 -- 172.21.15.112:0/3257635134 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fda9000ecd0 con 0x557e2e6c2fd0 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.043+0000 7fda7f7fe700 10 client.16470 handle_command_reply: tid=0 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.043+0000 7fdab0cbab80 1 client.16470 shutdown 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.043+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 msgr2=0x557e2e6b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.043+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6b23a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fda88011d10 tx=0x7fda8800f6a0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 shutdown_connections 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e6c2fd0 0x557e2e6c5430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a94d0 0x557e2e6b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3257635134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a9df0 0x557e2e6b23a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 >> 172.21.15.112:0/3257635134 conn(0x557e2e5d5100 msgr2=0x557e2e69e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fda7f7fe700 0 client.16470 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 shutdown_connections 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3257635134 wait complete. 2024-05-10T10:44:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.044+0000 7fdab0cbab80 20 client.16470 trim_cache size 0 max 16384 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.060+0000 7fdab0cbab80 1 Processor -- start 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fdab0cbab80 1 -- start start 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a9df0 0x557e2e6ab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6aba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01acea0 con 0x557e2e6a94d0 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01aeb60 con 0x557e2e6adc80 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01ad570 con 0x557e2e6a9df0 2024-05-10T10:44:40.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58584/0 (socket says 172.21.15.112:58584) 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 -- 172.21.15.112:0/3191801327 learned_addr learned my addr 172.21.15.112:0/3191801327 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda97fff700 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a9df0 0x557e2e6ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 -- 172.21.15.112:0/3191801327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a9df0 msgr2=0x557e2e6ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a9df0 0x557e2e6ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 -- 172.21.15.112:0/3191801327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 msgr2=0x557e2e6aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 -- 172.21.15.112:0/3191801327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e2e6a0810 con 0x557e2e6a94d0 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.061+0000 7fda9ffff700 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bef20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x557e2e6a4c80 tx=0x7fda90004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7afe4ec1affb9d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.062+0000 7fda95ffb700 1 -- 172.21.15.112:0/3191801327 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9000ea40 con 0x557e2e6a94d0 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.062+0000 7fda95ffb700 1 -- 172.21.15.112:0/3191801327 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda90033070 con 0x557e2e6a94d0 2024-05-10T10:44:40.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.062+0000 7fda95ffb700 1 -- 172.21.15.112:0/3191801327 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90025680 con 0x557e2e6a94d0 2024-05-10T10:44:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3191801327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 msgr2=0x557e2e6bef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bef20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x557e2e6a4c80 tx=0x7fda90004690 comp rx=0 tx=0).stop 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3191801327 shutdown_connections 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6adc80 0x557e2e6aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bef20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/3191801327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6a9df0 0x557e2e6ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3191801327 >> 172.21.15.112:0/3191801327 conn(0x557e2e5d5100 msgr2=0x557e2e6b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3191801327 shutdown_connections 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 -- 172.21.15.112:0/3191801327 wait complete. 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 -- start start 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.063+0000 7fdab0cbab80 1 Processor -- start 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6a9df0 0x557e2e6bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdab0cbab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6adc80 0x557e2e6c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01ae3d0 con 0x557e2e6a94d0 2024-05-10T10:44:40.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01add00 con 0x557e2e6a9df0 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdab0cbab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01aae40 con 0x557e2e6adc80 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdab0cbab80 10 client.0 _ll_register_callbacks cb 0x557e2e593920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda9ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda97fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6a9df0 0x557e2e6bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda97fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6a9df0 0x557e2e6bb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34896/0 (socket says 172.21.15.112:34896) 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda97fff700 1 -- 172.21.15.112:0/4253487483 learned_addr learned my addr 172.21.15.112:0/4253487483 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fdaa49c4700 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6adc80 0x557e2e6c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda9ffff700 1 -- 172.21.15.112:0/4253487483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6adc80 msgr2=0x557e2e6c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda9ffff700 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6adc80 0x557e2e6c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda9ffff700 1 -- 172.21.15.112:0/4253487483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6a9df0 msgr2=0x557e2e6bb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda9ffff700 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6a9df0 0x557e2e6bb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.064+0000 7fda9ffff700 1 -- 172.21.15.112:0/4253487483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e2e6a9c20 con 0x557e2e6a94d0 2024-05-10T10:44:40.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fda9ffff700 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bb2a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x557e2e6a4c80 tx=0x7fda9000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ec8290b32e5871 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fda7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fda7f7fe700 1 -- 172.21.15.112:0/4253487483 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9003b070 con 0x557e2e6a94d0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fda7f7fe700 1 -- 172.21.15.112:0/4253487483 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fda9002e380 con 0x557e2e6a94d0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda90033070 con 0x557e2e6a94d0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fda7f7fe700 1 -- 172.21.15.112:0/4253487483 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fda9002e580 con 0x557e2e6a94d0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 10 client.16476 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557e2e72ccb0 con 0x557e2e6a94d0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fda7f7fe700 1 -- 172.21.15.112:0/4253487483 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda90033070 con 0x557e2e6a94d0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fda7f7fe700 1 -- 172.21.15.112:0/4253487483 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fda90025a70 con 0x557e2e6a94d0 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 10 client.16476 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 10 client.16476 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 20 client.16476 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:40.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e72d050 0x557e2e72f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 4 client.16476 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.065+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e2e675650 con 0x557e2e72d050 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.066+0000 7fda97fff700 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e72d050 0x557e2e72f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.066+0000 7fda7f7fe700 10 client.16476 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.066+0000 7fda97fff700 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e72d050 0x557e2e72f470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bba3b08680a8a80a server_cookie=69a3b1664d1a2e16 in_seq=0 out_seq=0 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.066+0000 7fda7f7fe700 1 -- 172.21.15.112:0/4253487483 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e2e675650 con 0x557e2e72d050 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.066+0000 7fda7f7fe700 10 client.16476 handle_command_reply: tid=0 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.066+0000 7fdab0cbab80 1 client.16476 shutdown 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 msgr2=0x557e2e6bb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bb2a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x557e2e6a4c80 tx=0x7fda9000e900 comp rx=0 tx=0).stop 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 shutdown_connections 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557e2e72d050 0x557e2e72f470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e2e6adc80 0x557e2e6c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e2e6a94d0 0x557e2e6bb2a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 --2- 172.21.15.112:0/4253487483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557e2e6a9df0 0x557e2e6bb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 >> 172.21.15.112:0/4253487483 conn(0x557e2e5d5100 msgr2=0x557e2e69de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fda7f7fe700 0 client.16476 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 shutdown_connections 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 1 -- 172.21.15.112:0/4253487483 wait complete. 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdab0cbab80 20 client.16476 trim_cache size 0 max 16384 2024-05-10T10:44:40.073 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda8c0b2880 msgr2=0x7fda8c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda8c0b2880 0x7fda8c0b4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdaa019cbb0 tx=0x7fda90009260 comp rx=0 tx=0).stop 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 msgr2=0x7fdaa019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa019ba70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fda9800b780 tx=0x7fda98004d00 comp rx=0 tx=0).stop 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 shutdown_connections 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaa010fea0 0x7fdaa019ba70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda8c0b2880 0x7fda8c0b4d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.067+0000 7fdaa6427700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaa00756d0 0x7fdaa01a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.068+0000 7fdaa6427700 1 --2- 172.21.15.112:0/3911829704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaa0075ff0 0x7fdaa01a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:40.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.068+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 >> 172.21.15.112:0/3911829704 conn(0x7fdaa007a600 msgr2=0x7fdaa01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:40.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.068+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 shutdown_connections 2024-05-10T10:44:40.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:40.068+0000 7fdaa6427700 1 -- 172.21.15.112:0/3911829704 wait complete. 2024-05-10T10:44:40.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:44:41.101 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:41.371 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:41 smithi112 ceph-mon[30100]: pgmap v563: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-05-10T10:44:41.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:41 smithi148 ceph-mon[33568]: pgmap v563: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-05-10T10:44:41.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:41 smithi195 ceph-mon[33773]: pgmap v563: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-05-10T10:44:42.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:42 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:44:42.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:42 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:44:42.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:42 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:44:43.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 -- 172.21.15.112:0/2330612154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab8102d60 msgr2=0x7fdab810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 --2- 172.21.15.112:0/2330612154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab8102d60 0x7fdab810f250 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdab000b0a0 tx=0x7fdab000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 -- 172.21.15.112:0/2330612154 shutdown_connections 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 --2- 172.21.15.112:0/2330612154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab8102d60 0x7fdab810f250 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 --2- 172.21.15.112:0/2330612154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8108410 0x7fdab81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 --2- 172.21.15.112:0/2330612154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab81023a0 0x7fdab8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 -- 172.21.15.112:0/2330612154 >> 172.21.15.112:0/2330612154 conn(0x7fdab8079350 msgr2=0x7fdab8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 -- 172.21.15.112:0/2330612154 shutdown_connections 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.013+0000 7fdabf546700 1 -- 172.21.15.112:0/2330612154 wait complete. 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.014+0000 7fdabf546700 1 Processor -- start 2024-05-10T10:44:43.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.014+0000 7fdabf546700 1 -- start start 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.014+0000 7fdabf546700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab81023a0 0x7fdab81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.014+0000 7fdabf546700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8102d60 0x7fdab81a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabf546700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8108410 0x7fdab819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabf546700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8112400 con 0x7fdab81023a0 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabf546700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8112280 con 0x7fdab8108410 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabf546700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8112580 con 0x7fdab8102d60 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabd2e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab81023a0 0x7fdab81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8102d60 0x7fdab81a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8102d60 0x7fdab81a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:36036/0 (socket says 172.21.15.112:36036) 2024-05-10T10:44:43.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 -- 172.21.15.112:0/2723040257 learned_addr learned my addr 172.21.15.112:0/2723040257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabdae3700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8108410 0x7fdab819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 -- 172.21.15.112:0/2723040257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8108410 msgr2=0x7fdab819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8108410 0x7fdab819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 -- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab81023a0 msgr2=0x7fdab81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab81023a0 0x7fdab81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 -- 172.21.15.112:0/2723040257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab819c170 con 0x7fdab8102d60 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.015+0000 7fdabcae1700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8102d60 0x7fdab81a1d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c600 tx=0x7fdaa800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.016+0000 7fdaae7fc700 1 -- 172.21.15.112:0/2723040257 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8023070 con 0x7fdab8102d60 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.016+0000 7fdaae7fc700 1 -- 172.21.15.112:0/2723040257 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa801e8c0 con 0x7fdab8102d60 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.016+0000 7fdaae7fc700 1 -- 172.21.15.112:0/2723040257 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8016450 con 0x7fdab8102d60 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.016+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab819c400 con 0x7fdab8102d60 2024-05-10T10:44:43.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.016+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab8109200 con 0x7fdab8102d60 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.018+0000 7fdaae7fc700 1 -- 172.21.15.112:0/2723040257 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdaa8026070 con 0x7fdab8102d60 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.019+0000 7fdaae7fc700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdaa40b2900 0x7fdaa40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.019+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdaa40b2900 0x7fdaa40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.019+0000 7fdaae7fc700 1 -- 172.21.15.112:0/2723040257 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fdaa80ea510 con 0x7fdab8102d60 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdaa40b2900 0x7fdaa40b4dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdab819cb50 tx=0x7fdab4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 Processor -- start 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 -- start start 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe6b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe6c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c54fdb9860 con 0x55c54fe6bc80 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c54fd587c0 con 0x55c54fe67df0 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c54fdb5820 con 0x55c54fe674d0 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe6b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdabd2e2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdabdae3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe6b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34918/0 (socket says 172.21.15.112:34918) 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.020+0000 7fdabcae1700 1 -- 172.21.15.112:0/2775244053 learned_addr learned my addr 172.21.15.112:0/2775244053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdabd2e2700 1 -- 172.21.15.112:0/2775244053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 msgr2=0x55c54fe6b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/2775244053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe6b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdabd2e2700 1 -- 172.21.15.112:0/2775244053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 msgr2=0x55c54fe6c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/2775244053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe6c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdabd2e2700 1 -- 172.21.15.112:0/2775244053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54fd068d0 con 0x55c54fe674d0 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/2775244053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe678b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009fd0 tx=0x7fdab4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96261633e4616c97 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdaa37fe700 1 -- 172.21.15.112:0/2775244053 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab400ea10 con 0x55c54fe674d0 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdaa37fe700 1 -- 172.21.15.112:0/2775244053 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdab400ebb0 con 0x55c54fe674d0 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.021+0000 7fdaa37fe700 1 -- 172.21.15.112:0/2775244053 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab401e3f0 con 0x55c54fe674d0 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/2775244053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 msgr2=0x55c54fe678b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/2775244053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe678b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009fd0 tx=0x7fdab4014000 comp rx=0 tx=0).stop 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/2775244053 shutdown_connections 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/2775244053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe6c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/2775244053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe678b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/2775244053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe6b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/2775244053 >> 172.21.15.112:0/2775244053 conn(0x55c54fd93100 msgr2=0x55c54fe61c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/2775244053 shutdown_connections 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/2775244053 wait complete. 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 -- start start 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.022+0000 7fdac9dd9b80 1 Processor -- start 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe70480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c54fe73e90 con 0x55c54fe6bc80 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c54fd1fc30 con 0x55c54fe67df0 2024-05-10T10:44:43.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c54fe74010 con 0x55c54fe674d0 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdac9dd9b80 10 client.0 _ll_register_callbacks cb 0x55c54fd51920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58644/0 (socket says 172.21.15.112:58644) 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 -- 172.21.15.112:0/4152566301 learned_addr learned my addr 172.21.15.112:0/4152566301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabcae1700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe70480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 -- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 msgr2=0x55c54fe730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 -- 172.21.15.112:0/4152566301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 msgr2=0x55c54fe70480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe70480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabdae3700 1 -- 172.21.15.112:0/4152566301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54fd55690 con 0x55c54fe6bc80 2024-05-10T10:44:43.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabcae1700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe70480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.023+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdabdae3700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe709c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdab000bd50 tx=0x7fdab000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706b5bdfef0dea4f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdaa0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/4152566301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab00074e0 con 0x55c54fe6bc80 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/4152566301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fdab0007680 con 0x55c54fe6bc80 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab400ebb0 con 0x55c54fe6bc80 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/4152566301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdab0007880 con 0x55c54fe6bc80 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdac9dd9b80 10 client.16488 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c54fe7e730 con 0x55c54fe6bc80 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.024+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/4152566301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdab400ebb0 con 0x55c54fe6bc80 2024-05-10T10:44:43.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/4152566301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fdab0011070 con 0x55c54fe6bc80 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdac9dd9b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdac9dd9b80 10 client.16488 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdac9dd9b80 20 client.16488 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54fe80d70 0x55c54fe83160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdac9dd9b80 4 client.16488 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c54fe33650 con 0x55c54fe80d70 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54fe80d70 0x55c54fe83160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54fe80d70 0x55c54fe83160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=34d9fda353ceb546 server_cookie=85b2e10e2ee18cbb in_seq=0 out_seq=0 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.025+0000 7fdaa0ff9700 10 client.16488 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/4152566301 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c54fe33650 con 0x55c54fe80d70 2024-05-10T10:44:43.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdaa0ff9700 10 client.16488 handle_command_reply: tid=0 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 client.16488 shutdown 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 msgr2=0x55c54fe709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe709c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdab000bd50 tx=0x7fdab000bd80 comp rx=0 tx=0).stop 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 shutdown_connections 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54fe80d70 0x55c54fe83160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdaa0ff9700 0 client.16488 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe6bc80 0x55c54fe709c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe674d0 0x55c54fe730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/4152566301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe67df0 0x55c54fe70480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 >> 172.21.15.112:0/4152566301 conn(0x55c54fd93100 msgr2=0x55c54fe5c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 shutdown_connections 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.026+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/4152566301 wait complete. 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.027+0000 7fdac9dd9b80 20 client.16488 trim_cache size 0 max 16384 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 Processor -- start 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 -- start start 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe674d0 0x55c54fe7cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe67df0 0x55c54fe6f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81acf20 con 0x55c54fe67df0 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81aebe0 con 0x55c54fe674d0 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81ad5f0 con 0x55c54fe6bc80 2024-05-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe67df0 0x55c54fe6f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe67df0 0x55c54fe6f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58648/0 (socket says 172.21.15.112:58648) 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.046+0000 7fdabcae1700 1 -- 172.21.15.112:0/1061288025 learned_addr learned my addr 172.21.15.112:0/1061288025 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe674d0 0x55c54fe7cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabdae3700 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabcae1700 1 -- 172.21.15.112:0/1061288025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 msgr2=0x55c54fe889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabcae1700 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe889c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabcae1700 1 -- 172.21.15.112:0/1061288025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe674d0 msgr2=0x55c54fe7cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabcae1700 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe674d0 0x55c54fe7cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabcae1700 1 -- 172.21.15.112:0/1061288025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54fe6e9f0 con 0x55c54fe67df0 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdabcae1700 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe67df0 0x55c54fe6f600 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b1200 tx=0x7fdaa80d4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f40033dca2c18d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdaa2ffd700 1 -- 172.21.15.112:0/1061288025 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80abe60 con 0x55c54fe67df0 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdaa2ffd700 1 -- 172.21.15.112:0/1061288025 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa80b1740 con 0x55c54fe67df0 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.047+0000 7fdaa2ffd700 1 -- 172.21.15.112:0/1061288025 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80ba110 con 0x55c54fe67df0 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/1061288025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe67df0 msgr2=0x55c54fe6f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe67df0 0x55c54fe6f600 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b1200 tx=0x7fdaa80d4040 comp rx=0 tx=0).stop 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/1061288025 shutdown_connections 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe674d0 0x55c54fe7cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/1061288025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe67df0 0x55c54fe6f600 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/1061288025 >> 172.21.15.112:0/1061288025 conn(0x55c54fd93100 msgr2=0x55c54fe61bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/1061288025 shutdown_connections 2024-05-10T10:44:43.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/1061288025 wait complete. 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 -- start start 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.048+0000 7fdac9dd9b80 1 Processor -- start 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe674d0 0x55c54fe79600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe74930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdac9dd9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe80d70 0x55c54fe74e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81ae450 con 0x55c54fe674d0 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81add80 con 0x55c54fe80d70 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdac9dd9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81aaee0 con 0x55c54fe6bc80 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdac9dd9b80 10 client.0 _ll_register_callbacks cb 0x55c54fd51920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabd2e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe674d0 0x55c54fe79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe74930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe74930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:36062/0 (socket says 172.21.15.112:36062) 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabd2e2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe674d0 0x55c54fe79600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58662/0 (socket says 172.21.15.112:58662) 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 -- 172.21.15.112:0/400202099 learned_addr learned my addr 172.21.15.112:0/400202099 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabdae3700 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe80d70 0x55c54fe74e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 -- 172.21.15.112:0/400202099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe80d70 msgr2=0x55c54fe74e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe80d70 0x55c54fe74e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 -- 172.21.15.112:0/400202099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe674d0 msgr2=0x55c54fe79600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe674d0 0x55c54fe79600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.049+0000 7fdabcae1700 1 -- 172.21.15.112:0/400202099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54fe754b0 con 0x55c54fe6bc80 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdabcae1700 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe74930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b16d0 tx=0x7fdaa8005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1853ffa18417e3ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdaa0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/400202099 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80abe60 con 0x55c54fe6bc80 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa80b1740 con 0x55c54fe6bc80 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/400202099 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fdaa80add30 con 0x55c54fe6bc80 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/400202099 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdaa80ba5a0 con 0x55c54fe6bc80 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 10 client.25903 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c54feeb3b0 con 0x55c54fe6bc80 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/400202099 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa80b1740 con 0x55c54fe6bc80 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/400202099 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fdaa80b09a0 con 0x55c54fe6bc80 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 10 client.25903 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 10 client.25903 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 20 client.25903 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:43.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54feeba10 0x55c54feede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 4 client.25903 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.050+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c54fdc7240 con 0x55c54feeba10 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54feeba10 0x55c54feede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdaa0ff9700 10 client.25903 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdabd2e2700 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54feeba10 0x55c54feede30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28b9bb0c5067e28a server_cookie=dc610ce55daa9d62 in_seq=0 out_seq=0 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdaa0ff9700 1 -- 172.21.15.112:0/400202099 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c54fdc7240 con 0x55c54feeba10 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdaa0ff9700 10 client.25903 handle_command_reply: tid=0 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 client.25903 shutdown 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 msgr2=0x55c54fe74930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe74930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b16d0 tx=0x7fdaa8005b40 comp rx=0 tx=0).stop 2024-05-10T10:44:43.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 shutdown_connections 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55c54feeba10 0x55c54feede30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54fe80d70 0x55c54fe74e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdaa0ff9700 0 client.25903 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c54fe674d0 0x55c54fe79600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 --2- 172.21.15.112:0/400202099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c54fe6bc80 0x55c54fe74930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.051+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 >> 172.21.15.112:0/400202099 conn(0x55c54fd93100 msgr2=0x55c54fe6f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.052+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 shutdown_connections 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.052+0000 7fdac9dd9b80 1 -- 172.21.15.112:0/400202099 wait complete. 2024-05-10T10:44:43.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.052+0000 7fdac9dd9b80 20 client.25903 trim_cache size 0 max 16384 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.052+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdaa40b2900 msgr2=0x7fdaa40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.052+0000 7fdabf546700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdaa40b2900 0x7fdaa40b4dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdab819cb50 tx=0x7fdab4009260 comp rx=0 tx=0).stop 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.052+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8102d60 msgr2=0x7fdab81a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8102d60 0x7fdab81a1d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c600 tx=0x7fdaa800c910 comp rx=0 tx=0).stop 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 shutdown_connections 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8108410 0x7fdab819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdab81023a0 0x7fdab81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdaa40b2900 0x7fdaa40b4dc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 --2- 172.21.15.112:0/2723040257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8102d60 0x7fdab81a1d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 >> 172.21.15.112:0/2723040257 conn(0x7fdab8079350 msgr2=0x7fdab806b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 shutdown_connections 2024-05-10T10:44:43.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:43.053+0000 7fdabf546700 1 -- 172.21.15.112:0/2723040257 wait complete. 2024-05-10T10:44:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: pgmap v564: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 586 op/s 2024-05-10T10:44:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:44:43.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:44:43.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:44:43.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: pgmap v564: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 586 op/s 2024-05-10T10:44:43.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:44:43.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:44:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: pgmap v564: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 586 op/s 2024-05-10T10:44:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:44:43.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:44:43.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:44:43.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.802+0000 7f53bb61c640 1 -- 172.21.15.112:0/2853727935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 msgr2=0x7f53b4102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.802+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2853727935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4102f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009900 tx=0x7f53a402a410 comp rx=0 tx=0).stop 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- 172.21.15.112:0/2853727935 shutdown_connections 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2853727935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b41034c0 0x7f53b4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2853727935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4102f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2853727935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b4101910 0x7f53b4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- 172.21.15.112:0/2853727935 >> 172.21.15.112:0/2853727935 conn(0x7f53b40fce80 msgr2=0x7f53b40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- 172.21.15.112:0/2853727935 shutdown_connections 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- 172.21.15.112:0/2853727935 wait complete. 2024-05-10T10:44:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 Processor -- start 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- start start 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4101910 0x7f53b4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b41034c0 0x7f53b41a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53b410d320 con 0x7f53b41034c0 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53b410d1a0 con 0x7f53b4102b00 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.803+0000 7f53bb61c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53b410d4a0 con 0x7f53b4101910 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b41034c0 0x7f53b41a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b93b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4101910 0x7f53b4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b93b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4101910 0x7f53b4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:36070/0 (socket says 172.21.15.112:36070) 2024-05-10T10:44:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b41034c0 0x7f53b41a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58674/0 (socket says 172.21.15.112:58674) 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b93b8640 1 -- 172.21.15.112:0/2585929651 learned_addr learned my addr 172.21.15.112:0/2585929651 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34954/0 (socket says 172.21.15.112:34954) 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 -- 172.21.15.112:0/2585929651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4101910 msgr2=0x7f53b4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4101910 0x7f53b4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 -- 172.21.15.112:0/2585929651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 msgr2=0x7f53b4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 -- 172.21.15.112:0/2585929651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b41a0710 con 0x7f53b41034c0 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b93b8640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4101910 0x7f53b4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53b9bb9640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b41034c0 0x7f53b41a00a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00da10 tx=0x7f53ac00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53aa7fc640 1 -- 172.21.15.112:0/2585929651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00bd70 con 0x7f53b41034c0 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53aa7fc640 1 -- 172.21.15.112:0/2585929651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53ac003e00 con 0x7f53b41034c0 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53aa7fc640 1 -- 172.21.15.112:0/2585929651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0106c0 con 0x7f53b41034c0 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b41a0a00 con 0x7f53b41034c0 2024-05-10T10:44:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.804+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b41a0ed0 con 0x7f53b41034c0 2024-05-10T10:44:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.807+0000 7f53aa7fc640 1 -- 172.21.15.112:0/2585929651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f53ac003970 con 0x7f53b41034c0 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.807+0000 7f53be419740 1 Processor -- start 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53be419740 1 -- start start 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53be419740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bd51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53be419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836bda2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53be419740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bda910 0x556836bdcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53be419740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556836b395d0 con 0x556836bd4db0 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53be419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556836af2890 con 0x556836bd57a0 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53be419740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556836b35590 con 0x556836bda910 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53aa7fc640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53a00b28a0 0x7f53a00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836bda2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bd51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836bda2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34956/0 (socket says 172.21.15.112:34956) 2024-05-10T10:44:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 -- 172.21.15.112:0/4137654755 learned_addr learned my addr 172.21.15.112:0/4137654755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b9bb9640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53a00b28a0 0x7f53a00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bda910 0x556836bdcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53aa7fc640 1 -- 172.21.15.112:0/2585929651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f53ac0e2940 con 0x7f53b41034c0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 -- 172.21.15.112:0/4137654755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bda910 msgr2=0x556836bdcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bda910 0x556836bdcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 -- 172.21.15.112:0/4137654755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 msgr2=0x556836bd51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bd51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 -- 172.21.15.112:0/4137654755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683684b210 con 0x556836bd57a0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b93b8640 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836bda2a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53b00115f0 tx=0x7f53b0011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df7919e5915171 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.808+0000 7f53b9bb9640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53a00b28a0 0x7f53a00b4d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0a6e40 tx=0x7f53ac0734c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.809+0000 7f539f7fe640 1 -- 172.21.15.112:0/4137654755 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b000f670 con 0x556836bd57a0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.809+0000 7f539f7fe640 1 -- 172.21.15.112:0/4137654755 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53b000fca0 con 0x556836bd57a0 2024-05-10T10:44:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.809+0000 7f539f7fe640 1 -- 172.21.15.112:0/4137654755 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0019c50 con 0x556836bd57a0 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 -- 172.21.15.112:0/4137654755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 msgr2=0x556836bda2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836bda2a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53b00115f0 tx=0x7f53b0011ab0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 -- 172.21.15.112:0/4137654755 shutdown_connections 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bda910 0x556836bdcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836bda2a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 --2- 172.21.15.112:0/4137654755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bd51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 -- 172.21.15.112:0/4137654755 >> 172.21.15.112:0/4137654755 conn(0x556836bd0420 msgr2=0x556836bd0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 -- 172.21.15.112:0/4137654755 shutdown_connections 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 -- 172.21.15.112:0/4137654755 wait complete. 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 -- start start 2024-05-10T10:44:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.810+0000 7f53be419740 1 Processor -- start 2024-05-10T10:44:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53be419740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bdef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53be419740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd57a0 0x556836bdf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53be419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bda910 0x556836be6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53be419740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556836bdd8c0 con 0x556836bd4db0 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53be419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556836af6370 con 0x556836bda910 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53be419740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556836bdda40 con 0x556836bd57a0 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53be419740 10 client.0 _ll_register_callbacks cb 0x55683684aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bdef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bdef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58690/0 (socket says 172.21.15.112:58690) 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53b8bb7640 1 -- 172.21.15.112:0/2135348250 learned_addr learned my addr 172.21.15.112:0/2135348250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53b93b8640 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd57a0 0x556836bdf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.811+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bda910 0x556836be6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53b8bb7640 1 -- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd57a0 msgr2=0x556836bdf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd57a0 0x556836bdf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53b8bb7640 1 -- 172.21.15.112:0/2135348250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bda910 msgr2=0x556836be6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bda910 0x556836be6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53b8bb7640 1 -- 172.21.15.112:0/2135348250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556836917620 con 0x556836bd4db0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bdef90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f53a4031460 tx=0x7f53a4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7606f6c12e4a19bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f539cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f539cff9640 1 -- 172.21.15.112:0/2135348250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4035070 con 0x556836bd4db0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b000fca0 con 0x556836bd4db0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f539cff9640 1 -- 172.21.15.112:0/2135348250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f53a40317d0 con 0x556836bd4db0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f539cff9640 1 -- 172.21.15.112:0/2135348250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f53a4054050 con 0x556836bd4db0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53be419740 10 client.16506 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556836bed9c0 con 0x556836bd4db0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.812+0000 7f539cff9640 1 -- 172.21.15.112:0/2135348250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53b000fca0 con 0x556836bd4db0 2024-05-10T10:44:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f539cff9640 1 -- 172.21.15.112:0/2135348250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f53a403d980 con 0x556836bd4db0 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53be419740 10 client.16506 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53be419740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53be419740 20 client.16506 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53be419740 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836bef580 0x556836bf1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53be419740 4 client.16506 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556836b471e0 con 0x556836bef580 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53b93b8640 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836bef580 0x556836bf1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f53b93b8640 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836bef580 0x556836bf1a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32bbbc793bb75ef6 server_cookie=6534d1964886076a in_seq=0 out_seq=0 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.813+0000 7f539cff9640 10 client.16506 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f539cff9640 1 -- 172.21.15.112:0/2135348250 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556836b471e0 con 0x556836bef580 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f539cff9640 10 client.16506 handle_command_reply: tid=0 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f53be419740 1 client.16506 shutdown 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 msgr2=0x556836bdef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f53be419740 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bdef90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f53a4031460 tx=0x7f53a4005a20 comp rx=0 tx=0).stop 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 shutdown_connections 2024-05-10T10:44:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f53be419740 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836bef580 0x556836bf1a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f53be419740 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bda910 0x556836be6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.814+0000 7f53be419740 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd57a0 0x556836bdf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.815+0000 7f53be419740 1 --2- 172.21.15.112:0/2135348250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bd4db0 0x556836bdef90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.815+0000 7f539cff9640 0 client.16506 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.815+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 >> 172.21.15.112:0/2135348250 conn(0x556836bd0420 msgr2=0x556836bcf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.815+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 shutdown_connections 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.815+0000 7f53be419740 1 -- 172.21.15.112:0/2135348250 wait complete. 2024-05-10T10:44:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.815+0000 7f53be419740 20 client.16506 trim_cache size 0 max 16384 2024-05-10T10:44:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 Processor -- start 2024-05-10T10:44:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 -- start start 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836bd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836bd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41a9120 con 0x556836bda910 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41aade0 con 0x556836bd57a0 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.834+0000 7f53be419740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41a97f0 con 0x556836bd4db0 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836bd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:34976/0 (socket says 172.21.15.112:34976) 2024-05-10T10:44:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836bd3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:36102/0 (socket says 172.21.15.112:36102) 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 -- 172.21.15.112:0/1994238611 learned_addr learned my addr 172.21.15.112:0/1994238611 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836bd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 -- 172.21.15.112:0/1994238611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 msgr2=0x556836bd3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836bd3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 -- 172.21.15.112:0/1994238611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 msgr2=0x556836bd3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836bd3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b93b8640 1 -- 172.21.15.112:0/1994238611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556836bd0250 con 0x556836bd57a0 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.835+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836bd3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.836+0000 7f53b93b8640 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be64a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009230 tx=0x7f53b0005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd26596b8cdb908e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.836+0000 7f539effd640 1 -- 172.21.15.112:0/1994238611 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0004020 con 0x556836bd57a0 2024-05-10T10:44:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.836+0000 7f539effd640 1 -- 172.21.15.112:0/1994238611 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53b00054d0 con 0x556836bd57a0 2024-05-10T10:44:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.836+0000 7f539effd640 1 -- 172.21.15.112:0/1994238611 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00048a0 con 0x556836bd57a0 2024-05-10T10:44:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 -- 172.21.15.112:0/1994238611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 msgr2=0x556836be64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be64a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009230 tx=0x7f53b0005c70 comp rx=0 tx=0).stop 2024-05-10T10:44:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 -- 172.21.15.112:0/1994238611 shutdown_connections 2024-05-10T10:44:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836bd3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be64a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 --2- 172.21.15.112:0/1994238611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836bd3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 -- 172.21.15.112:0/1994238611 >> 172.21.15.112:0/1994238611 conn(0x556836bd0420 msgr2=0x556836bcb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 -- 172.21.15.112:0/1994238611 shutdown_connections 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 -- 172.21.15.112:0/1994238611 wait complete. 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 -- start start 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.837+0000 7f53be419740 1 Processor -- start 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53be419740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836be14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53be419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53be419740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836be4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53be419740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41aa650 con 0x556836bda910 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53be419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41a9f80 con 0x556836bd57a0 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53be419740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41a7100 con 0x556836bd4db0 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53be419740 10 client.0 _ll_register_callbacks cb 0x7f53a4009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836be4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836be4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58720/0 (socket says 172.21.15.112:58720) 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 -- 172.21.15.112:0/4099961348 learned_addr learned my addr 172.21.15.112:0/4099961348 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b93b8640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836be14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 -- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 msgr2=0x556836be14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836be14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 -- 172.21.15.112:0/4099961348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 msgr2=0x556836be1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.838+0000 7f53b8bb7640 1 -- 172.21.15.112:0/4099961348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556836be5040 con 0x556836bda910 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f53b93b8640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f53b8bb7640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836be4a50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f53a4001eb0 tx=0x7f53a4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06b6b9fd72372f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f539cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f539cff9640 1 -- 172.21.15.112:0/4099961348 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4046040 con 0x556836bda910 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f539cff9640 1 -- 172.21.15.112:0/4099961348 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f53a40317d0 con 0x556836bda910 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b00054d0 con 0x556836bda910 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f539cff9640 1 -- 172.21.15.112:0/4099961348 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f53a4035040 con 0x556836bda910 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f53be419740 10 client.16512 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556836be6350 con 0x556836bda910 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.839+0000 7f539cff9640 1 -- 172.21.15.112:0/4099961348 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53b00054d0 con 0x556836bda910 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f539cff9640 1 -- 172.21.15.112:0/4099961348 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f53a404d580 con 0x556836bda910 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53be419740 10 client.16512 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53be419740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53be419740 20 client.16512 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53be419740 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836c6b100 0x556836c6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53be419740 4 client.16512 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556836b41050 con 0x556836c6b100 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53b93b8640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836c6b100 0x556836c6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f53b93b8640 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836c6b100 0x556836c6d520 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533255a14e5fc11f server_cookie=171477a4a5245e06 in_seq=0 out_seq=0 2024-05-10T10:44:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.840+0000 7f539cff9640 10 client.16512 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f539cff9640 1 -- 172.21.15.112:0/4099961348 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556836b41050 con 0x556836c6b100 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f539cff9640 10 client.16512 handle_command_reply: tid=0 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 client.16512 shutdown 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 msgr2=0x556836be4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836be4a50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f53a4001eb0 tx=0x7f53a4009680 comp rx=0 tx=0).stop 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 shutdown_connections 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556836bda910 0x556836be4a50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556836c6b100 0x556836c6d520 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556836bd57a0 0x556836be1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 --2- 172.21.15.112:0/4099961348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556836bd4db0 0x556836be14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f539cff9640 0 client.16512 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 >> 172.21.15.112:0/4099961348 conn(0x556836bd0420 msgr2=0x556836bd3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 shutdown_connections 2024-05-10T10:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 1 -- 172.21.15.112:0/4099961348 wait complete. 2024-05-10T10:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.858+0000 7f53be419740 20 client.16512 trim_cache size 0 max 16384 2024-05-10T10:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53a00b28a0 msgr2=0x7f53a00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53a00b28a0 0x7f53a00b4d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0a6e40 tx=0x7f53ac0734c0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b41034c0 msgr2=0x7f53b41a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b41034c0 0x7f53b41a00a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00da10 tx=0x7f53ac00ded0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 shutdown_connections 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f53a00b28a0 0x7f53a00b4d60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53b41034c0 0x7f53b41a00a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53b4102b00 0x7f53b4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 --2- 172.21.15.112:0/2585929651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4101910 0x7f53b4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 >> 172.21.15.112:0/2585929651 conn(0x7f53b40fce80 msgr2=0x7f53b40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 shutdown_connections 2024-05-10T10:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:43.859+0000 7f53bb61c640 1 -- 172.21.15.112:0/2585929651 wait complete. 2024-05-10T10:44:43.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:44:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:44.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:44 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-05-10T10:44:44.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:44 smithi112 ceph-mon[30100]: Unable to set osd_memory_target on smithi148 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-10T10:44:44.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:44 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi112 to 176.1M 2024-05-10T10:44:44.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:44 smithi112 ceph-mon[30100]: Unable to set osd_memory_target on smithi112 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-10T10:44:44.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:44 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-05-10T10:44:44.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:44 smithi148 ceph-mon[33568]: Unable to set osd_memory_target on smithi148 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-10T10:44:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:44 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi112 to 176.1M 2024-05-10T10:44:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:44 smithi148 ceph-mon[33568]: Unable to set osd_memory_target on smithi112 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-10T10:44:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:44 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi148 to 176.2M 2024-05-10T10:44:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:44 smithi195 ceph-mon[33773]: Unable to set osd_memory_target on smithi148 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-10T10:44:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:44 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi112 to 176.1M 2024-05-10T10:44:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:44 smithi195 ceph-mon[33773]: Unable to set osd_memory_target on smithi112 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-10T10:44:44.856 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9ff7e4 2024-05-10T10:44:45.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:45 smithi112 ceph-mon[30100]: pgmap v565: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 550 op/s 2024-05-10T10:44:45.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:45 smithi195 ceph-mon[33773]: pgmap v565: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 550 op/s 2024-05-10T10:44:45.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:45 smithi148 ceph-mon[33568]: pgmap v565: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 550 op/s 2024-05-10T10:44:46.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 -- 172.21.15.112:0/1344767645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108300 msgr2=0x7f61581086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 --2- 172.21.15.112:0/1344767645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108300 0x7f61581086e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f614c00b8e0 tx=0x7f614c00bbf0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 -- 172.21.15.112:0/1344767645 shutdown_connections 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 --2- 172.21.15.112:0/1344767645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61581076e0 0x7f615810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 --2- 172.21.15.112:0/1344767645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108300 0x7f61581086e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 --2- 172.21.15.112:0/1344767645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6158108cb0 0x7f6158107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 -- 172.21.15.112:0/1344767645 >> 172.21.15.112:0/1344767645 conn(0x7f61580785a0 msgr2=0x7f61580ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 -- 172.21.15.112:0/1344767645 shutdown_connections 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.004+0000 7f615ec80700 1 -- 172.21.15.112:0/1344767645 wait complete. 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 Processor -- start 2024-05-10T10:44:46.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 -- start start 2024-05-10T10:44:46.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61581076e0 0x7f615819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6158108300 0x7f615819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108cb0 0x7f6158197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6158111370 con 0x7f6158108cb0 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61581111f0 con 0x7f61581076e0 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.005+0000 7f615ec80700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61581114f0 con 0x7f6158108300 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61581076e0 0x7f615819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f615d21d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108cb0 0x7f6158197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61581076e0 0x7f615819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51134/0 (socket says 172.21.15.112:51134) 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f615ca1c700 1 -- 172.21.15.112:0/1980142579 learned_addr learned my addr 172.21.15.112:0/1980142579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f615d21d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108cb0 0x7f6158197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34042/0 (socket says 172.21.15.112:34042) 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6157fff700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6158108300 0x7f615819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6157fff700 1 -- 172.21.15.112:0/1980142579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61581076e0 msgr2=0x7f615819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6157fff700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61581076e0 0x7f615819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6157fff700 1 -- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108cb0 msgr2=0x7f6158197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6157fff700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108cb0 0x7f6158197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6157fff700 1 -- 172.21.15.112:0/1980142579 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6158197ff0 con 0x7f6158108300 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f615d21d700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108cb0 0x7f6158197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6157fff700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6158108300 0x7f615819db80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6144012be0 tx=0x7f6144012ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6155ffb700 1 -- 172.21.15.112:0/1980142579 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144010670 con 0x7f6158108300 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f6155ffb700 1 -- 172.21.15.112:0/1980142579 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f614401d070 con 0x7f6158108300 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.006+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6158198280 con 0x7f6158108300 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.007+0000 7f6155ffb700 1 -- 172.21.15.112:0/1980142579 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614400bc40 con 0x7f6158108300 2024-05-10T10:44:46.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.007+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581a45c0 con 0x7f6158108300 2024-05-10T10:44:46.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.010+0000 7f6155ffb700 1 -- 172.21.15.112:0/1980142579 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6144016050 con 0x7f6158108300 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 Processor -- start 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6155ffb700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f61480b2880 0x7f61480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 -- start start 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f615d21d700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f61480b2880 0x7f61480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6155ffb700 1 -- 172.21.15.112:0/1980142579 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f61440e6280 con 0x7f6158108300 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c31b0 0x5628d77c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c3ad0 0x5628d77bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5628d770f940 con 0x5628d77bdd50 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c31b0 0x5628d77c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5628d7674da0 con 0x5628d77c31b0 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6157fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c3ad0 0x5628d77bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c31b0 0x5628d77c3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51144/0 (socket says 172.21.15.112:51144) 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f615ca1c700 1 -- 172.21.15.112:0/4213020758 learned_addr learned my addr 172.21.15.112:0/4213020758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f6169513b80 1 -- 172.21.15.112:0/4213020758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5628d770b900 con 0x5628d77c3ad0 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.011+0000 7f615d21d700 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615d21d700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f61480b2880 0x7f61480b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6150003e80 tx=0x7f61500099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615ca1c700 1 -- 172.21.15.112:0/4213020758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c3ad0 msgr2=0x5628d77bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615ca1c700 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c3ad0 0x5628d77bd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615ca1c700 1 -- 172.21.15.112:0/4213020758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 msgr2=0x5628d77be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615ca1c700 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615ca1c700 1 -- 172.21.15.112:0/4213020758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628d765c8d0 con 0x5628d77c31b0 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615d21d700 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f6157fff700 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c3ad0 0x5628d77bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f615ca1c700 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c31b0 0x5628d77c3590 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f614c005d30 tx=0x7f614c00b300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5198ef0b5889561f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:46.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f6142ffd700 1 -- 172.21.15.112:0/4213020758 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c018070 con 0x5628d77c31b0 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f6142ffd700 1 -- 172.21.15.112:0/4213020758 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f614c019040 con 0x5628d77c31b0 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.012+0000 7f6142ffd700 1 -- 172.21.15.112:0/4213020758 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c005560 con 0x5628d77c31b0 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 -- 172.21.15.112:0/4213020758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c31b0 msgr2=0x5628d77c3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c31b0 0x5628d77c3590 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f614c005d30 tx=0x7f614c00b300 comp rx=0 tx=0).stop 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 -- 172.21.15.112:0/4213020758 shutdown_connections 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c31b0 0x5628d77c3590 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 --2- 172.21.15.112:0/4213020758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c3ad0 0x5628d77bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 -- 172.21.15.112:0/4213020758 >> 172.21.15.112:0/4213020758 conn(0x5628d76e9b70 msgr2=0x5628d77b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 -- 172.21.15.112:0/4213020758 shutdown_connections 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.013+0000 7f6169513b80 1 -- 172.21.15.112:0/4213020758 wait complete. 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 -- start start 2024-05-10T10:44:46.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 Processor -- start 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c31b0 0x5628d77c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5628d77ca3f0 con 0x5628d77bdd50 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5628d76c8ed0 con 0x5628d77c3ad0 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5628d77ca570 con 0x5628d77c31b0 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.014+0000 7f6169513b80 10 client.0 _ll_register_callbacks cb 0x5628d76a35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6157fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c31b0 0x5628d77c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34052/0 (socket says 172.21.15.112:34052) 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f615ca1c700 1 -- 172.21.15.112:0/2567723110 learned_addr learned my addr 172.21.15.112:0/2567723110 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f615d21d700 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6157fff700 1 -- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 msgr2=0x5628d77c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6157fff700 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6157fff700 1 -- 172.21.15.112:0/2567723110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 msgr2=0x5628d77c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6157fff700 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6157fff700 1 -- 172.21.15.112:0/2567723110 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628d76c6760 con 0x5628d77c31b0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f615ca1c700 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6157fff700 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c31b0 0x5628d77c69c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61440e39f0 tx=0x7f61440ada50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5f6db53eb9f4b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6137fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6137fff700 1 -- 172.21.15.112:0/2567723110 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440e60c0 con 0x5628d77c31b0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614c019040 con 0x5628d77c31b0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6137fff700 1 -- 172.21.15.112:0/2567723110 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f61440950d0 con 0x5628d77c31b0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6137fff700 1 -- 172.21.15.112:0/2567723110 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f61440ad020 con 0x5628d77c31b0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6169513b80 10 client.25933 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.015+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5628d77d4b90 con 0x5628d77c31b0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6137fff700 1 -- 172.21.15.112:0/2567723110 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f614c019040 con 0x5628d77c31b0 2024-05-10T10:44:46.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6137fff700 1 -- 172.21.15.112:0/2567723110 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f61440a8e80 con 0x5628d77c31b0 2024-05-10T10:44:46.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6169513b80 10 client.25933 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:46.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6169513b80 10 client.25933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:44:46.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6169513b80 20 client.25933 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:46.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6169513b80 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d77d7170 0x5628d77d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6169513b80 4 client.25933 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:46.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628d771d320 con 0x5628d77d7170 2024-05-10T10:44:46.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.016+0000 7f615ca1c700 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d77d7170 0x5628d77d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.017+0000 7f615ca1c700 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d77d7170 0x5628d77d95f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=213b1479568e7b5c server_cookie=62b95e5f64e191f8 in_seq=0 out_seq=0 2024-05-10T10:44:46.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.017+0000 7f6137fff700 10 client.25933 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:44:46.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6137fff700 1 -- 172.21.15.112:0/2567723110 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628d771d320 con 0x5628d77d7170 2024-05-10T10:44:46.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6137fff700 10 client.25933 handle_command_reply: tid=0 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 client.25933 shutdown 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c31b0 msgr2=0x5628d77c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c31b0 0x5628d77c69c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61440e39f0 tx=0x7f61440ada50 comp rx=0 tx=0).stop 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 shutdown_connections 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d77d7170 0x5628d77d95f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77bdd50 0x5628d77c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 --2- 172.21.15.112:0/2567723110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77c31b0 0x5628d77c69c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 >> 172.21.15.112:0/2567723110 conn(0x5628d76e9b70 msgr2=0x5628d77b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6137fff700 0 client.25933 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 shutdown_connections 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 1 -- 172.21.15.112:0/2567723110 wait complete. 2024-05-10T10:44:46.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.018+0000 7f6169513b80 20 client.25933 trim_cache size 0 max 16384 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.035+0000 7f6169513b80 1 Processor -- start 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.035+0000 7f6169513b80 1 -- start start 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6169513b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6169513b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6169513b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6169513b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61581acf80 con 0x5628d77c31b0 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6169513b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61581aec40 con 0x5628d77c3ad0 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6169513b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61581ad650 con 0x5628d77bdd50 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34066/0 (socket says 172.21.15.112:34066) 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 -- 172.21.15.112:0/1132849596 learned_addr learned my addr 172.21.15.112:0/1132849596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f615ca1c700 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f615d21d700 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 -- 172.21.15.112:0/1132849596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 msgr2=0x5628d77d3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77d3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 -- 172.21.15.112:0/1132849596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 msgr2=0x5628d77c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 -- 172.21.15.112:0/1132849596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628d77c6790 con 0x5628d77c31b0 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f615ca1c700 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77d3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f6157fff700 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77b2c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f61440e3d60 tx=0x7f61440e3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dded59209b3b1aac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.036+0000 7f61427fc700 1 -- 172.21.15.112:0/1132849596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440adc10 con 0x5628d77c31b0 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.037+0000 7f61427fc700 1 -- 172.21.15.112:0/1132849596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61440ad510 con 0x5628d77c31b0 2024-05-10T10:44:46.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.037+0000 7f61427fc700 1 -- 172.21.15.112:0/1132849596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614401fde0 con 0x5628d77c31b0 2024-05-10T10:44:46.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.037+0000 7f6169513b80 1 -- 172.21.15.112:0/1132849596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 msgr2=0x5628d77b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.037+0000 7f6169513b80 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77b2c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f61440e3d60 tx=0x7f61440e3a40 comp rx=0 tx=0).stop 2024-05-10T10:44:46.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.037+0000 7f6169513b80 1 -- 172.21.15.112:0/1132849596 shutdown_connections 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77d3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 --2- 172.21.15.112:0/1132849596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77b2c40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 -- 172.21.15.112:0/1132849596 >> 172.21.15.112:0/1132849596 conn(0x5628d76e9b70 msgr2=0x5628d77c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 -- 172.21.15.112:0/1132849596 shutdown_connections 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 -- 172.21.15.112:0/1132849596 wait complete. 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 -- start start 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 Processor -- start 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61581ae4b0 con 0x5628d77c31b0 2024-05-10T10:44:46.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.038+0000 7f6169513b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61581adde0 con 0x5628d77c3ad0 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6169513b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61581aaf20 con 0x5628d77bdd50 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6169513b80 10 client.0 _ll_register_callbacks cb 0x5628d76a35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6157fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6157fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34080/0 (socket says 172.21.15.112:34080) 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6157fff700 1 -- 172.21.15.112:0/1015507614 learned_addr learned my addr 172.21.15.112:0/1015507614 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:59934/0 (socket says 172.21.15.112:59934) 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615d21d700 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 -- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 msgr2=0x5628d77ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 -- 172.21.15.112:0/1015507614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 msgr2=0x5628d77cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 -- 172.21.15.112:0/1015507614 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628d77de800 con 0x5628d77bdd50 2024-05-10T10:44:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6157fff700 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f615ca1c700 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77cf9e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f614c005ed0 tx=0x7f614c0082e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd9996178636ccad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6137fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6137fff700 1 -- 172.21.15.112:0/1015507614 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c02f070 con 0x5628d77bdd50 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.039+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61440ad510 con 0x5628d77bdd50 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6137fff700 1 -- 172.21.15.112:0/1015507614 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f614c00ad00 con 0x5628d77bdd50 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6137fff700 1 -- 172.21.15.112:0/1015507614 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f614c038050 con 0x5628d77bdd50 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 10 client.25939 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5628d78415f0 con 0x5628d77bdd50 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6137fff700 1 -- 172.21.15.112:0/1015507614 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61440ad510 con 0x5628d77bdd50 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6137fff700 1 -- 172.21.15.112:0/1015507614 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f614c032020 con 0x5628d77bdd50 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 10 client.25939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:44:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 20 client.25939 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d7841870 0x5628d7843c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 4 client.25939 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.040+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628d77173c0 con 0x5628d7841870 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6157fff700 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d7841870 0x5628d7843c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6137fff700 10 client.25939 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6157fff700 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d7841870 0x5628d7843c60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63475d8c37815c8c server_cookie=8fdbf2a222bf67ef in_seq=0 out_seq=0 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6137fff700 1 -- 172.21.15.112:0/1015507614 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628d77173c0 con 0x5628d7841870 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6137fff700 10 client.25939 handle_command_reply: tid=0 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6169513b80 1 client.25939 shutdown 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 msgr2=0x5628d77cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.041+0000 7f6169513b80 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77cf9e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f614c005ed0 tx=0x7f614c0082e0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 shutdown_connections 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d77c3ad0 0x5628d77ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628d77bdd50 0x5628d77cf9e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x5628d7841870 0x5628d7843c60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 --2- 172.21.15.112:0/1015507614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5628d77c31b0 0x5628d77cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6137fff700 0 client.25939 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 >> 172.21.15.112:0/1015507614 conn(0x5628d76e9b70 msgr2=0x5628d77c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 shutdown_connections 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 1 -- 172.21.15.112:0/1015507614 wait complete. 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.042+0000 7f6169513b80 20 client.25939 trim_cache size 0 max 16384 2024-05-10T10:44:46.049 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:44:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f61480b2880 msgr2=0x7f61480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f61480b2880 0x7f61480b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6150003e80 tx=0x7f61500099e0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6158108300 msgr2=0x7f615819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6158108300 0x7f615819db80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6144012be0 tx=0x7f6144012ef0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 shutdown_connections 2024-05-10T10:44:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6158108cb0 0x7f6158197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61581076e0 0x7f615819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f61480b2880 0x7f61480b4d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 --2- 172.21.15.112:0/1980142579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6158108300 0x7f615819db80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 >> 172.21.15.112:0/1980142579 conn(0x7f61580785a0 msgr2=0x7f61581033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 shutdown_connections 2024-05-10T10:44:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:46.043+0000 7f615ec80700 1 -- 172.21.15.112:0/1980142579 wait complete. 2024-05-10T10:44:46.476 INFO:tasks.workunit.client.0.smithi112.stdout:10000 write 0x723fbd thru 0x733aaa (0xfaee bytes) 2024-05-10T10:44:46.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:44:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:47.381 INFO:tasks.workunit.client.0.smithi112.stdout:fallocating to largest ever: 0xa00000 2024-05-10T10:44:47.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:47 smithi112 ceph-mon[30100]: pgmap v566: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 610 op/s 2024-05-10T10:44:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:47 smithi148 ceph-mon[33568]: pgmap v566: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 610 op/s 2024-05-10T10:44:47.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:47 smithi195 ceph-mon[33773]: pgmap v566: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 610 op/s 2024-05-10T10:44:49.372 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:49 smithi112 ceph-mon[30100]: pgmap v567: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.7 MiB/s wr, 585 op/s 2024-05-10T10:44:49.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:49 smithi148 ceph-mon[33568]: pgmap v567: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.7 MiB/s wr, 585 op/s 2024-05-10T10:44:49.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:49 smithi195 ceph-mon[33773]: pgmap v567: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.7 MiB/s wr, 585 op/s 2024-05-10T10:44:49.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 -- 172.21.15.112:0/1177489114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 msgr2=0x7f12e0103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:49.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 --2- 172.21.15.112:0/1177489114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e0103b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f12d00099c0 tx=0x7f12d0009cd0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 -- 172.21.15.112:0/1177489114 shutdown_connections 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 --2- 172.21.15.112:0/1177489114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0108880 0x7f12e0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 --2- 172.21.15.112:0/1177489114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e0103b90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 --2- 172.21.15.112:0/1177489114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e0104160 0x7f12e0108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 -- 172.21.15.112:0/1177489114 >> 172.21.15.112:0/1177489114 conn(0x7f12e007a580 msgr2=0x7f12e007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 -- 172.21.15.112:0/1177489114 shutdown_connections 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 -- 172.21.15.112:0/1177489114 wait complete. 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.756+0000 7f12e5874700 1 Processor -- start 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12e5874700 1 -- start start 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12e5874700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12e5874700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e0104160 0x7f12e019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:49.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12e5874700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0108880 0x7f12e01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12e5874700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0113500 con 0x7f12e01037b0 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12e5874700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0113380 con 0x7f12e0104160 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12e5874700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0113680 con 0x7f12e0108880 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34098/0 (socket says 172.21.15.112:34098) 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 -- 172.21.15.112:0/45119523 learned_addr learned my addr 172.21.15.112:0/45119523 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12de7fc700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e0104160 0x7f12e019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 -- 172.21.15.112:0/45119523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0108880 msgr2=0x7f12e01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0108880 0x7f12e01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 -- 172.21.15.112:0/45119523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e0104160 msgr2=0x7f12e019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e0104160 0x7f12e019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 -- 172.21.15.112:0/45119523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e0199f80 con 0x7f12e01037b0 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.757+0000 7f12deffd700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e019f620 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f12d0005f50 tx=0x7f12d00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.758+0000 7f12e4872700 1 -- 172.21.15.112:0/45119523 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0027070 con 0x7f12e01037b0 2024-05-10T10:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.758+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e019a210 con 0x7f12e01037b0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.758+0000 7f12e4872700 1 -- 172.21.15.112:0/45119523 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d000fd60 con 0x7f12e01037b0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.758+0000 7f12e4872700 1 -- 172.21.15.112:0/45119523 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0022e30 con 0x7f12e01037b0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.758+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e01a6480 con 0x7f12e01037b0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.760+0000 7f12e4872700 1 -- 172.21.15.112:0/45119523 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f12d002a070 con 0x7f12e01037b0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.761+0000 7f12e4872700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f12c80b2880 0x7f12c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.761+0000 7f12de7fc700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f12c80b2880 0x7f12c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.761+0000 7f12e4872700 1 -- 172.21.15.112:0/45119523 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f12d00e32b0 con 0x7f12e01037b0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12de7fc700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f12c80b2880 0x7f12c80b4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f12e0103110 tx=0x7f12d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 Processor -- start 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 -- start start 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce4d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce47810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce48190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef2cd99940 con 0x55ef2ce47d50 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ccfeda0 con 0x55ef2ce4d1b0 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12f0107b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ef2cd95900 con 0x55ef2ce4dad0 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce48190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce48190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34104/0 (socket says 172.21.15.112:34104) 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 -- 172.21.15.112:0/2733320054 learned_addr learned my addr 172.21.15.112:0/2733320054 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12de7fc700 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce47810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 -- 172.21.15.112:0/2733320054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4dad0 msgr2=0x55ef2ce47810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce47810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12deffd700 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce4d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 -- 172.21.15.112:0/2733320054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4d1b0 msgr2=0x55ef2ce4d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce4d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.762+0000 7f12df7fe700 1 -- 172.21.15.112:0/2733320054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2cce68d0 con 0x55ef2ce47d50 2024-05-10T10:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12df7fe700 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce48190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f12d800d190 tx=0x7f12d800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a77943897a05c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12c5ffb700 1 -- 172.21.15.112:0/2733320054 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8013620 con 0x55ef2ce47d50 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12c5ffb700 1 -- 172.21.15.112:0/2733320054 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d8018070 con 0x55ef2ce47d50 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12c5ffb700 1 -- 172.21.15.112:0/2733320054 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d80128c0 con 0x55ef2ce47d50 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12f0107b80 1 -- 172.21.15.112:0/2733320054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 msgr2=0x55ef2ce48190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12f0107b80 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce48190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f12d800d190 tx=0x7f12d800d4a0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12f0107b80 1 -- 172.21.15.112:0/2733320054 shutdown_connections 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12f0107b80 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce48190 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12f0107b80 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce4d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12f0107b80 1 --2- 172.21.15.112:0/2733320054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce47810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.763+0000 7f12f0107b80 1 -- 172.21.15.112:0/2733320054 >> 172.21.15.112:0/2733320054 conn(0x55ef2cd73b70 msgr2=0x55ef2ce422a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 -- 172.21.15.112:0/2733320054 shutdown_connections 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 -- 172.21.15.112:0/2733320054 wait complete. 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 -- start start 2024-05-10T10:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 Processor -- start 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce50ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ce54450 con 0x55ef2ce47d50 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.764+0000 7f12f0107b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ef2cd52ed0 con 0x55ef2ce4dad0 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12f0107b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ce545d0 con 0x55ef2ce4d1b0 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12f0107b80 10 client.0 _ll_register_callbacks cb 0x55ef2cd2d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34114/0 (socket says 172.21.15.112:34114) 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 -- 172.21.15.112:0/3288191929 learned_addr learned my addr 172.21.15.112:0/3288191929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 -- 172.21.15.112:0/3288191929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 msgr2=0x55ef2ce50ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce50ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 -- 172.21.15.112:0/3288191929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 msgr2=0x55ef2ce510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 -- 172.21.15.112:0/3288191929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2cd50760 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12deffd700 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce537b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f12d00aa270 tx=0x7f12d00aaab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a051d54a38400d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12beffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12beffd700 1 -- 172.21.15.112:0/3288191929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00aa080 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12beffd700 1 -- 172.21.15.112:0/3288191929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f12d000f8d0 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d8018070 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12beffd700 1 -- 172.21.15.112:0/3288191929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f12d00aac70 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12f0107b80 10 client.16542 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef2ce63190 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.765+0000 7f12beffd700 1 -- 172.21.15.112:0/3288191929 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d8018070 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12beffd700 1 -- 172.21.15.112:0/3288191929 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f12d00050b0 con 0x55ef2ce47d50 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12f0107b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12f0107b80 10 client.16542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:44:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12f0107b80 20 client.16542 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ce65660 0x55ef2ce67ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12f0107b80 4 client.16542 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef2cda7320 con 0x55ef2ce65660 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.766+0000 7f12de7fc700 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ce65660 0x55ef2ce67ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.767+0000 7f12beffd700 10 client.16542 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.767+0000 7f12de7fc700 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ce65660 0x55ef2ce67ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=42fd2e57ebe7771a server_cookie=d72ad73f1e8ae268 in_seq=0 out_seq=0 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.767+0000 7f12beffd700 1 -- 172.21.15.112:0/3288191929 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef2cda7320 con 0x55ef2ce65660 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.767+0000 7f12beffd700 10 client.16542 handle_command_reply: tid=0 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.767+0000 7f12f0107b80 1 client.16542 shutdown 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 msgr2=0x55ef2ce537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce537b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f12d00aa270 tx=0x7f12d00aaab0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 shutdown_connections 2024-05-10T10:44:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ce65660 0x55ef2ce67ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce537b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3288191929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce50ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12beffd700 0 client.16542 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 >> 172.21.15.112:0/3288191929 conn(0x55ef2cd73b70 msgr2=0x55ef2ce3cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 shutdown_connections 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 1 -- 172.21.15.112:0/3288191929 wait complete. 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.768+0000 7f12f0107b80 20 client.16542 trim_cache size 0 max 16384 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 Processor -- start 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 -- start start 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce47d50 0x55ef2ce47810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce3cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01acd00 con 0x55ef2ce4d1b0 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01ae9c0 con 0x55ef2ce4dad0 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12f0107b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01ad3d0 con 0x55ef2ce47d50 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12de7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce47d50 0x55ef2ce47810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce47d50 0x55ef2ce47810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:59984/0 (socket says 172.21.15.112:59984) 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 -- 172.21.15.112:0/3617085859 learned_addr learned my addr 172.21.15.112:0/3617085859 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12df7fe700 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce3cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 -- 172.21.15.112:0/3617085859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 msgr2=0x55ef2ce3cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce3cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 -- 172.21.15.112:0/3617085859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce4d1b0 msgr2=0x55ef2ce5d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce5d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 -- 172.21.15.112:0/3617085859 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2ce4d7e0 con 0x55ef2ce47d50 2024-05-10T10:44:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12de7fc700 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce5d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.786+0000 7f12deffd700 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce47d50 0x55ef2ce47810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12d00a65f0 tx=0x7f12d0005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f0e21d0f5647d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12c57fa700 1 -- 172.21.15.112:0/3617085859 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00aa080 con 0x55ef2ce47d50 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12c57fa700 1 -- 172.21.15.112:0/3617085859 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d00aa570 con 0x55ef2ce47d50 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12c57fa700 1 -- 172.21.15.112:0/3617085859 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0090b00 con 0x55ef2ce47d50 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 -- 172.21.15.112:0/3617085859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce47d50 msgr2=0x55ef2ce47810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce47d50 0x55ef2ce47810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12d00a65f0 tx=0x7f12d0005c80 comp rx=0 tx=0).stop 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 -- 172.21.15.112:0/3617085859 shutdown_connections 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce3cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce47d50 0x55ef2ce47810 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 --2- 172.21.15.112:0/3617085859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce5d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 -- 172.21.15.112:0/3617085859 >> 172.21.15.112:0/3617085859 conn(0x55ef2cd73b70 msgr2=0x55ef2ce4e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 -- 172.21.15.112:0/3617085859 shutdown_connections 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.787+0000 7f12f0107b80 1 -- 172.21.15.112:0/3617085859 wait complete. 2024-05-10T10:44:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 -- start start 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 Processor -- start 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce6ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce6d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce6d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01ae230 con 0x55ef2ce47d50 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01adb60 con 0x55ef2ce4dad0 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01aaca0 con 0x55ef2ce4d1b0 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12f0107b80 10 client.0 _ll_register_callbacks cb 0x55ef2cd2d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce6ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12df7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce6d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12de7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce6d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce6ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34130/0 (socket says 172.21.15.112:34130) 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.788+0000 7f12deffd700 1 -- 172.21.15.112:0/1304717161 learned_addr learned my addr 172.21.15.112:0/1304717161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12deffd700 1 -- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 msgr2=0x55ef2ce6d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12deffd700 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce6d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12deffd700 1 -- 172.21.15.112:0/1304717161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 msgr2=0x55ef2ce6d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12deffd700 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce6d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12deffd700 1 -- 172.21.15.112:0/1304717161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2cecf240 con 0x55ef2ce47d50 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12de7fc700 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce6d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12df7fe700 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce6d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12deffd700 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce6ce80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f12d00aa500 tx=0x7f12d00052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86de4345b8ae5879 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12beffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12beffd700 1 -- 172.21.15.112:0/1304717161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00c9070 con 0x55ef2ce47d50 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12beffd700 1 -- 172.21.15.112:0/1304717161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f12d00a99e0 con 0x55ef2ce47d50 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d00aa570 con 0x55ef2ce47d50 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12beffd700 1 -- 172.21.15.112:0/1304717161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f12d00aa080 con 0x55ef2ce47d50 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12f0107b80 10 client.16548 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef2cecf240 con 0x55ef2ce47d50 2024-05-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.789+0000 7f12beffd700 1 -- 172.21.15.112:0/1304717161 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d00aa570 con 0x55ef2ce47d50 2024-05-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12beffd700 1 -- 172.21.15.112:0/1304717161 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f12d00a5ee0 con 0x55ef2ce47d50 2024-05-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12f0107b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12f0107b80 10 client.16548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12f0107b80 20 client.16548 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12f0107b80 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ced0780 0x55ef2ced2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12f0107b80 4 client.16548 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef2cda13c0 con 0x55ef2ced0780 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12de7fc700 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ced0780 0x55ef2ced2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12beffd700 10 client.16548 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12de7fc700 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ced0780 0x55ef2ced2b70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=413b7070c5b95b87 server_cookie=70230be2d93853e3 in_seq=0 out_seq=0 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12beffd700 1 -- 172.21.15.112:0/1304717161 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef2cda13c0 con 0x55ef2ced0780 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12beffd700 10 client.16548 handle_command_reply: tid=0 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.790+0000 7f12f0107b80 1 client.16548 shutdown 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 msgr2=0x55ef2ce6ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce6ce80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f12d00aa500 tx=0x7f12d00052e0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 shutdown_connections 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55ef2ced0780 0x55ef2ced2b70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ef2ce4dad0 0x55ef2ce6d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef2ce47d50 0x55ef2ce6ce80 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 --2- 172.21.15.112:0/1304717161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef2ce4d1b0 0x55ef2ce6d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 >> 172.21.15.112:0/1304717161 conn(0x55ef2cd73b70 msgr2=0x55ef2ce4c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12beffd700 0 client.16548 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 shutdown_connections 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 1 -- 172.21.15.112:0/1304717161 wait complete. 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.791+0000 7f12f0107b80 20 client.16548 trim_cache size 0 max 16384 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f12c80b2880 msgr2=0x7f12c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f12c80b2880 0x7f12c80b4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f12e0103110 tx=0x7f12d400b410 comp rx=0 tx=0).stop 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 msgr2=0x7f12e019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e019f620 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f12d0005f50 tx=0x7f12d00057c0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 shutdown_connections 2024-05-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f12c80b2880 0x7f12c80b4d40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0108880 0x7f12e01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12e01037b0 0x7f12e019f620 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 --2- 172.21.15.112:0/45119523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e0104160 0x7f12e019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 >> 172.21.15.112:0/45119523 conn(0x7f12e007a580 msgr2=0x7f12e0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 shutdown_connections 2024-05-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:49.792+0000 7f12e5874700 1 -- 172.21.15.112:0/45119523 wait complete. 2024-05-10T10:44:51.435 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:51 smithi112 ceph-mon[30100]: pgmap v568: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 519 op/s 2024-05-10T10:44:51.467 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:44:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:51 smithi148 ceph-mon[33568]: pgmap v568: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 519 op/s 2024-05-10T10:44:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:51 smithi195 ceph-mon[33773]: pgmap v568: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 519 op/s 2024-05-10T10:44:51.748 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:53.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:53 smithi112 ceph-mon[30100]: pgmap v569: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.1 MiB/s wr, 753 op/s 2024-05-10T10:44:53.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:53 smithi148 ceph-mon[33568]: pgmap v569: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.1 MiB/s wr, 753 op/s 2024-05-10T10:44:53.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:53 smithi195 ceph-mon[33773]: pgmap v569: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.1 MiB/s wr, 753 op/s 2024-05-10T10:44:53.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 -- 172.21.15.112:0/4242921169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 msgr2=0x7faa60110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 --2- 172.21.15.112:0/4242921169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa60110340 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7faa5800b0a0 tx=0x7faa5800b3b0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 -- 172.21.15.112:0/4242921169 shutdown_connections 2024-05-10T10:44:53.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 --2- 172.21.15.112:0/4242921169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa60110340 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 --2- 172.21.15.112:0/4242921169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa60109520 0x7faa60109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 --2- 172.21.15.112:0/4242921169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa60101a90 0x7faa60101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 -- 172.21.15.112:0/4242921169 >> 172.21.15.112:0/4242921169 conn(0x7faa60078a10 msgr2=0x7faa600ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:53.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 -- 172.21.15.112:0/4242921169 shutdown_connections 2024-05-10T10:44:53.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.634+0000 7faa6557e700 1 -- 172.21.15.112:0/4242921169 wait complete. 2024-05-10T10:44:53.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.635+0000 7faa6557e700 1 Processor -- start 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.635+0000 7faa6557e700 1 -- start start 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa6557e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa60101a90 0x7faa6019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa6557e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa6019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa6557e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa60109520 0x7faa60199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa6557e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa60113500 con 0x7faa60102450 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa6557e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa60113380 con 0x7faa60101a90 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa6557e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faa60113680 con 0x7faa60109520 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa6019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa6019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34138/0 (socket says 172.21.15.112:34138) 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5e7fc700 1 -- 172.21.15.112:0/1323477413 learned_addr learned my addr 172.21.15.112:0/1323477413 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa60109520 0x7faa60199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5effd700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa60101a90 0x7faa6019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5effd700 1 -- 172.21.15.112:0/1323477413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa60109520 msgr2=0x7faa60199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5effd700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa60109520 0x7faa60199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5effd700 1 -- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 msgr2=0x7faa6019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5effd700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa6019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5effd700 1 -- 172.21.15.112:0/1323477413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6019a000 con 0x7faa60101a90 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa60109520 0x7faa60199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa6019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.636+0000 7faa5effd700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa60101a90 0x7faa6019f6a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa5000e9f0 tx=0x7faa5000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.637+0000 7faa47fff700 1 -- 172.21.15.112:0/1323477413 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5000c9c0 con 0x7faa60101a90 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.637+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6019a290 con 0x7faa60101a90 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.637+0000 7faa47fff700 1 -- 172.21.15.112:0/1323477413 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa50010430 con 0x7faa60101a90 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.637+0000 7faa47fff700 1 -- 172.21.15.112:0/1323477413 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50018680 con 0x7faa60101a90 2024-05-10T10:44:53.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.637+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa601a6500 con 0x7faa60101a90 2024-05-10T10:44:53.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.640+0000 7faa47fff700 1 -- 172.21.15.112:0/1323477413 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7faa50018820 con 0x7faa60101a90 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.640+0000 7faa6fe11b80 1 Processor -- start 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.640+0000 7faa47fff700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faa480b2880 0x7faa480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.640+0000 7faa6fe11b80 1 -- start start 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faa480b2880 0x7faa480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa47fff700 1 -- 172.21.15.112:0/1323477413 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7faa500e26a0 con 0x7faa60101a90 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f0089970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f0083c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0084610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5652effd1940 con 0x5652f0089590 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5652eff36da0 con 0x5652f0089eb0 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5652effcd900 con 0x5652f00841b0 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0084610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f0089970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0084610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60026/0 (socket says 172.21.15.112:60026) 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5f7fe700 1 -- 172.21.15.112:0/3050731204 learned_addr learned my addr 172.21.15.112:0/3050731204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f0083c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5e7fc700 1 -- 172.21.15.112:0/3050731204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 msgr2=0x5652f0084610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0084610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5e7fc700 1 -- 172.21.15.112:0/3050731204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 msgr2=0x5652f0089970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f0089970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5e7fc700 1 -- 172.21.15.112:0/3050731204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652eff1e8d0 con 0x5652f0089eb0 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0084610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5effd700 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f0089970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:53.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.641+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faa480b2880 0x7faa480b4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa58009550 tx=0x7faa58009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:53.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.642+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f0083c20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa60108c70 tx=0x7faa540118a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a5536911790525a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:53.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.642+0000 7faa457fa700 1 -- 172.21.15.112:0/3050731204 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5401e070 con 0x5652f0089eb0 2024-05-10T10:44:53.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.642+0000 7faa457fa700 1 -- 172.21.15.112:0/3050731204 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa54019ac0 con 0x5652f0089eb0 2024-05-10T10:44:53.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.642+0000 7faa457fa700 1 -- 172.21.15.112:0/3050731204 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5400fa40 con 0x5652f0089eb0 2024-05-10T10:44:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3050731204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 msgr2=0x5652f0083c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f0083c20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa60108c70 tx=0x7faa540118a0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3050731204 shutdown_connections 2024-05-10T10:44:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0084610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f0089970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3050731204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f0083c20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3050731204 >> 172.21.15.112:0/3050731204 conn(0x5652effabb70 msgr2=0x5652f007a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3050731204 shutdown_connections 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3050731204 wait complete. 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 -- start start 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.643+0000 7faa6fe11b80 1 Processor -- start 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f008fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f008d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5652f00908e0 con 0x5652f0089590 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5652eff8aed0 con 0x5652f0089eb0 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5652f0090a60 con 0x5652f00841b0 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 10 client.0 _ll_register_callbacks cb 0x5652eff655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34150/0 (socket says 172.21.15.112:34150) 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5e7fc700 1 -- 172.21.15.112:0/85092723 learned_addr learned my addr 172.21.15.112:0/85092723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5effd700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f008fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f008d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5effd700 1 -- 172.21.15.112:0/85092723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 msgr2=0x5652f008d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5effd700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f008d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5effd700 1 -- 172.21.15.112:0/85092723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 msgr2=0x5652f008cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5effd700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5effd700 1 -- 172.21.15.112:0/85092723 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652eff88760 con 0x5652f00841b0 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f008d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa5effd700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f008fc10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa5000e9c0 tx=0x7faa50009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5579cb1d0825eba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa42ff5700 1 -- 172.21.15.112:0/85092723 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa500a9440 con 0x5652f00841b0 2024-05-10T10:44:53.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa42ff5700 1 -- 172.21.15.112:0/85092723 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7faa500cc0b0 con 0x5652f00841b0 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.644+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa54019ac0 con 0x5652f00841b0 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa42ff5700 1 -- 172.21.15.112:0/85092723 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faa500aa030 con 0x5652f00841b0 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 10 client.25960 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5652f009b290 con 0x5652f00841b0 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa42ff5700 1 -- 172.21.15.112:0/85092723 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa54019ac0 con 0x5652f00841b0 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa42ff5700 1 -- 172.21.15.112:0/85092723 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7faa500cabe0 con 0x5652f00841b0 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 10 client.25960 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 20 client.25960 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f009d920 0x5652f009fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 4 client.25960 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652effdf320 con 0x5652f009d920 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.645+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f009d920 0x5652f009fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.646+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f009d920 0x5652f009fdc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d2256201e14392 server_cookie=82e567033a08a1ad in_seq=0 out_seq=0 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.646+0000 7faa42ff5700 10 client.25960 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa42ff5700 1 -- 172.21.15.112:0/85092723 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652effdf320 con 0x5652f009d920 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa42ff5700 10 client.25960 handle_command_reply: tid=0 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 client.25960 shutdown 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 msgr2=0x5652f008fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f008fc10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa5000e9c0 tx=0x7faa50009790 comp rx=0 tx=0).stop 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 shutdown_connections 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f009d920 0x5652f009fdc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089eb0 0x5652f008d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f008fc10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/85092723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 >> 172.21.15.112:0/85092723 conn(0x5652effabb70 msgr2=0x5652f0074b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa42ff5700 0 client.25960 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:53.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 shutdown_connections 2024-05-10T10:44:53.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 1 -- 172.21.15.112:0/85092723 wait complete. 2024-05-10T10:44:53.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.647+0000 7faa6fe11b80 20 client.25960 trim_cache size 0 max 16384 2024-05-10T10:44:53.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 Processor -- start 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 -- start start 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652effabb70 0x5652f0074c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0099970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa601acf40 con 0x5652f0089590 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa601aec00 con 0x5652effabb70 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.663+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faa601ad610 con 0x5652f00841b0 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34166/0 (socket says 172.21.15.112:34166) 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 -- 172.21.15.112:0/3437780103 learned_addr learned my addr 172.21.15.112:0/3437780103 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0099970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5effd700 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652effabb70 0x5652f0074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 -- 172.21.15.112:0/3437780103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 msgr2=0x5652f0099970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0099970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 -- 172.21.15.112:0/3437780103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652effabb70 msgr2=0x5652f0074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652effabb70 0x5652f0074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 -- 172.21.15.112:0/3437780103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f0084950 con 0x5652f0089590 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5effd700 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652effabb70 0x5652f0074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa5f7fe700 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008a9a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faa580077f0 tx=0x7faa58029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afec9b1d6dc7b0ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.664+0000 7faa44ff9700 1 -- 172.21.15.112:0/3437780103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58004bb0 con 0x5652f0089590 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa44ff9700 1 -- 172.21.15.112:0/3437780103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa58004d50 con 0x5652f0089590 2024-05-10T10:44:53.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa44ff9700 1 -- 172.21.15.112:0/3437780103 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58005280 con 0x5652f0089590 2024-05-10T10:44:53.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3437780103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 msgr2=0x5652f008a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008a9a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faa580077f0 tx=0x7faa58029000 comp rx=0 tx=0).stop 2024-05-10T10:44:53.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3437780103 shutdown_connections 2024-05-10T10:44:53.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f0089590 0x5652f008a9a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652effabb70 0x5652f0074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/3437780103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652f00841b0 0x5652f0099970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3437780103 >> 172.21.15.112:0/3437780103 conn(0x5652f0087820 msgr2=0x5652f008a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3437780103 shutdown_connections 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.665+0000 7faa6fe11b80 1 -- 172.21.15.112:0/3437780103 wait complete. 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 -- start start 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 Processor -- start 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652effabb70 0x5652f0095cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f00841b0 0x5652f0096230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089590 0x5652f00a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa601ae470 con 0x5652f00841b0 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa601adda0 con 0x5652f0089590 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faa601aaee0 con 0x5652effabb70 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa6fe11b80 10 client.0 _ll_register_callbacks cb 0x5652eff655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:53.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f00841b0 0x5652f0096230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa5f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089590 0x5652f00a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f00841b0 0x5652f0096230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34168/0 (socket says 172.21.15.112:34168) 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa5e7fc700 1 -- 172.21.15.112:0/635065994 learned_addr learned my addr 172.21.15.112:0/635065994 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.666+0000 7faa5effd700 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652effabb70 0x5652f0095cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa5e7fc700 1 -- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652effabb70 msgr2=0x5652f0095cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652effabb70 0x5652f0095cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa5e7fc700 1 -- 172.21.15.112:0/635065994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089590 msgr2=0x5652f00a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089590 0x5652f00a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa5e7fc700 1 -- 172.21.15.112:0/635065994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f00a4f30 con 0x5652f00841b0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa5e7fc700 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f00841b0 0x5652f0096230 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7faa54000c00 tx=0x7faa540195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e4e35f80864d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa42ff5700 1 -- 172.21.15.112:0/635065994 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5401e040 con 0x5652f00841b0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa42ff5700 1 -- 172.21.15.112:0/635065994 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7faa54005240 con 0x5652f00841b0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa58004d50 con 0x5652f00841b0 2024-05-10T10:44:53.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa42ff5700 1 -- 172.21.15.112:0/635065994 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faa5403e050 con 0x5652f00841b0 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa6fe11b80 10 client.16566 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5652f0107d40 con 0x5652f00841b0 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa42ff5700 1 -- 172.21.15.112:0/635065994 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa58004d50 con 0x5652f00841b0 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.667+0000 7faa42ff5700 1 -- 172.21.15.112:0/635065994 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7faa54004ba0 con 0x5652f00841b0 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 10 client.16566 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 20 client.16566 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f0089eb0 0x5652f010a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 4 client.16566 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652effd93c0 con 0x5652f0089eb0 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa5effd700 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f0089eb0 0x5652f010a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa42ff5700 10 client.16566 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa5effd700 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f0089eb0 0x5652f010a340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37904e73e7dcc5d3 server_cookie=e3a1482d549471d in_seq=0 out_seq=0 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa42ff5700 1 -- 172.21.15.112:0/635065994 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652effd93c0 con 0x5652f0089eb0 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa42ff5700 10 client.16566 handle_command_reply: tid=0 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 client.16566 shutdown 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f00841b0 msgr2=0x5652f0096230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f00841b0 0x5652f0096230 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7faa54000c00 tx=0x7faa540195a0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 shutdown_connections 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5652f0089eb0 0x5652f010a340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5652f0089590 0x5652f00a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5652effabb70 0x5652f0095cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 --2- 172.21.15.112:0/635065994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5652f00841b0 0x5652f0096230 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.668+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 >> 172.21.15.112:0/635065994 conn(0x5652f0087820 msgr2=0x5652f008ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa42ff5700 0 client.16566 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 shutdown_connections 2024-05-10T10:44:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6fe11b80 1 -- 172.21.15.112:0/635065994 wait complete. 2024-05-10T10:44:53.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6fe11b80 20 client.16566 trim_cache size 0 max 16384 2024-05-10T10:44:53.676 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:44:53.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faa480b2880 msgr2=0x7faa480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faa480b2880 0x7faa480b4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa58009550 tx=0x7faa58009580 comp rx=0 tx=0).stop 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa60101a90 msgr2=0x7faa6019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa60101a90 0x7faa6019f6a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa5000e9f0 tx=0x7faa5000ed00 comp rx=0 tx=0).stop 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 shutdown_connections 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7faa480b2880 0x7faa480b4d40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa60109520 0x7faa60199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa60101a90 0x7faa6019f6a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 --2- 172.21.15.112:0/1323477413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa60102450 0x7faa6019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.669+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 >> 172.21.15.112:0/1323477413 conn(0x7faa60078a10 msgr2=0x7faa60100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.670+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 shutdown_connections 2024-05-10T10:44:53.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:53.670+0000 7faa6557e700 1 -- 172.21.15.112:0/1323477413 wait complete. 2024-05-10T10:44:55.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:55 smithi148 ceph-mon[33568]: pgmap v570: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 489 op/s 2024-05-10T10:44:55.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:55 smithi195 ceph-mon[33773]: pgmap v570: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 489 op/s 2024-05-10T10:44:55.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:55 smithi112 ceph-mon[30100]: pgmap v570: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 489 op/s 2024-05-10T10:44:55.817 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:44:55.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:44:56.112 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:44:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 -- 172.21.15.112:0/797977968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 msgr2=0x7f3c48102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 --2- 172.21.15.112:0/797977968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48102f80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009900 tx=0x7f3c3002a410 comp rx=0 tx=0).stop 2024-05-10T10:44:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 -- 172.21.15.112:0/797977968 shutdown_connections 2024-05-10T10:44:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 --2- 172.21.15.112:0/797977968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c481034c0 0x7f3c48109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 --2- 172.21.15.112:0/797977968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48102f80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 --2- 172.21.15.112:0/797977968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c48101910 0x7f3c48101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 -- 172.21.15.112:0/797977968 >> 172.21.15.112:0/797977968 conn(0x7f3c480fce80 msgr2=0x7f3c480ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 -- 172.21.15.112:0/797977968 shutdown_connections 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.965+0000 7f3c4d464640 1 -- 172.21.15.112:0/797977968 wait complete. 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 Processor -- start 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 -- start start 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48101910 0x7f3c48197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c481034c0 0x7f3c4819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4810d2f0 con 0x7f3c48102b00 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4810d170 con 0x7f3c48101910 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c4d464640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4810d470 con 0x7f3c481034c0 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48101910 0x7f3c48197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47100/0 (socket says 172.21.15.112:47100) 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c3ffff640 1 -- 172.21.15.112:0/3860672482 learned_addr learned my addr 172.21.15.112:0/3860672482 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.966+0000 7f3c477fe640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c481034c0 0x7f3c4819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c3ffff640 1 -- 172.21.15.112:0/3860672482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c481034c0 msgr2=0x7f3c4819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c481034c0 0x7f3c4819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c3ffff640 1 -- 172.21.15.112:0/3860672482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48101910 msgr2=0x7f3c48197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48101910 0x7f3c48197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c3ffff640 1 -- 172.21.15.112:0/3860672482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4819f660 con 0x7f3c48102b00 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48101910 0x7f3c48197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48197f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009230 tx=0x7f3c30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c44ff9640 1 -- 172.21.15.112:0/3860672482 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30046070 con 0x7f3c48102b00 2024-05-10T10:44:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4806a060 con 0x7f3c48102b00 2024-05-10T10:44:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c44ff9640 1 -- 172.21.15.112:0/3860672482 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c30032e10 con 0x7f3c48102b00 2024-05-10T10:44:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c44ff9640 1 -- 172.21.15.112:0/3860672482 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30035050 con 0x7f3c48102b00 2024-05-10T10:44:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.967+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4806a550 con 0x7f3c48102b00 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c44ff9640 1 -- 172.21.15.112:0/3860672482 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3c30049070 con 0x7f3c48102b00 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 Processor -- start 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 -- start start 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eaa91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56461ea0d5d0 con 0x56461eaa97a0 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56461e9c6890 con 0x56461eaa8db0 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c5017e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56461ea09590 con 0x56461eaae910 2024-05-10T10:44:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47112/0 (socket says 172.21.15.112:47112) 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eaa91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c46ffd640 1 -- 172.21.15.112:0/2873468588 learned_addr learned my addr 172.21.15.112:0/2873468588 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c46ffd640 1 -- 172.21.15.112:0/2873468588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 msgr2=0x56461eab0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c46ffd640 1 -- 172.21.15.112:0/2873468588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 msgr2=0x56461eaa91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.970+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eaa91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c46ffd640 1 -- 172.21.15.112:0/2873468588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461e71f210 con 0x56461eaa97a0 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c477fe640 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eaa91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c44ff9640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c340b28a0 0x7f3c340b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaae2a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3c380114f0 tx=0x7f3c380119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e121c8695e14f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c477fe640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c340b28a0 0x7f3c340b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c44ff9640 1 -- 172.21.15.112:0/3860672482 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3c301040c0 con 0x7f3c48102b00 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c3d7fa640 1 -- 172.21.15.112:0/2873468588 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3801e070 con 0x56461eaa97a0 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c3d7fa640 1 -- 172.21.15.112:0/2873468588 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c380035f0 con 0x56461eaa97a0 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.971+0000 7f3c3d7fa640 1 -- 172.21.15.112:0/2873468588 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380044c0 con 0x56461eaa97a0 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 -- 172.21.15.112:0/2873468588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 msgr2=0x56461eaae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaae2a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3c380114f0 tx=0x7f3c380119b0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c477fe640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c340b28a0 0x7f3c340b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56461eaaa000 tx=0x7f3c40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 -- 172.21.15.112:0/2873468588 shutdown_connections 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaae2a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 --2- 172.21.15.112:0/2873468588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eaa91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 -- 172.21.15.112:0/2873468588 >> 172.21.15.112:0/2873468588 conn(0x56461eaa4420 msgr2=0x56461eaa4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 -- 172.21.15.112:0/2873468588 shutdown_connections 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.972+0000 7f3c5017e740 1 -- 172.21.15.112:0/2873468588 wait complete. 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 -- start start 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 Processor -- start 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaa97a0 0x56461eab3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaae910 0x56461eaba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56461eab18c0 con 0x56461eaae910 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56461e9ca370 con 0x56461eaa8db0 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56461eab1a40 con 0x56461eaa97a0 2024-05-10T10:44:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c5017e740 10 client.0 _ll_register_callbacks cb 0x56461e71ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaa97a0 0x56461eab3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:42158/0 (socket says 172.21.15.112:42158) 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c46ffd640 1 -- 172.21.15.112:0/4217249720 learned_addr learned my addr 172.21.15.112:0/4217249720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.973+0000 7f3c477fe640 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaae910 0x56461eaba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.974+0000 7f3c3ffff640 1 -- 172.21.15.112:0/4217249720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 msgr2=0x56461eab3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.974+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.974+0000 7f3c3ffff640 1 -- 172.21.15.112:0/4217249720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaae910 msgr2=0x56461eaba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.974+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaae910 0x56461eaba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.974+0000 7f3c3ffff640 1 -- 172.21.15.112:0/4217249720 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461e7eb620 con 0x56461eaa97a0 2024-05-10T10:44:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.975+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaa97a0 0x56461eab3540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c30101880 tx=0x7f3c30005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb09ddbd5d89912 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.975+0000 7f3c2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:44:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.975+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/4217249720 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c300c7210 con 0x56461eaa97a0 2024-05-10T10:44:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.975+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/4217249720 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3c300d3f30 con 0x56461eaa97a0 2024-05-10T10:44:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.975+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c380035f0 con 0x56461eaa97a0 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/4217249720 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3c300caac0 con 0x56461eaa97a0 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 10 client.25978 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56461eab1930 con 0x56461eaa97a0 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/4217249720 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c380035f0 con 0x56461eaa97a0 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/4217249720 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f3c300c6cc0 con 0x56461eaa97a0 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 10 client.25978 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 10 client.25978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 20 client.25978 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eabee60 0x56461eac12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 4 client.25978 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.976+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56461ea1b1e0 con 0x56461eabee60 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.977+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eabee60 0x56461eac12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.977+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eabee60 0x56461eac12e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9684f5500bf93f10 server_cookie=d9c43712daea145b in_seq=0 out_seq=0 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.977+0000 7f3c2e7fc640 10 client.25978 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/4217249720 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56461ea1b1e0 con 0x56461eabee60 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c2e7fc640 10 client.25978 handle_command_reply: tid=0 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 client.25978 shutdown 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaa97a0 msgr2=0x56461eab3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaa97a0 0x56461eab3540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c30101880 tx=0x7f3c30005e20 comp rx=0 tx=0).stop 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 shutdown_connections 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaae910 0x56461eaba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaa97a0 0x56461eab3540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eabee60 0x56461eac12e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 --2- 172.21.15.112:0/4217249720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c2e7fc640 0 client.25978 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 >> 172.21.15.112:0/4217249720 conn(0x56461eaa4420 msgr2=0x56461eaa3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 shutdown_connections 2024-05-10T10:44:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 1 -- 172.21.15.112:0/4217249720 wait complete. 2024-05-10T10:44:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.978+0000 7f3c5017e740 20 client.25978 trim_cache size 0 max 16384 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 Processor -- start 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 -- start start 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaa7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eaa7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481a9290 con 0x56461eaa97a0 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481aaf50 con 0x56461eaa8db0 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c5017e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481a9960 con 0x56461eaae910 2024-05-10T10:44:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaa7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaa7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47122/0 (socket says 172.21.15.112:47122) 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.998+0000 7f3c3ffff640 1 -- 172.21.15.112:0/934353764 learned_addr learned my addr 172.21.15.112:0/934353764 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c477fe640 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eaa7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3ffff640 1 -- 172.21.15.112:0/934353764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 msgr2=0x56461eaa7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eaa7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3ffff640 1 -- 172.21.15.112:0/934353764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 msgr2=0x56461eab2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3ffff640 1 -- 172.21.15.112:0/934353764 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461eaa4250 con 0x56461eaa97a0 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaa7100 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3c30101ba0 tx=0x7f3c30107980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bc9826806212bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3cff9640 1 -- 172.21.15.112:0/934353764 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c300caca0 con 0x56461eaa97a0 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3cff9640 1 -- 172.21.15.112:0/934353764 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c300cae60 con 0x56461eaa97a0 2024-05-10T10:44:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:56.999+0000 7f3c3cff9640 1 -- 172.21.15.112:0/934353764 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c300cb120 con 0x56461eaa97a0 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 -- 172.21.15.112:0/934353764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 msgr2=0x56461eaa7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaa7100 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3c30101ba0 tx=0x7f3c30107980 comp rx=0 tx=0).stop 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 -- 172.21.15.112:0/934353764 shutdown_connections 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eaa7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eaa7100 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 --2- 172.21.15.112:0/934353764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 -- 172.21.15.112:0/934353764 >> 172.21.15.112:0/934353764 conn(0x56461eaa4420 msgr2=0x56461eab22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 -- 172.21.15.112:0/934353764 shutdown_connections 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 -- 172.21.15.112:0/934353764 wait complete. 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 -- start start 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.000+0000 7f3c5017e740 1 Processor -- start 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eab7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c5017e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c5017e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481aa7c0 con 0x56461eaa97a0 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c5017e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481aa0f0 con 0x56461eaa8db0 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c5017e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481a7270 con 0x56461eaae910 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c5017e740 10 client.0 _ll_register_callbacks cb 0x7f3c30107aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eab7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eab7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47136/0 (socket says 172.21.15.112:47136) 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:42170/0 (socket says 172.21.15.112:42170) 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c3ffff640 1 -- 172.21.15.112:0/768219552 learned_addr learned my addr 172.21.15.112:0/768219552 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c477fe640 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 -- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 msgr2=0x56461eab8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 -- 172.21.15.112:0/768219552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 msgr2=0x56461eab7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eab7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 -- 172.21.15.112:0/768219552 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461eab8f10 con 0x56461eaa8db0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eab7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.001+0000 7f3c46ffd640 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab73d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c38014040 tx=0x7f3c38005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df585625c3a1ce92 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/768219552 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3801e040 con 0x56461eaa8db0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/768219552 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3c380035f0 con 0x56461eaa8db0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c300cae60 con 0x56461eaa8db0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/768219552 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3c38039050 con 0x56461eaa8db0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 10 client.25958 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56461eb298d0 con 0x56461eaa8db0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/768219552 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c300cae60 con 0x56461eaa8db0 2024-05-10T10:44:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/768219552 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f3c3802a540 con 0x56461eaa8db0 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 10 client.25958 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 10 client.25958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 20 client.25958 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eb2a100 0x56461eb2c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 4 client.25958 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56461ea15050 con 0x56461eb2a100 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.002+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eb2a100 0x56461eb2c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.003+0000 7f3c2e7fc640 10 client.25958 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.003+0000 7f3c3ffff640 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eb2a100 0x56461eb2c520 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d677dd2354d2833e server_cookie=b30a793d9d14491e in_seq=0 out_seq=0 2024-05-10T10:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c2e7fc640 1 -- 172.21.15.112:0/768219552 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56461ea15050 con 0x56461eb2a100 2024-05-10T10:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c2e7fc640 10 client.25958 handle_command_reply: tid=0 2024-05-10T10:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 client.25958 shutdown 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 msgr2=0x56461eab73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab73d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c38014040 tx=0x7f3c38005f50 comp rx=0 tx=0).stop 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 shutdown_connections 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56461eaae910 0x56461eab8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56461eaa97a0 0x56461eab7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56461eb2a100 0x56461eb2c520 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 --2- 172.21.15.112:0/768219552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56461eaa8db0 0x56461eab73d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c2e7fc640 0 client.25958 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.021+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 >> 172.21.15.112:0/768219552 conn(0x56461eaa4420 msgr2=0x56461eab22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 shutdown_connections 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c5017e740 1 -- 172.21.15.112:0/768219552 wait complete. 2024-05-10T10:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c5017e740 20 client.25958 trim_cache size 0 max 16384 2024-05-10T10:44:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:44:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:44:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:44:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c340b28a0 msgr2=0x7f3c340b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c340b28a0 0x7f3c340b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56461eaaa000 tx=0x7f3c40009500 comp rx=0 tx=0).stop 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 msgr2=0x7f3c48197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48197f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009230 tx=0x7f3c30004270 comp rx=0 tx=0).stop 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 shutdown_connections 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3c340b28a0 0x7f3c340b4d60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c481034c0 0x7f3c4819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c48102b00 0x7f3c48197f80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 --2- 172.21.15.112:0/3860672482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c48101910 0x7f3c48197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 >> 172.21.15.112:0/3860672482 conn(0x7f3c480fce80 msgr2=0x7f3c48105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 shutdown_connections 2024-05-10T10:44:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:44:57.022+0000 7f3c4d464640 1 -- 172.21.15.112:0/3860672482 wait complete. 2024-05-10T10:44:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:57 smithi148 ceph-mon[33568]: pgmap v571: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 648 op/s 2024-05-10T10:44:57.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:57 smithi195 ceph-mon[33773]: pgmap v571: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 648 op/s 2024-05-10T10:44:57.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:57 smithi112 ceph-mon[30100]: pgmap v571: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 648 op/s 2024-05-10T10:44:59.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:44:59 smithi112 ceph-mon[30100]: pgmap v572: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.6 MiB/s wr, 559 op/s 2024-05-10T10:44:59.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 -- 172.21.15.112:0/2787068691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480108410 msgr2=0x7f14801087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 --2- 172.21.15.112:0/2787068691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480108410 0x7f14801087f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f147c009990 tx=0x7f147c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 -- 172.21.15.112:0/2787068691 shutdown_connections 2024-05-10T10:44:59.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 --2- 172.21.15.112:0/2787068691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1480102d60 0x7f148010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 --2- 172.21.15.112:0/2787068691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480108410 0x7f14801087f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 --2- 172.21.15.112:0/2787068691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14801023a0 0x7f1480102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 -- 172.21.15.112:0/2787068691 >> 172.21.15.112:0/2787068691 conn(0x7f1480079350 msgr2=0x7f1480079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:59.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 -- 172.21.15.112:0/2787068691 shutdown_connections 2024-05-10T10:44:59.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.225+0000 7f1487392700 1 -- 172.21.15.112:0/2787068691 wait complete. 2024-05-10T10:44:59.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 Processor -- start 2024-05-10T10:44:59.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 -- start start 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14801023a0 0x7f148019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480102d60 0x7f148019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1480108410 0x7f14801a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1480112410 con 0x7f1480102d60 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1480112290 con 0x7f1480108410 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f1487392700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1480112590 con 0x7f14801023a0 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480102d60 0x7f148019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480102d60 0x7f148019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47156/0 (socket says 172.21.15.112:47156) 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 -- 172.21.15.112:0/2476176109 learned_addr learned my addr 172.21.15.112:0/2476176109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148592f700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1480108410 0x7f14801a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148512e700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14801023a0 0x7f148019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 -- 172.21.15.112:0/2476176109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14801023a0 msgr2=0x7f148019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14801023a0 0x7f148019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 -- 172.21.15.112:0/2476176109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1480108410 msgr2=0x7f14801a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1480108410 0x7f14801a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.226+0000 7f148492d700 1 -- 172.21.15.112:0/2476176109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14801a5e70 con 0x7f1480102d60 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.227+0000 7f148492d700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480102d60 0x7f148019adf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f147000e9c0 tx=0x7f147000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.227+0000 7f14767fc700 1 -- 172.21.15.112:0/2476176109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000cbd0 con 0x7f1480102d60 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.227+0000 7f14767fc700 1 -- 172.21.15.112:0/2476176109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f147000cd70 con 0x7f1480102d60 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.227+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14801a60a0 con 0x7f1480102d60 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.227+0000 7f14767fc700 1 -- 172.21.15.112:0/2476176109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470005540 con 0x7f1480102d60 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.227+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14801a65e0 con 0x7f1480102d60 2024-05-10T10:44:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.229+0000 7f14767fc700 1 -- 172.21.15.112:0/2476176109 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1470021440 con 0x7f1480102d60 2024-05-10T10:44:59.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.230+0000 7f14767fc700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f146c0b2900 0x7f146c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.230+0000 7f14767fc700 1 -- 172.21.15.112:0/2476176109 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14700e2810 con 0x7f1480102d60 2024-05-10T10:44:59.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.230+0000 7f148512e700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f146c0b2900 0x7f146c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f148512e700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f146c0b2900 0x7f146c0b4dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f147c00b2e0 tx=0x7f147c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:59.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 Processor -- start 2024-05-10T10:44:59.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 -- start start 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b9241259c0 con 0x55b9241ddf30 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b92408ae20 con 0x55b9241dd610 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f1491c25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b924121980 con 0x55b9241d8230 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.231+0000 7f148492d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241d7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47162/0 (socket says 172.21.15.112:47162) 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 -- 172.21.15.112:0/1223834933 learned_addr learned my addr 172.21.15.112:0/1223834933 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148512e700 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148592f700 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 -- 172.21.15.112:0/1223834933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 msgr2=0x55b9241d8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241d8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 -- 172.21.15.112:0/1223834933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 msgr2=0x55b9241dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 -- 172.21.15.112:0/1223834933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b924072950 con 0x55b9241ddf30 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148592f700 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241d8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148512e700 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f148492d700 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241d7ca0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f14700a6cf0 tx=0x7f14700a9480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1ae9c99b6a4fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f146b7fe700 1 -- 172.21.15.112:0/1223834933 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700b2230 con 0x55b9241ddf30 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f146b7fe700 1 -- 172.21.15.112:0/1223834933 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14700a9590 con 0x55b9241ddf30 2024-05-10T10:44:59.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.232+0000 7f146b7fe700 1 -- 172.21.15.112:0/1223834933 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700b2450 con 0x55b9241ddf30 2024-05-10T10:44:59.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 -- 172.21.15.112:0/1223834933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 msgr2=0x55b9241d7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241d7ca0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f14700a6cf0 tx=0x7f14700a9480 comp rx=0 tx=0).stop 2024-05-10T10:44:59.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 -- 172.21.15.112:0/1223834933 shutdown_connections 2024-05-10T10:44:59.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241d8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1223834933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241d7ca0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 -- 172.21.15.112:0/1223834933 >> 172.21.15.112:0/1223834933 conn(0x55b9240ffbf0 msgr2=0x55b9241ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 -- 172.21.15.112:0/1223834933 shutdown_connections 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.233+0000 7f1491c25b80 1 -- 172.21.15.112:0/1223834933 wait complete. 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 -- start start 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 Processor -- start 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b9241e4960 con 0x55b9241ddf30 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b9240def50 con 0x55b9241dd610 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b9241e4ae0 con 0x55b9241d8230 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f1491c25b80 10 client.0 _ll_register_callbacks cb 0x55b9240b9630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f148592f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f148592f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241e14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47170/0 (socket says 172.21.15.112:47170) 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f148592f700 1 -- 172.21.15.112:0/1184000851 learned_addr learned my addr 172.21.15.112:0/1184000851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f148512e700 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.234+0000 7f148492d700 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f148592f700 1 -- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 msgr2=0x55b9241e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f148592f700 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f148592f700 1 -- 172.21.15.112:0/1184000851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 msgr2=0x55b9241e0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f148592f700 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f148592f700 1 -- 172.21.15.112:0/1184000851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9240dc7e0 con 0x55b9241ddf30 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f148592f700 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241e14a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1478009550 tx=0x7f1478007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeb7aa2f9d5ae2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f1468ff9700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f1468ff9700 1 -- 172.21.15.112:0/1184000851 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147801a070 con 0x55b9241ddf30 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14700a9590 con 0x55b9241ddf30 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f1468ff9700 1 -- 172.21.15.112:0/1184000851 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1478015860 con 0x55b9241ddf30 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f1468ff9700 1 -- 172.21.15.112:0/1184000851 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1478026050 con 0x55b9241ddf30 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f1491c25b80 10 client.16596 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.235+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b9241ef2c0 con 0x55b9241ddf30 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1468ff9700 1 -- 172.21.15.112:0/1184000851 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14700a9590 con 0x55b9241ddf30 2024-05-10T10:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1468ff9700 1 -- 172.21.15.112:0/1184000851 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f1478016c20 con 0x55b9241ddf30 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1491c25b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1491c25b80 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1491c25b80 20 client.16596 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b9241f1850 0x55b9241f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1491c25b80 4 client.16596 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9241333a0 con 0x55b9241f1850 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f148512e700 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b9241f1850 0x55b9241f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f1468ff9700 10 client.16596 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.236+0000 7f148512e700 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b9241f1850 0x55b9241f3c40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fa50cd7db52c1b server_cookie=22ec00dedcaac2ef in_seq=0 out_seq=0 2024-05-10T10:44:59.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.237+0000 7f1468ff9700 1 -- 172.21.15.112:0/1184000851 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b9241333a0 con 0x55b9241f1850 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.237+0000 7f1468ff9700 10 client.16596 handle_command_reply: tid=0 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.237+0000 7f1491c25b80 1 client.16596 shutdown 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 msgr2=0x55b9241e14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241e14a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1478009550 tx=0x7f1478007ba0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 shutdown_connections 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b9241f1850 0x55b9241f3c40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241e14a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 --2- 172.21.15.112:0/1184000851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 >> 172.21.15.112:0/1184000851 conn(0x55b9240ffbf0 msgr2=0x55b9241c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1468ff9700 0 client.16596 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 shutdown_connections 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 1 -- 172.21.15.112:0/1184000851 wait complete. 2024-05-10T10:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.238+0000 7f1491c25b80 20 client.16596 trim_cache size 0 max 16384 2024-05-10T10:44:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 Processor -- start 2024-05-10T10:44:59.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 -- start start 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241dff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14801ad000 con 0x55b9241ddf30 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14801aecc0 con 0x55b9241dd610 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f1491c25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14801ad6d0 con 0x55b9241d8230 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f148512e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f148512e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241dff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49540/0 (socket says 172.21.15.112:49540) 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f148512e700 1 -- 172.21.15.112:0/3695866429 learned_addr learned my addr 172.21.15.112:0/3695866429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.262+0000 7f148492d700 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f148592f700 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f148512e700 1 -- 172.21.15.112:0/3695866429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 msgr2=0x55b9241eda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f148512e700 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241eda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f148512e700 1 -- 172.21.15.112:0/3695866429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 msgr2=0x55b9241c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f148512e700 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f148512e700 1 -- 172.21.15.112:0/3695866429 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9241cad00 con 0x55b9241d8230 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f148512e700 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241dff00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f147c000c00 tx=0x7f147c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4716a939252664fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f146affd700 1 -- 172.21.15.112:0/3695866429 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00f8c0 con 0x55b9241d8230 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f146affd700 1 -- 172.21.15.112:0/3695866429 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f147c0351d0 con 0x55b9241d8230 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.263+0000 7f146affd700 1 -- 172.21.15.112:0/3695866429 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c023670 con 0x55b9241d8230 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 -- 172.21.15.112:0/3695866429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 msgr2=0x55b9241dff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241dff00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f147c000c00 tx=0x7f147c01e000 comp rx=0 tx=0).stop 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 -- 172.21.15.112:0/3695866429 shutdown_connections 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241ddf30 0x55b9241c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241d8230 0x55b9241dff00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 --2- 172.21.15.112:0/3695866429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 -- 172.21.15.112:0/3695866429 >> 172.21.15.112:0/3695866429 conn(0x55b9240ffbf0 msgr2=0x55b9241d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 -- 172.21.15.112:0/3695866429 shutdown_connections 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.264+0000 7f1491c25b80 1 -- 172.21.15.112:0/3695866429 wait complete. 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.265+0000 7f1491c25b80 1 -- start start 2024-05-10T10:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.265+0000 7f1491c25b80 1 Processor -- start 2024-05-10T10:44:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241d8230 0x55b9241e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1491c25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241ddf30 0x55b9241e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1491c25b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14801ae530 con 0x55b9241d8230 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1491c25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14801ade60 con 0x55b9241dd610 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1491c25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14801aafa0 con 0x55b9241ddf30 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1491c25b80 10 client.0 _ll_register_callbacks cb 0x55b9240b9630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:42230/0 (socket says 172.21.15.112:42230) 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 -- 172.21.15.112:0/2344279292 learned_addr learned my addr 172.21.15.112:0/2344279292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148592f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241ddf30 0x55b9241e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148512e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241d8230 0x55b9241e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 -- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241ddf30 msgr2=0x55b9241e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241ddf30 0x55b9241e77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 -- 172.21.15.112:0/2344279292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241d8230 msgr2=0x55b9241e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241d8230 0x55b9241e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 -- 172.21.15.112:0/2344279292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b92425c100 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148592f700 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241ddf30 0x55b9241e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f148492d700 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e72b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f14700a5b00 tx=0x7f14700b2680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402084d62d61303c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.266+0000 7f1468ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1468ff9700 1 -- 172.21.15.112:0/2344279292 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700b2230 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1468ff9700 1 -- 172.21.15.112:0/2344279292 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14700cc320 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c0351d0 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1468ff9700 1 -- 172.21.15.112:0/2344279292 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f14700a9a80 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 10 client.25976 fetch_fsmap learned FSMap version 36 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b92425d440 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1468ff9700 1 -- 172.21.15.112:0/2344279292 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f147c0351d0 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1468ff9700 1 -- 172.21.15.112:0/2344279292 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f14700a53f0 con 0x55b9241dd610 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 10 client.25976 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:44:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 20 client.25976 populate_metadata read hostname 'smithi112' 2024-05-10T10:44:59.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b92425d7b0 0x55b92425fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:44:59.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 4 client.25976 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:44:59.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b92412d440 con 0x55b92425d7b0 2024-05-10T10:44:59.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.267+0000 7f148512e700 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b92425d7b0 0x55b92425fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:44:59.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.268+0000 7f1468ff9700 10 client.25976 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:59.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.268+0000 7f148512e700 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b92425d7b0 0x55b92425fba0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38786953f0b52b81 server_cookie=74ca84a9646ed58a in_seq=0 out_seq=0 2024-05-10T10:44:59.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1468ff9700 1 -- 172.21.15.112:0/2344279292 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b92412d440 con 0x55b92425d7b0 2024-05-10T10:44:59.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1468ff9700 10 client.25976 handle_command_reply: tid=0 2024-05-10T10:44:59.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 client.25976 shutdown 2024-05-10T10:44:59.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 msgr2=0x55b9241e72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e72b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f14700a5b00 tx=0x7f14700b2680 comp rx=0 tx=0).stop 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 shutdown_connections 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b92425d7b0 0x55b92425fba0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9241ddf30 0x55b9241e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9241d8230 0x55b9241e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 --2- 172.21.15.112:0/2344279292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9241dd610 0x55b9241e72b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1468ff9700 0 client.25976 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 >> 172.21.15.112:0/2344279292 conn(0x55b9240ffbf0 msgr2=0x55b9241d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 shutdown_connections 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 1 -- 172.21.15.112:0/2344279292 wait complete. 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.269+0000 7f1491c25b80 20 client.25976 trim_cache size 0 max 16384 2024-05-10T10:44:59.274 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:44:59.275 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:44:59.275 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "7b778590-cdbc-4c14-9133-67628c929081", 2024-05-10T10:44:59.275 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:44:59.275 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f146c0b2900 msgr2=0x7f146c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f146c0b2900 0x7f146c0b4dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f147c00b2e0 tx=0x7f147c01a040 comp rx=0 tx=0).stop 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480102d60 msgr2=0x7f148019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480102d60 0x7f148019adf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f147000e9c0 tx=0x7f147000ecd0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 shutdown_connections 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1480108410 0x7f14801a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f146c0b2900 0x7f146c0b4dc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14801023a0 0x7f148019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 --2- 172.21.15.112:0/2476176109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1480102d60 0x7f148019adf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 >> 172.21.15.112:0/2476176109 conn(0x7f1480079350 msgr2=0x7f14801011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 shutdown_connections 2024-05-10T10:44:59.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:44:59.270+0000 7f1487392700 1 -- 172.21.15.112:0/2476176109 wait complete. 2024-05-10T10:44:59.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:44:59 smithi148 ceph-mon[33568]: pgmap v572: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.6 MiB/s wr, 559 op/s 2024-05-10T10:44:59.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:44:59 smithi195 ceph-mon[33773]: pgmap v572: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.6 MiB/s wr, 559 op/s 2024-05-10T10:44:59.566 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:44:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: 2024-05-10T10:44:59.295+0000 7ff3584cc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002bb [...2,head] /volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp/ auth{0=1,3=1,4=1} v5735 ap=2 f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1) n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2) (inest sync->mix g=0,3,4) caps={15294=pAsLsXsFs/-@1} | lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x5572a8cae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000002bb:[<0x10000000007/ltp v436>,<0x10000000006/xfstests-dev v72>,<0x10000000005/tmp v10>,<0x10000000003/client.0 v12>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v28>,<0x10000000001/sv_0 v52>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x100000002bb:[<0x10000000007/ltp v5735>,<0x10000000006/xfstests-dev v12369>,<0x10000000005/tmp v6802>,<0x10000000003/client.0 v388>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v396>,<0x10000000001/sv_0 v407>,<0x10000000000/qa v396>,<0x1/volumes v382>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T10:42:55.038905+0000 15=14+1)","ondisk_value.rstat":"n(v0 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","memory_value.dirstat":"f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1)","memory_value.rstat":"n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T10:45:00.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:45:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:00.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:45:00.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:45:00.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:45:00.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:45:00.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:00.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:45:00.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:01.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:01.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:01 smithi148 ceph-mon[33568]: pgmap v573: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.2 MiB/s wr, 483 op/s 2024-05-10T10:45:01.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:01.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:01.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:01 smithi195 ceph-mon[33773]: pgmap v573: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.2 MiB/s wr, 483 op/s 2024-05-10T10:45:01.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:01.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:01 smithi112 ceph-mon[30100]: pgmap v573: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.2 MiB/s wr, 483 op/s 2024-05-10T10:45:01.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:45:02.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.387+0000 7fbd40614700 1 -- 172.21.15.112:0/1794910678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd3806b180 msgr2=0x7fbd3806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.387+0000 7fbd40614700 1 --2- 172.21.15.112:0/1794910678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd3806b180 0x7fbd3806b600 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009990 tx=0x7fbd28009ca0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.388+0000 7fbd40614700 1 -- 172.21.15.112:0/1794910678 shutdown_connections 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.388+0000 7fbd40614700 1 --2- 172.21.15.112:0/1794910678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 0x7fbd38110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.388+0000 7fbd40614700 1 --2- 172.21.15.112:0/1794910678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd38109510 0x7fbd381098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.388+0000 7fbd40614700 1 --2- 172.21.15.112:0/1794910678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd3806b180 0x7fbd3806b600 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.388+0000 7fbd40614700 1 -- 172.21.15.112:0/1794910678 >> 172.21.15.112:0/1794910678 conn(0x7fbd38079350 msgr2=0x7fbd38079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.388+0000 7fbd40614700 1 -- 172.21.15.112:0/1794910678 shutdown_connections 2024-05-10T10:45:02.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.388+0000 7fbd40614700 1 -- 172.21.15.112:0/1794910678 wait complete. 2024-05-10T10:45:02.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.389+0000 7fbd40614700 1 Processor -- start 2024-05-10T10:45:02.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.389+0000 7fbd40614700 1 -- start start 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.389+0000 7fbd40614700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3806b180 0x7fbd38074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.389+0000 7fbd40614700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 0x7fbd38102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.389+0000 7fbd40614700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd38109510 0x7fbd381029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3e3b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3806b180 0x7fbd38074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3e3b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3806b180 0x7fbd38074fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49582/0 (socket says 172.21.15.112:49582) 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 0x7fbd38102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3ebb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd38109510 0x7fbd381029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 0x7fbd38102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:42258/0 (socket says 172.21.15.112:42258) 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3e3b0700 1 -- 172.21.15.112:0/1921309756 learned_addr learned my addr 172.21.15.112:0/1921309756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:02.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38113520 con 0x7fbd38109510 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381133a0 con 0x7fbd3806bb40 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381136a0 con 0x7fbd3806b180 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 -- 172.21.15.112:0/1921309756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3806b180 msgr2=0x7fbd38074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3806b180 0x7fbd38074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 -- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd38109510 msgr2=0x7fbd381029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd38109510 0x7fbd381029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 -- 172.21.15.112:0/1921309756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd381030e0 con 0x7fbd3806bb40 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd38109510 0x7fbd381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.390+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 0x7fbd38102470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006010 tx=0x7fbd280057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.391+0000 7fbd2f7fe700 1 -- 172.21.15.112:0/1921309756 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28026070 con 0x7fbd3806bb40 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.391+0000 7fbd2f7fe700 1 -- 172.21.15.112:0/1921309756 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd28021410 con 0x7fbd3806bb40 2024-05-10T10:45:02.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.391+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3806db80 con 0x7fbd3806bb40 2024-05-10T10:45:02.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.391+0000 7fbd2f7fe700 1 -- 172.21.15.112:0/1921309756 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28017650 con 0x7fbd3806bb40 2024-05-10T10:45:02.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.391+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3806e010 con 0x7fbd3806bb40 2024-05-10T10:45:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.393+0000 7fbd2f7fe700 1 -- 172.21.15.112:0/1921309756 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbd28029070 con 0x7fbd3806bb40 2024-05-10T10:45:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.394+0000 7fbd2f7fe700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbd240b2880 0x7fbd240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.394+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbd240b2880 0x7fbd240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.394+0000 7fbd2f7fe700 1 -- 172.21.15.112:0/1921309756 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbd280e3570 con 0x7fbd3806bb40 2024-05-10T10:45:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbd240b2880 0x7fbd240b4d40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009ca0 tx=0x7fbd34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:02.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 Processor -- start 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 -- start start 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611418444d0 0x5611418448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x561141849100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561141796860 con 0x5611418444d0 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5611417357c0 con 0x561141844df0 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561141792820 con 0x561141848c80 2024-05-10T10:45:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd3e3b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611418444d0 0x5611418448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd3dbaf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd3e3b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611418444d0 0x5611418448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47212/0 (socket says 172.21.15.112:47212) 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.395+0000 7fbd3e3b0700 1 -- 172.21.15.112:0/749913875 learned_addr learned my addr 172.21.15.112:0/749913875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x561141849100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3dbaf700 1 -- 172.21.15.112:0/749913875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 msgr2=0x561141849100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x561141849100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3dbaf700 1 -- 172.21.15.112:0/749913875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611418444d0 msgr2=0x5611418448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611418444d0 0x5611418448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3dbaf700 1 -- 172.21.15.112:0/749913875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611416e38d0 con 0x561141844df0 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611418444d0 0x5611418448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418486f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd280aa530 tx=0x7fbd280aa020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b4c6be53f65b70 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:02.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd2cff9700 1 -- 172.21.15.112:0/749913875 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd280cd0e0 con 0x561141844df0 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd2cff9700 1 -- 172.21.15.112:0/749913875 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd280aa340 con 0x561141844df0 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.396+0000 7fbd2cff9700 1 -- 172.21.15.112:0/749913875 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd280219c0 con 0x561141844df0 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/749913875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 msgr2=0x5611418486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418486f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd280aa530 tx=0x7fbd280aa020 comp rx=0 tx=0).stop 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/749913875 shutdown_connections 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x561141849100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611418444d0 0x5611418448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/749913875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418486f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/749913875 >> 172.21.15.112:0/749913875 conn(0x561141770100 msgr2=0x56114183ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/749913875 shutdown_connections 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/749913875 wait complete. 2024-05-10T10:45:02.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 -- start start 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.397+0000 7fbd4aea7b80 1 Processor -- start 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x5611418500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141844df0 0x56114184d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x56114184d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561141850c50 con 0x561141844df0 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5611416fcc30 con 0x5611418444d0 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561141850dd0 con 0x561141848c80 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd4aea7b80 10 client.0 _ll_register_callbacks cb 0x56114172e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x56114184d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x56114184d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49600/0 (socket says 172.21.15.112:49600) 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1774341901 learned_addr learned my addr 172.21.15.112:0/1774341901 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141844df0 0x56114184d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x5611418500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1774341901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 msgr2=0x5611418500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x5611418500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1774341901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141844df0 msgr2=0x56114184d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141844df0 0x56114184d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1774341901 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561141732690 con 0x561141848c80 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.398+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x5611418500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x56114184d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd30012e70 tx=0x7fbd30004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6a392fbbf825a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1774341901 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30007700 con 0x561141848c80 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1774341901 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbd300078a0 con 0x561141848c80 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd280aa340 con 0x561141848c80 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1774341901 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd30007aa0 con 0x561141848c80 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd4aea7b80 10 client.26014 fetch_fsmap learned FSMap version 36 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.399+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56114185b460 con 0x561141848c80 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1774341901 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd280aa340 con 0x561141848c80 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1774341901 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fbd3001fc10 con 0x561141848c80 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd4aea7b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd4aea7b80 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd4aea7b80 20 client.26014 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56114185daf0 0x56114185ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd4aea7b80 4 client.26014 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.400+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561141810650 con 0x56114185daf0 2024-05-10T10:45:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.401+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56114185daf0 0x56114185ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.402+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56114185daf0 0x56114185ff00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dae299b93a4f652 server_cookie=1808d4537916a in_seq=0 out_seq=0 2024-05-10T10:45:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.402+0000 7fbd21ffb700 10 client.26014 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1774341901 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561141810650 con 0x56114185daf0 2024-05-10T10:45:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd21ffb700 10 client.26014 handle_command_reply: tid=0 2024-05-10T10:45:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 client.26014 shutdown 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 msgr2=0x56114184d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x56114184d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd30012e70 tx=0x7fbd30004320 comp rx=0 tx=0).stop 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 shutdown_connections 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56114185daf0 0x56114185ff00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141848c80 0x56114184d9c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x5611418500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1774341901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141844df0 0x56114184d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd21ffb700 0 client.26014 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 >> 172.21.15.112:0/1774341901 conn(0x561141770100 msgr2=0x561141839720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 shutdown_connections 2024-05-10T10:45:02.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1774341901 wait complete. 2024-05-10T10:45:02.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.403+0000 7fbd4aea7b80 20 client.26014 trim_cache size 0 max 16384 2024-05-10T10:45:02.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.426+0000 7fbd4aea7b80 1 Processor -- start 2024-05-10T10:45:02.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.426+0000 7fbd4aea7b80 1 -- start start 2024-05-10T10:45:02.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.426+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x561141859c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.426+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141844df0 0x5611418395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.426+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x5611418465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381b5450 con 0x561141848c80 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381b7110 con 0x5611418444d0 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381b5b20 con 0x561141844df0 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd3ebb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x5611418465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd3ebb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x5611418465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47228/0 (socket says 172.21.15.112:47228) 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1025625758 learned_addr learned my addr 172.21.15.112:0/1025625758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd3e3b0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x561141859c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.427+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141844df0 0x5611418395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1025625758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141844df0 msgr2=0x5611418395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141844df0 0x5611418395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1025625758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 msgr2=0x561141859c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x561141859c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd3ebb1700 1 -- 172.21.15.112:0/1025625758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561141850450 con 0x561141848c80 2024-05-10T10:45:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x5611418465f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbd30012e40 tx=0x7fbd300097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e4079fafd49ee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd23fff700 1 -- 172.21.15.112:0/1025625758 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3001e6a0 con 0x561141848c80 2024-05-10T10:45:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd23fff700 1 -- 172.21.15.112:0/1025625758 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd30036070 con 0x561141848c80 2024-05-10T10:45:02.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.428+0000 7fbd23fff700 1 -- 172.21.15.112:0/1025625758 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30005ec0 con 0x561141848c80 2024-05-10T10:45:02.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.429+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1025625758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 msgr2=0x5611418465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.429+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x5611418465f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbd30012e40 tx=0x7fbd300097e0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.429+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1025625758 shutdown_connections 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x5611418465f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611418444d0 0x561141859c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1025625758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561141844df0 0x5611418395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1025625758 >> 172.21.15.112:0/1025625758 conn(0x561141770100 msgr2=0x561141838e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1025625758 shutdown_connections 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1025625758 wait complete. 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 -- start start 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 Processor -- start 2024-05-10T10:45:02.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611418444d0 0x561141851b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x561141852600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381b6980 con 0x561141848c80 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381b62b0 con 0x561141844df0 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.430+0000 7fbd4aea7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381b33f0 con 0x5611418444d0 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd4aea7b80 10 client.0 _ll_register_callbacks cb 0x56114172e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611418444d0 0x561141851b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611418444d0 0x561141851b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:49608/0 (socket says 172.21.15.112:49608) 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 -- 172.21.15.112:0/1078565062 learned_addr learned my addr 172.21.15.112:0/1078565062 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x561141852600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 -- 172.21.15.112:0/1078565062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 msgr2=0x5611418520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 -- 172.21.15.112:0/1078565062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 msgr2=0x561141852600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x561141852600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 -- 172.21.15.112:0/1078565062 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611418c7400 con 0x5611418444d0 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3ebb1700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x561141852600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd3e3b0700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611418444d0 0x561141851b80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009f40 tx=0x7fbd340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c11ecd2715e598db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.431+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1078565062 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3401d980 con 0x5611418444d0 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd30036070 con 0x5611418444d0 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1078565062 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fbd3401db20 con 0x5611418444d0 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1078565062 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd3403c050 con 0x5611418444d0 2024-05-10T10:45:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 10 client.26020 fetch_fsmap learned FSMap version 36 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5611418c8750 con 0x5611418444d0 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1078565062 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd30036070 con 0x5611418444d0 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1078565062 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fbd34034420 con 0x5611418444d0 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 10 client.26020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 20 client.26020 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5611418c8f20 0x5611418cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 4 client.26020 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5611417a4240 con 0x5611418c8f20 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.432+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5611418c8f20 0x5611418cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd21ffb700 10 client.26020 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd3dbaf700 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5611418c8f20 0x5611418cb340 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4d757c118579d5 server_cookie=f8ccac7ba7712130 in_seq=0 out_seq=0 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd21ffb700 1 -- 172.21.15.112:0/1078565062 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5611417a4240 con 0x5611418c8f20 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd21ffb700 10 client.26020 handle_command_reply: tid=0 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 client.26020 shutdown 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611418444d0 msgr2=0x561141851b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611418444d0 0x561141851b80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009f40 tx=0x7fbd340047c0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 shutdown_connections 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5611418c8f20 0x5611418cb340 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561141848c80 0x561141852600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5611418444d0 0x561141851b80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 --2- 172.21.15.112:0/1078565062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561141844df0 0x5611418520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 >> 172.21.15.112:0/1078565062 conn(0x561141770100 msgr2=0x561141839720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd21ffb700 0 client.26020 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 shutdown_connections 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 1 -- 172.21.15.112:0/1078565062 wait complete. 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.433+0000 7fbd4aea7b80 20 client.26020 trim_cache size 0 max 16384 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: "7b778590-cdbc-4c14-9133-67628c929081": { 2024-05-10T10:45:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-10T10:45:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "7b778590-cdbc-4c14-9133-67628c929081", 2024-05-10T10:45:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-10T10:45:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:45:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:45:02.441 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:45:02.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbd240b2880 msgr2=0x7fbd240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbd240b2880 0x7fbd240b4d40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009ca0 tx=0x7fbd34009260 comp rx=0 tx=0).stop 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 msgr2=0x7fbd38102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 0x7fbd38102470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006010 tx=0x7fbd280057c0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 shutdown_connections 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd38109510 0x7fbd381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fbd240b2880 0x7fbd240b4d40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3806b180 0x7fbd38074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 --2- 172.21.15.112:0/1921309756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3806bb40 0x7fbd38102470 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 >> 172.21.15.112:0/1921309756 conn(0x7fbd38079350 msgr2=0x7fbd38100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 shutdown_connections 2024-05-10T10:45:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:02.434+0000 7fbd40614700 1 -- 172.21.15.112:0/1921309756 wait complete. 2024-05-10T10:45:02.512 INFO:tasks.workunit.client.0.smithi112.stdout:20000 mapread 0x1c4208 thru 0x1c56d4 (0x14cd bytes) 2024-05-10T10:45:03.334 INFO:tasks.cephfs.filesystem:scrub status for tag:7b778590-cdbc-4c14-9133-67628c929081 - {'path': '/', 'tag': '7b778590-cdbc-4c14-9133-67628c929081', 'options': 'recursive,force'} 2024-05-10T10:45:03.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:03 smithi148 ceph-mon[33568]: pgmap v574: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 934 op/s 2024-05-10T10:45:03.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:03 smithi195 ceph-mon[33773]: pgmap v574: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 934 op/s 2024-05-10T10:45:03.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:03 smithi112 ceph-mon[30100]: pgmap v574: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 934 op/s 2024-05-10T10:45:05.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:05 smithi148 ceph-mon[33568]: pgmap v575: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.0 MiB/s wr, 790 op/s 2024-05-10T10:45:05.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:05 smithi195 ceph-mon[33773]: pgmap v575: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.0 MiB/s wr, 790 op/s 2024-05-10T10:45:05.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:05 smithi112 ceph-mon[30100]: pgmap v575: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.0 MiB/s wr, 790 op/s 2024-05-10T10:45:07.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:07 smithi148 ceph-mon[33568]: pgmap v576: 129 pgs: 129 active+clean; 41 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 973 op/s 2024-05-10T10:45:07.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:07 smithi195 ceph-mon[33773]: pgmap v576: 129 pgs: 129 active+clean; 41 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 973 op/s 2024-05-10T10:45:07.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:07 smithi112 ceph-mon[30100]: pgmap v576: 129 pgs: 129 active+clean; 41 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 973 op/s 2024-05-10T10:45:09.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:09 smithi148 ceph-mon[33568]: pgmap v577: 129 pgs: 129 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.1 MiB/s wr, 883 op/s 2024-05-10T10:45:09.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:09 smithi195 ceph-mon[33773]: pgmap v577: 129 pgs: 129 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.1 MiB/s wr, 883 op/s 2024-05-10T10:45:09.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:09 smithi112 ceph-mon[30100]: pgmap v577: 129 pgs: 129 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.1 MiB/s wr, 883 op/s 2024-05-10T10:45:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 -- 172.21.15.112:0/3550956207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 msgr2=0x7f430c1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 --2- 172.21.15.112:0/3550956207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c1041f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009900 tx=0x7f42fc02a400 comp rx=0 tx=0).stop 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 -- 172.21.15.112:0/3550956207 shutdown_connections 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 --2- 172.21.15.112:0/3550956207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c104730 0x7f430c10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 --2- 172.21.15.112:0/3550956207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c1041f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 --2- 172.21.15.112:0/3550956207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f430c100d00 0x7f430c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 -- 172.21.15.112:0/3550956207 >> 172.21.15.112:0/3550956207 conn(0x7f430c0fc630 msgr2=0x7f430c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 -- 172.21.15.112:0/3550956207 shutdown_connections 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 -- 172.21.15.112:0/3550956207 wait complete. 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 Processor -- start 2024-05-10T10:45:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.126+0000 7f43140a7640 1 -- start start 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f43140a7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c100d00 0x7f430c1036e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f43140a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f43140a7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f430c104730 0x7f430c102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f43140a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f430c10e440 con 0x7f430c103d70 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f43140a7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f430c10e2c0 con 0x7f430c100d00 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f43140a7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f430c10e5c0 con 0x7f430c104730 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311642640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311642640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48466/0 (socket says 172.21.15.112:48466) 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311642640 1 -- 172.21.15.112:0/1243456521 learned_addr learned my addr 172.21.15.112:0/1243456521 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311e43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c100d00 0x7f430c1036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4312644640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f430c104730 0x7f430c102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311e43640 1 -- 172.21.15.112:0/1243456521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f430c104730 msgr2=0x7f430c102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311e43640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f430c104730 0x7f430c102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311e43640 1 -- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 msgr2=0x7f430c101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311e43640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311e43640 1 -- 172.21.15.112:0/1243456521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c102af0 con 0x7f430c100d00 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311642640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4311e43640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c100d00 0x7f430c1036e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f430c06af40 tx=0x7f430800dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.127+0000 7f4302ffd640 1 -- 172.21.15.112:0/1243456521 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308017070 con 0x7f430c100d00 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.128+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c1a2c30 con 0x7f430c100d00 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.128+0000 7f4302ffd640 1 -- 172.21.15.112:0/1243456521 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4308010460 con 0x7f430c100d00 2024-05-10T10:45:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.128+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c1a3180 con 0x7f430c100d00 2024-05-10T10:45:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.128+0000 7f4302ffd640 1 -- 172.21.15.112:0/1243456521 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308021060 con 0x7f430c100d00 2024-05-10T10:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.130+0000 7f4302ffd640 1 -- 172.21.15.112:0/1243456521 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f430801a070 con 0x7f430c100d00 2024-05-10T10:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.130+0000 7f4316dc6740 1 Processor -- start 2024-05-10T10:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.130+0000 7f4316dc6740 1 -- start start 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32d81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4316dc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdd323c5d0 con 0x55cdd32dd910 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4316dc6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cdd31f5890 con 0x55cdd32d87a0 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4316dc6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cdd3238590 con 0x55cdd32d7db0 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32dfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48476/0 (socket says 172.21.15.112:48476) 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 -- 172.21.15.112:0/616729599 learned_addr learned my addr 172.21.15.112:0/616729599 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311e43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4312644640 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32d81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 -- 172.21.15.112:0/616729599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 msgr2=0x55cdd32d81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32d81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 -- 172.21.15.112:0/616729599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 msgr2=0x55cdd32dd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32dd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 -- 172.21.15.112:0/616729599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd2f4e210 con 0x55cdd32dd910 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4312644640 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32d81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4302ffd640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f42f80b28a0 0x7f42f80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4311642640 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32dfda0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009a20 tx=0x7f42fc031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf7e616f068b32d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4312644640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f42f80b28a0 0x7f42f80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f4302ffd640 1 -- 172.21.15.112:0/1243456521 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f43080e1fb0 con 0x7f430c100d00 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f42f7fff640 1 -- 172.21.15.112:0/616729599 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc035030 con 0x55cdd32dd910 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f42f7fff640 1 -- 172.21.15.112:0/616729599 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42fc040070 con 0x55cdd32dd910 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.131+0000 7f42f7fff640 1 -- 172.21.15.112:0/616729599 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc003ca0 con 0x55cdd32dd910 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 -- 172.21.15.112:0/616729599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 msgr2=0x55cdd32dfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32dfda0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009a20 tx=0x7f42fc031940 comp rx=0 tx=0).stop 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4312644640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f42f80b28a0 0x7f42f80b4d60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55cdd32d9000 tx=0x7f4304009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 -- 172.21.15.112:0/616729599 shutdown_connections 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32dfda0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 --2- 172.21.15.112:0/616729599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32d81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 -- 172.21.15.112:0/616729599 >> 172.21.15.112:0/616729599 conn(0x55cdd32d3420 msgr2=0x55cdd32d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 -- 172.21.15.112:0/616729599 shutdown_connections 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.132+0000 7f4316dc6740 1 -- 172.21.15.112:0/616729599 wait complete. 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 -- start start 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 Processor -- start 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d7db0 0x55cdd32e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d87a0 0x55cdd32e24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdd32e0860 con 0x55cdd32d7db0 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cdd31f9370 con 0x55cdd32dd910 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cdd32e09e0 con 0x55cdd32d87a0 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4316dc6740 10 client.0 _ll_register_callbacks cb 0x55cdd2f4da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4311e43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d7db0 0x55cdd32e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4311e43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d7db0 0x55cdd32e1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48492/0 (socket says 172.21.15.112:48492) 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4311e43640 1 -- 172.21.15.112:0/3927973745 learned_addr learned my addr 172.21.15.112:0/3927973745 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4312644640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.133+0000 7f4311642640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d87a0 0x55cdd32e24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4311e43640 1 -- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d87a0 msgr2=0x55cdd32e24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4311e43640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d87a0 0x55cdd32e24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4311e43640 1 -- 172.21.15.112:0/3927973745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 msgr2=0x55cdd32e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4311e43640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4311e43640 1 -- 172.21.15.112:0/3927973745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd301a620 con 0x55cdd32d7db0 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4311642640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d87a0 0x55cdd32e24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4312644640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4311e43640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d7db0 0x55cdd32e1f90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f43080a5310 tx=0x7f4308010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa428b2c44767a8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f42f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f42f57fa640 1 -- 172.21.15.112:0/3927973745 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080a8da0 con 0x55cdd32d7db0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f42f57fa640 1 -- 172.21.15.112:0/3927973745 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f43080a5520 con 0x55cdd32d7db0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42fc040070 con 0x55cdd32d7db0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f42f57fa640 1 -- 172.21.15.112:0/3927973745 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f43080a9290 con 0x55cdd32d7db0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4316dc6740 10 client.16632 fetch_fsmap learned FSMap version 36 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cdd32ec560 con 0x55cdd32d7db0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.134+0000 7f42f57fa640 1 -- 172.21.15.112:0/3927973745 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42fc040070 con 0x55cdd32d7db0 2024-05-10T10:45:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f42f57fa640 1 -- 172.21.15.112:0/3927973745 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f43080dfb80 con 0x55cdd32d7db0 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4316dc6740 10 client.16632 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4316dc6740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4316dc6740 20 client.16632 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4316dc6740 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd32edce0 0x55cdd32f0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4316dc6740 4 client.16632 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdd324a1e0 con 0x55cdd32edce0 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4311642640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd32edce0 0x55cdd32f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f4311642640 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd32edce0 0x55cdd32f0180 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6063ea3bfdbfd2d0 server_cookie=5f641bf3cd405fa in_seq=0 out_seq=0 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.135+0000 7f42f57fa640 10 client.16632 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f42f57fa640 1 -- 172.21.15.112:0/3927973745 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cdd324a1e0 con 0x55cdd32edce0 2024-05-10T10:45:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f42f57fa640 10 client.16632 handle_command_reply: tid=0 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f4316dc6740 1 client.16632 shutdown 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d7db0 msgr2=0x55cdd32e1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f4316dc6740 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d7db0 0x55cdd32e1f90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f43080a5310 tx=0x7f4308010b40 comp rx=0 tx=0).stop 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 shutdown_connections 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f4316dc6740 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd32edce0 0x55cdd32f0180 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.136+0000 7f4316dc6740 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.137+0000 7f42f57fa640 0 client.16632 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.137+0000 7f4316dc6740 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d87a0 0x55cdd32e24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.137+0000 7f4316dc6740 1 --2- 172.21.15.112:0/3927973745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d7db0 0x55cdd32e1f90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.137+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 >> 172.21.15.112:0/3927973745 conn(0x55cdd32d3420 msgr2=0x55cdd32d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.137+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 shutdown_connections 2024-05-10T10:45:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.137+0000 7f4316dc6740 1 -- 172.21.15.112:0/3927973745 wait complete. 2024-05-10T10:45:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.137+0000 7f4316dc6740 20 client.16632 trim_cache size 0 max 16384 2024-05-10T10:45:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.154+0000 7f4316dc6740 1 Processor -- start 2024-05-10T10:45:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.154+0000 7f4316dc6740 1 -- start start 2024-05-10T10:45:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32d6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4316dc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1ad570 con 0x55cdd32dd910 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4316dc6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1af230 con 0x55cdd32d87a0 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4316dc6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1adc40 con 0x55cdd32d7db0 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32d6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48508/0 (socket says 172.21.15.112:48508) 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 -- 172.21.15.112:0/304803090 learned_addr learned my addr 172.21.15.112:0/304803090 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4311e43640 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4311642640 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32d6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 -- 172.21.15.112:0/304803090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 msgr2=0x55cdd32dcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32dcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 -- 172.21.15.112:0/304803090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 msgr2=0x55cdd32d6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32d6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 -- 172.21.15.112:0/304803090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd32d3250 con 0x55cdd32dd910 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f4312644640 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32d6870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4304003a70 tx=0x7f430401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284091f72d5c42fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f42f77fe640 1 -- 172.21.15.112:0/304803090 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304004040 con 0x55cdd32dd910 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f42f77fe640 1 -- 172.21.15.112:0/304803090 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4304004670 con 0x55cdd32dd910 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.155+0000 7f42f77fe640 1 -- 172.21.15.112:0/304803090 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304029cd0 con 0x55cdd32dd910 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 -- 172.21.15.112:0/304803090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 msgr2=0x55cdd32d6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32d6870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4304003a70 tx=0x7f430401a000 comp rx=0 tx=0).stop 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 -- 172.21.15.112:0/304803090 shutdown_connections 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32dd910 0x55cdd32d6870 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32d87a0 0x55cdd32d6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 --2- 172.21.15.112:0/304803090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32dcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 -- 172.21.15.112:0/304803090 >> 172.21.15.112:0/304803090 conn(0x55cdd32d3420 msgr2=0x55cdd32ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 -- 172.21.15.112:0/304803090 shutdown_connections 2024-05-10T10:45:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.156+0000 7f4316dc6740 1 -- 172.21.15.112:0/304803090 wait complete. 2024-05-10T10:45:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 -- start start 2024-05-10T10:45:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 Processor -- start 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d87a0 0x55cdd32e6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1aeaa0 con 0x55cdd32d87a0 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1ae3d0 con 0x55cdd32dd910 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1ab550 con 0x55cdd32d7db0 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4316dc6740 10 client.0 _ll_register_callbacks cb 0x7f4308010be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4311642640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d87a0 0x55cdd32e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4311e43640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4311642640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d87a0 0x55cdd32e6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48516/0 (socket says 172.21.15.112:48516) 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4311e43640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32e6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:34000/0 (socket says 172.21.15.112:34000) 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4311642640 1 -- 172.21.15.112:0/2678910912 learned_addr learned my addr 172.21.15.112:0/2678910912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.157+0000 7f4312644640 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4311e43640 1 -- 172.21.15.112:0/2678910912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 msgr2=0x55cdd32e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4311e43640 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4311e43640 1 -- 172.21.15.112:0/2678910912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d87a0 msgr2=0x55cdd32e6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4311e43640 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d87a0 0x55cdd32e6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4311e43640 1 -- 172.21.15.112:0/2678910912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd32e7c10 con 0x55cdd32d7db0 2024-05-10T10:45:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4312644640 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4311e43640 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32e6120 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4308010c90 tx=0x7f43080a5750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9306c414f679a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f42f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f42f57fa640 1 -- 172.21.15.112:0/2678910912 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080a8630 con 0x55cdd32d7db0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f42f57fa640 1 -- 172.21.15.112:0/2678910912 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f43080e50c0 con 0x55cdd32d7db0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4304004670 con 0x55cdd32d7db0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f42f57fa640 1 -- 172.21.15.112:0/2678910912 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f43080a8da0 con 0x55cdd32d7db0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4316dc6740 10 client.26038 fetch_fsmap learned FSMap version 36 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.158+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cdd3358640 con 0x55cdd32d7db0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f42f57fa640 1 -- 172.21.15.112:0/2678910912 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4304004670 con 0x55cdd32d7db0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f42f57fa640 1 -- 172.21.15.112:0/2678910912 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f43080a4c00 con 0x55cdd32d7db0 2024-05-10T10:45:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4316dc6740 10 client.26038 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4316dc6740 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4316dc6740 20 client.26038 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4316dc6740 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd3358db0 0x55cdd335b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4316dc6740 4 client.26038 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdd3244050 con 0x55cdd3358db0 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4311642640 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd3358db0 0x55cdd335b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f42f57fa640 10 client.26038 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.159+0000 7f4311642640 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd3358db0 0x55cdd335b1d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf2e1911ddbd5 server_cookie=565024bf75f9fcfd in_seq=0 out_seq=0 2024-05-10T10:45:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.179+0000 7f42f57fa640 1 -- 172.21.15.112:0/2678910912 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdd3244050 con 0x55cdd3358db0 2024-05-10T10:45:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.179+0000 7f42f57fa640 10 client.26038 handle_command_reply: tid=0 2024-05-10T10:45:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.179+0000 7f4316dc6740 1 client.26038 shutdown 2024-05-10T10:45:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 msgr2=0x55cdd32e6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32e6120 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4308010c90 tx=0x7f43080a5750 comp rx=0 tx=0).stop 2024-05-10T10:45:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 shutdown_connections 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55cdd3358db0 0x55cdd335b1d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdd32dd910 0x55cdd32e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd32d87a0 0x55cdd32e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 --2- 172.21.15.112:0/2678910912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cdd32d7db0 0x55cdd32e6120 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f42f57fa640 0 client.26038 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 >> 172.21.15.112:0/2678910912 conn(0x55cdd32d3420 msgr2=0x55cdd32dcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 shutdown_connections 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 1 -- 172.21.15.112:0/2678910912 wait complete. 2024-05-10T10:45:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.180+0000 7f4316dc6740 20 client.26038 trim_cache size 0 max 16384 2024-05-10T10:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f42f80b28a0 msgr2=0x7f42f80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f42f80b28a0 0x7f42f80b4d60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55cdd32d9000 tx=0x7f4304009500 comp rx=0 tx=0).stop 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c100d00 msgr2=0x7f430c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c100d00 0x7f430c1036e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f430c06af40 tx=0x7f430800dbb0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 shutdown_connections 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f42f80b28a0 0x7f42f80b4d60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f430c104730 0x7f430c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f430c103d70 0x7f430c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 --2- 172.21.15.112:0/1243456521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c100d00 0x7f430c1036e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 >> 172.21.15.112:0/1243456521 conn(0x7f430c0fc630 msgr2=0x7f430c107130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 shutdown_connections 2024-05-10T10:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:10.181+0000 7f43140a7640 1 -- 172.21.15.112:0/1243456521 wait complete. 2024-05-10T10:45:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:45:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:45:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:45:10.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:10 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:45:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:45:10.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:45:10.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:45:10.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:10 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:45:10.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:45:10.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:45:10.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:45:10.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:10 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:45:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:11 smithi148 ceph-mon[33568]: pgmap v578: 129 pgs: 129 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.7 MiB/s wr, 804 op/s 2024-05-10T10:45:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:11 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:45:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:11 smithi195 ceph-mon[33773]: pgmap v578: 129 pgs: 129 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.7 MiB/s wr, 804 op/s 2024-05-10T10:45:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:11 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:45:11.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:11 smithi112 ceph-mon[30100]: pgmap v578: 129 pgs: 129 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.7 MiB/s wr, 804 op/s 2024-05-10T10:45:11.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:11 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:45:13.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:13 smithi148 ceph-mon[33568]: pgmap v579: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 1.03k op/s 2024-05-10T10:45:13.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:13 smithi195 ceph-mon[33773]: pgmap v579: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 1.03k op/s 2024-05-10T10:45:13.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:13 smithi112 ceph-mon[30100]: pgmap v579: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 1.03k op/s 2024-05-10T10:45:15.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:15 smithi148 ceph-mon[33568]: pgmap v580: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 2.9 MiB/s wr, 577 op/s 2024-05-10T10:45:15.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:15 smithi195 ceph-mon[33773]: pgmap v580: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 2.9 MiB/s wr, 577 op/s 2024-05-10T10:45:15.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:15 smithi112 ceph-mon[30100]: pgmap v580: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 2.9 MiB/s wr, 577 op/s 2024-05-10T10:45:17.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:17 smithi148 ceph-mon[33568]: pgmap v581: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 638 op/s 2024-05-10T10:45:17.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:17 smithi195 ceph-mon[33773]: pgmap v581: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 638 op/s 2024-05-10T10:45:17.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:17 smithi112 ceph-mon[30100]: pgmap v581: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 638 op/s 2024-05-10T10:45:18.536 INFO:tasks.workunit.client.0.smithi112.stdout:30000 mapread 0x3fc9cb thru 0x4052a3 (0x88d9 bytes) 2024-05-10T10:45:19.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:19 smithi148 ceph-mon[33568]: pgmap v582: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-05-10T10:45:19.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:19 smithi195 ceph-mon[33773]: pgmap v582: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-05-10T10:45:19.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:19 smithi112 ceph-mon[30100]: pgmap v582: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-05-10T10:45:21.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:21 smithi148 ceph-mon[33568]: pgmap v583: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 455 op/s 2024-05-10T10:45:21.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:21 smithi195 ceph-mon[33773]: pgmap v583: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 455 op/s 2024-05-10T10:45:21.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:21 smithi112 ceph-mon[30100]: pgmap v583: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 455 op/s 2024-05-10T10:45:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.279+0000 7f545e56a640 1 -- 172.21.15.112:0/97784240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458104000 msgr2=0x7f5458104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.279+0000 7f545e56a640 1 --2- 172.21.15.112:0/97784240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458104000 0x7f5458104420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5448009900 tx=0x7f544802a410 comp rx=0 tx=0).stop 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.279+0000 7f545e56a640 1 -- 172.21.15.112:0/97784240 shutdown_connections 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.279+0000 7f545e56a640 1 --2- 172.21.15.112:0/97784240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5458101430 0x7f545810afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.279+0000 7f545e56a640 1 --2- 172.21.15.112:0/97784240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5458100a70 0x7f5458100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.279+0000 7f545e56a640 1 --2- 172.21.15.112:0/97784240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458104000 0x7f5458104420 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.279+0000 7f545e56a640 1 -- 172.21.15.112:0/97784240 >> 172.21.15.112:0/97784240 conn(0x7f54580fc610 msgr2=0x7f54580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 -- 172.21.15.112:0/97784240 shutdown_connections 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 -- 172.21.15.112:0/97784240 wait complete. 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 Processor -- start 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 -- start start 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458100a70 0x7f5458198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5458101430 0x7f5458199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5458104000 0x7f54581a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f545810e3c0 con 0x7f5458100a70 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f545810e240 con 0x7f5458101430 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.280+0000 7f545e56a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f545810e540 con 0x7f5458104000 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458100a70 0x7f5458198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458100a70 0x7f5458198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46746/0 (socket says 172.21.15.112:46746) 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 -- 172.21.15.112:0/3515784042 learned_addr learned my addr 172.21.15.112:0/3515784042 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f545cb07640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5458104000 0x7f54581a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f54577fe640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5458101430 0x7f5458199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 -- 172.21.15.112:0/3515784042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5458104000 msgr2=0x7f54581a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5458104000 0x7f54581a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 -- 172.21.15.112:0/3515784042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5458101430 msgr2=0x7f5458199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5458101430 0x7f5458199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 -- 172.21.15.112:0/3515784042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54581a0710 con 0x7f5458100a70 2024-05-10T10:45:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f5457fff640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458100a70 0x7f5458198af0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5448009230 tx=0x7f54480040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f54557fa640 1 -- 172.21.15.112:0/3515784042 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544803f030 con 0x7f5458100a70 2024-05-10T10:45:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581a09a0 con 0x7f5458100a70 2024-05-10T10:45:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f54557fa640 1 -- 172.21.15.112:0/3515784042 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54480044c0 con 0x7f5458100a70 2024-05-10T10:45:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f54557fa640 1 -- 172.21.15.112:0/3515784042 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448035070 con 0x7f5458100a70 2024-05-10T10:45:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.281+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581a0ee0 con 0x7f5458100a70 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 Processor -- start 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 -- start start 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f54557fa640 1 -- 172.21.15.112:0/3515784042 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5448042030 con 0x7f5458100a70 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d70b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d712da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55787d66f5d0 con 0x55787d70adb0 2024-05-10T10:45:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55787d628890 con 0x55787d70b7a0 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.284+0000 7f5461283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55787d66b590 con 0x55787d710910 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d70b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d70b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46752/0 (socket says 172.21.15.112:46752) 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 -- 172.21.15.112:0/3969865670 learned_addr learned my addr 172.21.15.112:0/3969865670 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f5457fff640 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 -- 172.21.15.112:0/3969865670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 msgr2=0x55787d712da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d712da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 -- 172.21.15.112:0/3969865670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 msgr2=0x55787d7102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 -- 172.21.15.112:0/3969865670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787d381210 con 0x55787d70adb0 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f5457fff640 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f545cb07640 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d70b1d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5450009fd0 tx=0x7f54500118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f8f31c525f3c93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f543e7fc640 1 -- 172.21.15.112:0/3969865670 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450004d20 con 0x55787d70adb0 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f543e7fc640 1 -- 172.21.15.112:0/3969865670 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5450004050 con 0x55787d70adb0 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f54557fa640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f54400b28a0 0x7f54400b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f543e7fc640 1 -- 172.21.15.112:0/3969865670 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450005260 con 0x55787d70adb0 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f5457fff640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f54400b28a0 0x7f54400b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.285+0000 7f54557fa640 1 -- 172.21.15.112:0/3515784042 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5448102a70 con 0x7f5458100a70 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5457fff640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f54400b28a0 0x7f54400b4d60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5448100550 tx=0x7f54480ec040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 -- 172.21.15.112:0/3969865670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 msgr2=0x55787d70b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d70b1d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5450009fd0 tx=0x7f54500118d0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 -- 172.21.15.112:0/3969865670 shutdown_connections 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d712da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 --2- 172.21.15.112:0/3969865670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d70b1d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 -- 172.21.15.112:0/3969865670 >> 172.21.15.112:0/3969865670 conn(0x55787d706420 msgr2=0x55787d706820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 -- 172.21.15.112:0/3969865670 shutdown_connections 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 -- 172.21.15.112:0/3969865670 wait complete. 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 -- start start 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.286+0000 7f5461283740 1 Processor -- start 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d714f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d71c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55787d7138c0 con 0x55787d70adb0 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55787d62c370 con 0x55787d70b7a0 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55787d713a40 con 0x55787d710910 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 10 client.0 _ll_register_callbacks cb 0x55787d380a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d714f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d714f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46756/0 (socket says 172.21.15.112:46756) 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 -- 172.21.15.112:0/3498100897 learned_addr learned my addr 172.21.15.112:0/3498100897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f54577fe640 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 -- 172.21.15.112:0/3498100897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 msgr2=0x55787d71c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d71c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 -- 172.21.15.112:0/3498100897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 msgr2=0x55787d7154b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7154b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 -- 172.21.15.112:0/3498100897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787d44d620 con 0x55787d70adb0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f545cb07640 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d714f70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5450009b00 tx=0x7f5450004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8e785b7cc6606a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f54377fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f54377fe640 1 -- 172.21.15.112:0/3498100897 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450028020 con 0x55787d70adb0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f54377fe640 1 -- 172.21.15.112:0/3498100897 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5450005040 con 0x55787d70adb0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5450004050 con 0x55787d70adb0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f54377fe640 1 -- 172.21.15.112:0/3498100897 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5450033050 con 0x55787d70adb0 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 10 client.16650 fetch_fsmap learned FSMap version 36 2024-05-10T10:45:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.287+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55787d71f440 con 0x55787d70adb0 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f54377fe640 1 -- 172.21.15.112:0/3498100897 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5450004050 con 0x55787d70adb0 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f54377fe640 1 -- 172.21.15.112:0/3498100897 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f545000f4e0 con 0x55787d70adb0 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f5461283740 10 client.16650 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f5461283740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f5461283740 20 client.16650 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f5461283740 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d720a90 0x55787d722e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f5461283740 4 client.16650 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55787d67d1e0 con 0x55787d720a90 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f54577fe640 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d720a90 0x55787d722e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f54377fe640 10 client.16650 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.288+0000 7f54577fe640 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d720a90 0x55787d722e80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8159b7164ca17e95 server_cookie=1a2ba0b8e3db7cf9 in_seq=0 out_seq=0 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f54377fe640 1 -- 172.21.15.112:0/3498100897 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55787d67d1e0 con 0x55787d720a90 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f54377fe640 10 client.16650 handle_command_reply: tid=0 2024-05-10T10:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f5461283740 1 client.16650 shutdown 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 msgr2=0x55787d714f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f5461283740 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d714f70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5450009b00 tx=0x7f5450004cc0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 shutdown_connections 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f5461283740 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d71c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.289+0000 7f5461283740 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d720a90 0x55787d722e80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.290+0000 7f5461283740 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d7154b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.290+0000 7f5461283740 1 --2- 172.21.15.112:0/3498100897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d714f70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.290+0000 7f54377fe640 0 client.16650 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.290+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 >> 172.21.15.112:0/3498100897 conn(0x55787d706420 msgr2=0x55787d705580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.290+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 shutdown_connections 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.290+0000 7f5461283740 1 -- 172.21.15.112:0/3498100897 wait complete. 2024-05-10T10:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.290+0000 7f5461283740 20 client.16650 trim_cache size 0 max 16384 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.304+0000 7f5461283740 1 Processor -- start 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f5461283740 1 -- start start 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f5461283740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70adb0 0x55787d715290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f5461283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d70b7a0 0x55787d71c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f5461283740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d710910 0x55787d709330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f5461283740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54581ab440 con 0x55787d710910 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f5461283740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54581ad100 con 0x55787d70adb0 2024-05-10T10:45:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f5461283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54581abb10 con 0x55787d70b7a0 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d710910 0x55787d709330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d710910 0x55787d709330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46764/0 (socket says 172.21.15.112:46764) 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 -- 172.21.15.112:0/127093433 learned_addr learned my addr 172.21.15.112:0/127093433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f54577fe640 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d70b7a0 0x55787d71c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70adb0 0x55787d715290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 -- 172.21.15.112:0/127093433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d70b7a0 msgr2=0x55787d71c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d70b7a0 0x55787d71c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 -- 172.21.15.112:0/127093433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70adb0 msgr2=0x55787d715290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70adb0 0x55787d715290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 -- 172.21.15.112:0/127093433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787d701a70 con 0x55787d710910 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f54577fe640 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d70b7a0 0x55787d71c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f545cb07640 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d710910 0x55787d709330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5450006010 tx=0x7f545002f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faa512eeec83f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f543dffb640 1 -- 172.21.15.112:0/127093433 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450015020 con 0x55787d710910 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f543dffb640 1 -- 172.21.15.112:0/127093433 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f545000f4e0 con 0x55787d710910 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.305+0000 7f543dffb640 1 -- 172.21.15.112:0/127093433 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450028040 con 0x55787d710910 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 -- 172.21.15.112:0/127093433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d710910 msgr2=0x55787d709330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d710910 0x55787d709330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5450006010 tx=0x7f545002f040 comp rx=0 tx=0).stop 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 -- 172.21.15.112:0/127093433 shutdown_connections 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d710910 0x55787d709330 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d70b7a0 0x55787d71c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 --2- 172.21.15.112:0/127093433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70adb0 0x55787d715290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 -- 172.21.15.112:0/127093433 >> 172.21.15.112:0/127093433 conn(0x55787d706420 msgr2=0x55787d714b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 -- 172.21.15.112:0/127093433 shutdown_connections 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 -- 172.21.15.112:0/127093433 wait complete. 2024-05-10T10:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 -- start start 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.306+0000 7f5461283740 1 Processor -- start 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d719790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d719cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d71ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54581ac970 con 0x55787d70adb0 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54581ac2a0 con 0x55787d70b7a0 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54581a9420 con 0x55787d710910 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 10 client.0 _ll_register_callbacks cb 0x7f5450009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d719790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d719790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46772/0 (socket says 172.21.15.112:46772) 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 -- 172.21.15.112:0/1219703459 learned_addr learned my addr 172.21.15.112:0/1219703459 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f54577fe640 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d719cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d71ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 -- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 msgr2=0x55787d71ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d71ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 -- 172.21.15.112:0/1219703459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 msgr2=0x55787d719cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d719cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 -- 172.21.15.112:0/1219703459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787d71b260 con 0x55787d70adb0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f545cb07640 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d719790 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5450009de0 tx=0x7f5450004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ec2ed8d0432c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f54377fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f54377fe640 1 -- 172.21.15.112:0/1219703459 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450048060 con 0x55787d70adb0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f54377fe640 1 -- 172.21.15.112:0/1219703459 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5450003c40 con 0x55787d70adb0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f545000f4e0 con 0x55787d70adb0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.307+0000 7f54377fe640 1 -- 172.21.15.112:0/1219703459 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5450015070 con 0x55787d70adb0 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 10 client.16656 fetch_fsmap learned FSMap version 36 2024-05-10T10:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55787d71c570 con 0x55787d70adb0 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f54377fe640 1 -- 172.21.15.112:0/1219703459 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f545000f4e0 con 0x55787d70adb0 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f54377fe640 1 -- 172.21.15.112:0/1219703459 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7f5450019c60 con 0x55787d70adb0 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 10 client.16656 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 20 client.16656 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d78b770 0x55787d78db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 4 client.16656 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55787d710680 con 0x55787d78b770 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f54577fe640 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d78b770 0x55787d78db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f54577fe640 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d78b770 0x55787d78db90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ee503e1ff6a8e2 server_cookie=517f104b9412b284 in_seq=0 out_seq=0 2024-05-10T10:45:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.308+0000 7f54377fe640 10 client.16656 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.329+0000 7f54377fe640 1 -- 172.21.15.112:0/1219703459 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55787d710680 con 0x55787d78b770 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.329+0000 7f54377fe640 10 client.16656 handle_command_reply: tid=0 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 client.16656 shutdown 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 msgr2=0x55787d719790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d719790 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5450009de0 tx=0x7f5450004660 comp rx=0 tx=0).stop 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 shutdown_connections 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55787d710910 0x55787d71ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55787d78b770 0x55787d78db90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55787d70b7a0 0x55787d719cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 --2- 172.21.15.112:0/1219703459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55787d70adb0 0x55787d719790 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f54377fe640 0 client.16656 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 >> 172.21.15.112:0/1219703459 conn(0x55787d706420 msgr2=0x55787d709330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 shutdown_connections 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 1 -- 172.21.15.112:0/1219703459 wait complete. 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f5461283740 20 client.16656 trim_cache size 0 max 16384 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:45:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f54400b28a0 msgr2=0x7f54400b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f545e56a640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f54400b28a0 0x7f54400b4d60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5448100550 tx=0x7f54480ec040 comp rx=0 tx=0).stop 2024-05-10T10:45:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458100a70 msgr2=0x7f5458198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.330+0000 7f545e56a640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458100a70 0x7f5458198af0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5448009230 tx=0x7f54480040c0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 shutdown_connections 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f54400b28a0 0x7f54400b4d60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5458104000 0x7f54581a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5458101430 0x7f5458199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 --2- 172.21.15.112:0/3515784042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5458100a70 0x7f5458198af0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 >> 172.21.15.112:0/3515784042 conn(0x7f54580fc610 msgr2=0x7f5458107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 shutdown_connections 2024-05-10T10:45:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:23.331+0000 7f545e56a640 1 -- 172.21.15.112:0/3515784042 wait complete. 2024-05-10T10:45:23.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:23 smithi148 ceph-mon[33568]: pgmap v584: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 680 op/s 2024-05-10T10:45:23.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:23 smithi195 ceph-mon[33773]: pgmap v584: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 680 op/s 2024-05-10T10:45:23.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:23 smithi112 ceph-mon[30100]: pgmap v584: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 680 op/s 2024-05-10T10:45:25.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:25 smithi148 ceph-mon[33568]: pgmap v585: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 451 op/s 2024-05-10T10:45:25.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:25 smithi195 ceph-mon[33773]: pgmap v585: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 451 op/s 2024-05-10T10:45:25.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:25 smithi112 ceph-mon[30100]: pgmap v585: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 451 op/s 2024-05-10T10:45:27.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:27 smithi148 ceph-mon[33568]: pgmap v586: 129 pgs: 129 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 597 op/s 2024-05-10T10:45:27.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:27 smithi195 ceph-mon[33773]: pgmap v586: 129 pgs: 129 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 597 op/s 2024-05-10T10:45:27.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:27 smithi112 ceph-mon[30100]: pgmap v586: 129 pgs: 129 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.1 MiB/s wr, 597 op/s 2024-05-10T10:45:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:29 smithi148 ceph-mon[33568]: pgmap v587: 129 pgs: 129 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-05-10T10:45:29.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:29 smithi195 ceph-mon[33773]: pgmap v587: 129 pgs: 129 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-05-10T10:45:29.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:29 smithi112 ceph-mon[30100]: pgmap v587: 129 pgs: 129 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-05-10T10:45:31.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:31 smithi148 ceph-mon[33568]: pgmap v588: 129 pgs: 129 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.0 MiB/s wr, 443 op/s 2024-05-10T10:45:31.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:31 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:31.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:31 smithi195 ceph-mon[33773]: pgmap v588: 129 pgs: 129 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.0 MiB/s wr, 443 op/s 2024-05-10T10:45:31.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:31 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:31 smithi112 ceph-mon[30100]: pgmap v588: 129 pgs: 129 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.0 MiB/s wr, 443 op/s 2024-05-10T10:45:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:31 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:33.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:45:33.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:33 smithi148 ceph-mon[33568]: pgmap v589: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 651 op/s 2024-05-10T10:45:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:33 smithi195 ceph-mon[33773]: pgmap v589: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 651 op/s 2024-05-10T10:45:33.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:33 smithi112 ceph-mon[30100]: pgmap v589: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 651 op/s 2024-05-10T10:45:33.616 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:35.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:35 smithi112 ceph-mon[30100]: pgmap v590: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2024-05-10T10:45:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 -- 172.21.15.112:0/2413845132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103ee0 msgr2=0x7f262c110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 --2- 172.21.15.112:0/2413845132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103ee0 0x7f262c110370 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f262400b0a0 tx=0x7f262400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 -- 172.21.15.112:0/2413845132 shutdown_connections 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 --2- 172.21.15.112:0/2413845132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103ee0 0x7f262c110370 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 --2- 172.21.15.112:0/2413845132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c109530 0x7f262c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 --2- 172.21.15.112:0/2413845132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f262c103520 0x7f262c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 -- 172.21.15.112:0/2413845132 >> 172.21.15.112:0/2413845132 conn(0x7f262c07a570 msgr2=0x7f262c07a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 -- 172.21.15.112:0/2413845132 shutdown_connections 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.489+0000 7f2633dea700 1 -- 172.21.15.112:0/2413845132 wait complete. 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 Processor -- start 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 -- start start 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103520 0x7f262c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c103ee0 0x7f262c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f262c109530 0x7f262c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f262c113520 con 0x7f262c103520 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1133a0 con 0x7f262c103ee0 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2633dea700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1136a0 con 0x7f262c109530 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2631b86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103520 0x7f262c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2631385700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c103ee0 0x7f262c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2632387700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f262c109530 0x7f262c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2631b86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103520 0x7f262c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50928/0 (socket says 172.21.15.112:50928) 2024-05-10T10:45:35.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2631b86700 1 -- 172.21.15.112:0/3016216497 learned_addr learned my addr 172.21.15.112:0/3016216497 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2632387700 1 -- 172.21.15.112:0/3016216497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c103ee0 msgr2=0x7f262c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2632387700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c103ee0 0x7f262c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2632387700 1 -- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103520 msgr2=0x7f262c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2632387700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103520 0x7f262c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2632387700 1 -- 172.21.15.112:0/3016216497 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c199fe0 con 0x7f262c109530 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.490+0000 7f2631b86700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103520 0x7f262c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.491+0000 7f2632387700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f262c109530 0x7f262c199800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f262400b0a0 tx=0x7f2624004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.491+0000 7f2622ffd700 1 -- 172.21.15.112:0/3016216497 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624021070 con 0x7f262c109530 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.491+0000 7f2622ffd700 1 -- 172.21.15.112:0/3016216497 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2624007b80 con 0x7f262c109530 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.491+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c19a270 con 0x7f262c109530 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.491+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1a64e0 con 0x7f262c109530 2024-05-10T10:45:35.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.491+0000 7f2622ffd700 1 -- 172.21.15.112:0/3016216497 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624005040 con 0x7f262c109530 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.494+0000 7f2622ffd700 1 -- 172.21.15.112:0/3016216497 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2624024070 con 0x7f262c109530 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.494+0000 7f2622ffd700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f26180b2880 0x7f26180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.494+0000 7f2631b86700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f26180b2880 0x7f26180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.494+0000 7f2622ffd700 1 -- 172.21.15.112:0/3016216497 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f262400f020 con 0x7f262c109530 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f2631b86700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f26180b2880 0x7f26180b4d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2628011550 tx=0x7f2628009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 Processor -- start 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 -- start start 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8a4d0 0x55e136e8a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e8e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e8f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e136ddc860 con 0x55e136e8a4d0 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e136d7b7c0 con 0x55e136e8ec80 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.495+0000 7f263e67db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e136dd8820 con 0x55e136e8adf0 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8a4d0 0x55e136e8a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631385700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e8e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8a4d0 0x55e136e8a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55042/0 (socket says 172.21.15.112:55042) 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2632387700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e8f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 -- 172.21.15.112:0/2248973752 learned_addr learned my addr 172.21.15.112:0/2248973752 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631385700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e8e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:42758/0 (socket says 172.21.15.112:42758) 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 -- 172.21.15.112:0/2248973752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 msgr2=0x55e136e8e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e8e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 -- 172.21.15.112:0/2248973752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 msgr2=0x55e136e8f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e8f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 -- 172.21.15.112:0/2248973752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e136d298d0 con 0x55e136e8a4d0 2024-05-10T10:45:35.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2632387700 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e8f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:35.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2631b86700 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8a4d0 0x55e136e8a8b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f262800e730 tx=0x7f2628014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa070666be5a027 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:35.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2617fff700 1 -- 172.21.15.112:0/2248973752 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628004830 con 0x55e136e8a4d0 2024-05-10T10:45:35.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2617fff700 1 -- 172.21.15.112:0/2248973752 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2628005020 con 0x55e136e8a4d0 2024-05-10T10:45:35.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.496+0000 7f2617fff700 1 -- 172.21.15.112:0/2248973752 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262801e680 con 0x55e136e8a4d0 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 -- 172.21.15.112:0/2248973752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8a4d0 msgr2=0x55e136e8a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8a4d0 0x55e136e8a8b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f262800e730 tx=0x7f2628014000 comp rx=0 tx=0).stop 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 -- 172.21.15.112:0/2248973752 shutdown_connections 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e8f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8a4d0 0x55e136e8a8b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 --2- 172.21.15.112:0/2248973752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e8e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 -- 172.21.15.112:0/2248973752 >> 172.21.15.112:0/2248973752 conn(0x55e136db6100 msgr2=0x55e136e84c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 -- 172.21.15.112:0/2248973752 shutdown_connections 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.497+0000 7f263e67db80 1 -- 172.21.15.112:0/2248973752 wait complete. 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 -- start start 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 Processor -- start 2024-05-10T10:45:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e93920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e136e96e90 con 0x55e136e8adf0 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e136d42c30 con 0x55e136e8ec80 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e136e97010 con 0x55e136e8a4d0 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.498+0000 7f263e67db80 10 client.0 _ll_register_callbacks cb 0x55e136d74920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e933e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55054/0 (socket says 172.21.15.112:55054) 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 -- 172.21.15.112:0/2097924263 learned_addr learned my addr 172.21.15.112:0/2097924263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631b86700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2632387700 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e93920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 -- 172.21.15.112:0/2097924263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 msgr2=0x55e136e96040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e96040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 -- 172.21.15.112:0/2097924263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 msgr2=0x55e136e93920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e93920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 -- 172.21.15.112:0/2097924263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e136d78690 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f2631385700 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e933e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f261c0127f0 tx=0x7f261c012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eec85e2f8ea312 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f26157fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f26157fa700 1 -- 172.21.15.112:0/2097924263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c00f4e0 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f26157fa700 1 -- 172.21.15.112:0/2097924263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f261c00f680 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.499+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2628005020 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f26157fa700 1 -- 172.21.15.112:0/2097924263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f261c00f880 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 10 client.16668 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e136ea5ae0 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f26157fa700 1 -- 172.21.15.112:0/2097924263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2628005020 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f26157fa700 1 -- 172.21.15.112:0/2097924263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f261c010930 con 0x55e136e8adf0 2024-05-10T10:45:35.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 10 client.16668 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 20 client.16668 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136ea8600 0x55e136eaaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 4 client.16668 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e136e56650 con 0x55e136ea8600 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.500+0000 7f2631b86700 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136ea8600 0x55e136eaaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.501+0000 7f26157fa700 10 client.16668 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.501+0000 7f2631b86700 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136ea8600 0x55e136eaaa20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead254cc8234eb9e server_cookie=28a3a1a0fb02c886 in_seq=0 out_seq=0 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.501+0000 7f26157fa700 1 -- 172.21.15.112:0/2097924263 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e136e56650 con 0x55e136ea8600 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.501+0000 7f26157fa700 10 client.16668 handle_command_reply: tid=0 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.501+0000 7f263e67db80 1 client.16668 shutdown 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 msgr2=0x55e136e933e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e933e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f261c0127f0 tx=0x7f261c012b00 comp rx=0 tx=0).stop 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 shutdown_connections 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136ea8600 0x55e136eaaa20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136e93920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e96040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 --2- 172.21.15.112:0/2097924263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e933e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 >> 172.21.15.112:0/2097924263 conn(0x55e136db6100 msgr2=0x55e136e7f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f26157fa700 0 client.16668 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 shutdown_connections 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 1 -- 172.21.15.112:0/2097924263 wait complete. 2024-05-10T10:45:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.502+0000 7f263e67db80 20 client.16668 trim_cache size 0 max 16384 2024-05-10T10:45:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.525+0000 7f263e67db80 1 Processor -- start 2024-05-10T10:45:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.525+0000 7f263e67db80 1 -- start start 2024-05-10T10:45:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f263e67db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f263e67db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f263e67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136eb02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f263e67db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1aceb0 con 0x55e136e8adf0 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f263e67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1aeb10 con 0x55e136e8ec80 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f263e67db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1ad580 con 0x55e136e8a4d0 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e9ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55070/0 (socket says 172.21.15.112:55070) 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631385700 1 -- 172.21.15.112:0/801514096 learned_addr learned my addr 172.21.15.112:0/801514096 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631b86700 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2632387700 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136eb02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631b86700 1 -- 172.21.15.112:0/801514096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 msgr2=0x55e136eb02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631b86700 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136eb02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631b86700 1 -- 172.21.15.112:0/801514096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 msgr2=0x55e136e9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631b86700 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e9ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631b86700 1 -- 172.21.15.112:0/801514096 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e136e963b0 con 0x55e136e8a4d0 2024-05-10T10:45:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631385700 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.526+0000 7f2631b86700 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e92040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2628009f50 tx=0x7f2628028040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8e3add1f3c57f8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.527+0000 7f26177fe700 1 -- 172.21.15.112:0/801514096 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280054f0 con 0x55e136e8a4d0 2024-05-10T10:45:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.527+0000 7f26177fe700 1 -- 172.21.15.112:0/801514096 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2628005690 con 0x55e136e8a4d0 2024-05-10T10:45:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.527+0000 7f26177fe700 1 -- 172.21.15.112:0/801514096 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262801e640 con 0x55e136e8a4d0 2024-05-10T10:45:35.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.527+0000 7f263e67db80 1 -- 172.21.15.112:0/801514096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 msgr2=0x55e136e92040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e92040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2628009f50 tx=0x7f2628028040 comp rx=0 tx=0).stop 2024-05-10T10:45:35.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 -- 172.21.15.112:0/801514096 shutdown_connections 2024-05-10T10:45:35.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8ec80 0x55e136eb02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8a4d0 0x55e136e92040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 --2- 172.21.15.112:0/801514096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8adf0 0x55e136e9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 -- 172.21.15.112:0/801514096 >> 172.21.15.112:0/801514096 conn(0x55e136db6100 msgr2=0x55e136e7f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 -- 172.21.15.112:0/801514096 shutdown_connections 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 -- 172.21.15.112:0/801514096 wait complete. 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 -- start start 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.528+0000 7f263e67db80 1 Processor -- start 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8a4d0 0x55e136e9c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8ec80 0x55e136e97de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1ae3e0 con 0x55e136e8ec80 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1add10 con 0x55e136e8a4d0 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1aae50 con 0x55e136e8adf0 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 10 client.0 _ll_register_callbacks cb 0x55e136d74920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:35.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8a4d0 0x55e136e9c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2632387700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8ec80 0x55e136e97de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631385700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8a4d0 0x55e136e9c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48196/0 (socket says 172.21.15.112:48196) 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 -- 172.21.15.112:0/741225075 learned_addr learned my addr 172.21.15.112:0/741225075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 -- 172.21.15.112:0/741225075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 msgr2=0x55e136e978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 -- 172.21.15.112:0/741225075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8ec80 msgr2=0x55e136e97de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8ec80 0x55e136e97de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 -- 172.21.15.112:0/741225075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e136e98320 con 0x55e136e8a4d0 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2632387700 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8ec80 0x55e136e97de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f2631b86700 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8a4d0 0x55e136e9c4d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f262801e500 tx=0x7f262801e530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b767bfe29b31284a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f26157fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f26157fa700 1 -- 172.21.15.112:0/741225075 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262801ec50 con 0x55e136e8a4d0 2024-05-10T10:45:35.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f26157fa700 1 -- 172.21.15.112:0/741225075 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f262801edf0 con 0x55e136e8a4d0 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.529+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e136dea240 con 0x55e136e8a4d0 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f26157fa700 1 -- 172.21.15.112:0/741225075 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2628034120 con 0x55e136e8a4d0 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 10 client.26042 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e136f12bc0 con 0x55e136e8a4d0 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f26157fa700 1 -- 172.21.15.112:0/741225075 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e136dea240 con 0x55e136e8a4d0 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f26157fa700 1 -- 172.21.15.112:0/741225075 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f262803d750 con 0x55e136e8a4d0 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 10 client.26042 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 20 client.26042 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136f12f80 0x55e136f153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 4 client.26042 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e136de42e0 con 0x55e136f12f80 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.530+0000 7f2631385700 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136f12f80 0x55e136f153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:35.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f2631385700 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136f12f80 0x55e136f153a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eebbdb1b236fedd2 server_cookie=4ccfe45b15f3e8b1 in_seq=0 out_seq=0 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f26157fa700 10 client.26042 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f26157fa700 1 -- 172.21.15.112:0/741225075 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e136de42e0 con 0x55e136f12f80 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f26157fa700 10 client.26042 handle_command_reply: tid=0 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 client.26042 shutdown 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8a4d0 msgr2=0x55e136e9c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8a4d0 0x55e136e9c4d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f262801e500 tx=0x7f262801e530 comp rx=0 tx=0).stop 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 shutdown_connections 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55e136f12f80 0x55e136f153a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e136e8ec80 0x55e136e97de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e136e8a4d0 0x55e136e9c4d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 --2- 172.21.15.112:0/741225075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e136e8adf0 0x55e136e978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f26157fa700 0 client.26042 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:35.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 >> 172.21.15.112:0/741225075 conn(0x55e136db6100 msgr2=0x55e136e892e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:35.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.531+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 shutdown_connections 2024-05-10T10:45:35.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f263e67db80 1 -- 172.21.15.112:0/741225075 wait complete. 2024-05-10T10:45:35.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f263e67db80 20 client.26042 trim_cache size 0 max 16384 2024-05-10T10:45:35.539 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:45:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f26180b2880 msgr2=0x7f26180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f26180b2880 0x7f26180b4d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2628011550 tx=0x7f2628009260 comp rx=0 tx=0).stop 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f262c109530 msgr2=0x7f262c199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f262c109530 0x7f262c199800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f262400b0a0 tx=0x7f2624004d00 comp rx=0 tx=0).stop 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 shutdown_connections 2024-05-10T10:45:35.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f26180b2880 0x7f26180b4d40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f262c109530 0x7f262c199800 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c103520 0x7f262c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 --2- 172.21.15.112:0/3016216497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f262c103ee0 0x7f262c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:35.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 >> 172.21.15.112:0/3016216497 conn(0x7f262c07a570 msgr2=0x7f262c102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:35.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 shutdown_connections 2024-05-10T10:45:35.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:35.532+0000 7f2633dea700 1 -- 172.21.15.112:0/3016216497 wait complete. 2024-05-10T10:45:35.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:35 smithi148 ceph-mon[33568]: pgmap v590: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2024-05-10T10:45:35.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:35 smithi195 ceph-mon[33773]: pgmap v590: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2024-05-10T10:45:36.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:45:36.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.442+0000 7f35d1e83640 1 -- 172.21.15.112:0/2426098154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 msgr2=0x7f35cc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.442+0000 7f35d1e83640 1 --2- 172.21.15.112:0/2426098154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 0x7f35cc109d50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f35c400bd70 tx=0x7f35c402c430 comp rx=0 tx=0).stop 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 -- 172.21.15.112:0/2426098154 shutdown_connections 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 --2- 172.21.15.112:0/2426098154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 0x7f35cc109d50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 --2- 172.21.15.112:0/2426098154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 0x7f35cc102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 --2- 172.21.15.112:0/2426098154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc101910 0x7f35cc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 -- 172.21.15.112:0/2426098154 >> 172.21.15.112:0/2426098154 conn(0x7f35cc0fce80 msgr2=0x7f35cc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 -- 172.21.15.112:0/2426098154 shutdown_connections 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 -- 172.21.15.112:0/2426098154 wait complete. 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 Processor -- start 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.443+0000 7f35d1e83640 1 -- start start 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35d1e83640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc101910 0x7f35cc196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35d1e83640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 0x7f35cc196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35d1e83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 0x7f35cc19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35d1e83640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc10c1f0 con 0x7f35cc1034c0 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35d1e83640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc10c070 con 0x7f35cc101910 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35d1e83640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc10c370 con 0x7f35cc102b00 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35caffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 0x7f35cc196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc101910 0x7f35cc196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 0x7f35cc19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35caffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 0x7f35cc196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:42806/0 (socket says 172.21.15.112:42806) 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35caffd640 1 -- 172.21.15.112:0/39270598 learned_addr learned my addr 172.21.15.112:0/39270598 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc101910 0x7f35cc196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48216/0 (socket says 172.21.15.112:48216) 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cbfff640 1 -- 172.21.15.112:0/39270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 msgr2=0x7f35cc196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cbfff640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 0x7f35cc196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cbfff640 1 -- 172.21.15.112:0/39270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc101910 msgr2=0x7f35cc196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cbfff640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc101910 0x7f35cc196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cbfff640 1 -- 172.21.15.112:0/39270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc19e470 con 0x7f35cc1034c0 2024-05-10T10:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35caffd640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 0x7f35cc196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.444+0000 7f35cbfff640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 0x7f35cc19de00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f35c402c950 tx=0x7f35c4033a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.445+0000 7f35c8ff9640 1 -- 172.21.15.112:0/39270598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4043020 con 0x7f35cc1034c0 2024-05-10T10:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.445+0000 7f35c8ff9640 1 -- 172.21.15.112:0/39270598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35c4004040 con 0x7f35cc1034c0 2024-05-10T10:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.445+0000 7f35c8ff9640 1 -- 172.21.15.112:0/39270598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4038040 con 0x7f35cc1034c0 2024-05-10T10:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.445+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc19e700 con 0x7f35cc1034c0 2024-05-10T10:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.445+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc19eb70 con 0x7f35cc1034c0 2024-05-10T10:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.448+0000 7f35d4ba1740 1 Processor -- start 2024-05-10T10:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.448+0000 7f35c8ff9640 1 -- 172.21.15.112:0/39270598 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f35c4046020 con 0x7f35cc1034c0 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35d4ba1740 1 -- start start 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d61c0 0x5569743d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d6bb0 0x5569743db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5569743365d0 con 0x5569743d6bb0 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5569742ef890 con 0x5569743d61c0 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556974332590 con 0x5569743dbd20 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d61c0 0x5569743d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d6bb0 0x5569743db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d61c0 0x5569743d65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48222/0 (socket says 172.21.15.112:48222) 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 -- 172.21.15.112:0/249871209 learned_addr learned my addr 172.21.15.112:0/249871209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 -- 172.21.15.112:0/249871209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 msgr2=0x5569743de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 -- 172.21.15.112:0/249871209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d6bb0 msgr2=0x5569743db6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d6bb0 0x5569743db6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35caffd640 1 -- 172.21.15.112:0/249871209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556974048210 con 0x5569743d61c0 2024-05-10T10:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35c8ff9640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f35b40b2920 0x7f35b40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.449+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d6bb0 0x5569743db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.450+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f35b40b2920 0x7f35b40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.450+0000 7f35c8ff9640 1 -- 172.21.15.112:0/39270598 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f35c4102dc0 con 0x7f35cc1034c0 2024-05-10T10:45:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.450+0000 7f35caffd640 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d61c0 0x5569743d65e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009810 tx=0x7f35c0009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d99c23c3d1589f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.450+0000 7f35b1ffb640 1 -- 172.21.15.112:0/249871209 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000fc50 con 0x5569743d61c0 2024-05-10T10:45:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.450+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f35b40b2920 0x7f35b40b4de0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5569743d6fb0 tx=0x7f35bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.450+0000 7f35b1ffb640 1 -- 172.21.15.112:0/249871209 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35c000fdf0 con 0x5569743d61c0 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.450+0000 7f35b1ffb640 1 -- 172.21.15.112:0/249871209 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000e610 con 0x5569743d61c0 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 -- 172.21.15.112:0/249871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d61c0 msgr2=0x5569743d65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d61c0 0x5569743d65e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009810 tx=0x7f35c0009c80 comp rx=0 tx=0).stop 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 -- 172.21.15.112:0/249871209 shutdown_connections 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d6bb0 0x5569743db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/249871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d61c0 0x5569743d65e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 -- 172.21.15.112:0/249871209 >> 172.21.15.112:0/249871209 conn(0x5569743d1830 msgr2=0x5569743d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 -- 172.21.15.112:0/249871209 shutdown_connections 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 -- 172.21.15.112:0/249871209 wait complete. 2024-05-10T10:45:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.451+0000 7f35d4ba1740 1 -- start start 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 1 Processor -- start 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5569743decd0 con 0x5569743d61c0 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5569742f3370 con 0x5569743d6bb0 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5569743dee50 con 0x5569743dbd20 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35d4ba1740 10 client.0 _ll_register_callbacks cb 0x556974047a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35caffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35cbfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35caffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743e0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48230/0 (socket says 172.21.15.112:48230) 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.452+0000 7f35caffd640 1 -- 172.21.15.112:0/1721840949 learned_addr learned my addr 172.21.15.112:0/1721840949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cbfff640 1 -- 172.21.15.112:0/1721840949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 msgr2=0x5569743e0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cbfff640 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743e0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cbfff640 1 -- 172.21.15.112:0/1721840949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 msgr2=0x5569743e04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cbfff640 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cbfff640 1 -- 172.21.15.112:0/1721840949 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556974114620 con 0x5569743dbd20 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35cbfff640 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743e7a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35c4100580 tx=0x7f35c40c9880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4fe6873012cb19 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35aaffd640 1 -- 172.21.15.112:0/1721840949 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40ca050 con 0x5569743dbd20 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35aaffd640 1 -- 172.21.15.112:0/1721840949 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f35c40d2a20 con 0x5569743dbd20 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.453+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c000fdf0 con 0x5569743dbd20 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35aaffd640 1 -- 172.21.15.112:0/1721840949 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f35c40ce400 con 0x5569743dbd20 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 10 client.26080 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5569743deda0 con 0x5569743dbd20 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35aaffd640 1 -- 172.21.15.112:0/1721840949 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35c000fdf0 con 0x5569743dbd20 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35aaffd640 1 -- 172.21.15.112:0/1721840949 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f35c40c92b0 con 0x5569743dbd20 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 10 client.26080 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 10 client.26080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 20 client.26080 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5569743ec240 0x5569743ee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 4 client.26080 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569743441e0 con 0x5569743ec240 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.454+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5569743ec240 0x5569743ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.455+0000 7f35aaffd640 10 client.26080 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.455+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5569743ec240 0x5569743ee700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81dda07cf3326a3d server_cookie=e22c3305fee68bde in_seq=0 out_seq=0 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.455+0000 7f35aaffd640 1 -- 172.21.15.112:0/1721840949 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569743441e0 con 0x5569743ec240 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.455+0000 7f35aaffd640 10 client.26080 handle_command_reply: tid=0 2024-05-10T10:45:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.455+0000 7f35d4ba1740 1 client.26080 shutdown 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 msgr2=0x5569743e7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743e7a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35c4100580 tx=0x7f35c40c9880 comp rx=0 tx=0).stop 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 shutdown_connections 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743e7a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5569743ec240 0x5569743ee700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35aaffd640 0 client.26080 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743e0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721840949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 >> 172.21.15.112:0/1721840949 conn(0x5569743d1830 msgr2=0x5569743cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 shutdown_connections 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721840949 wait complete. 2024-05-10T10:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.456+0000 7f35d4ba1740 20 client.26080 trim_cache size 0 max 16384 2024-05-10T10:45:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.484+0000 7f35d4ba1740 1 Processor -- start 2024-05-10T10:45:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.484+0000 7f35d4ba1740 1 -- start start 2024-05-10T10:45:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743d7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743d8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1a90f0 con 0x5569743d61c0 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1aad50 con 0x5569743d6bb0 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1a97c0 con 0x5569743dbd20 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35caffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743d7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35caffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743d7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48242/0 (socket says 172.21.15.112:48242) 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55138/0 (socket says 172.21.15.112:55138) 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35caffd640 1 -- 172.21.15.112:0/1721075317 learned_addr learned my addr 172.21.15.112:0/1721075317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cbfff640 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743d8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cbfff640 1 -- 172.21.15.112:0/1721075317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 msgr2=0x5569743d7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cbfff640 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743d7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cbfff640 1 -- 172.21.15.112:0/1721075317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 msgr2=0x5569743c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cbfff640 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cbfff640 1 -- 172.21.15.112:0/1721075317 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569743d1660 con 0x5569743dbd20 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.485+0000 7f35caffd640 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743d7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.486+0000 7f35cbfff640 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743d8450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e090 tx=0x7f35c40c98f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a0b28affd0c91b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.486+0000 7f35b17fa640 1 -- 172.21.15.112:0/1721075317 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40ca050 con 0x5569743dbd20 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.486+0000 7f35b17fa640 1 -- 172.21.15.112:0/1721075317 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35c40d2eb0 con 0x5569743dbd20 2024-05-10T10:45:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.486+0000 7f35b17fa640 1 -- 172.21.15.112:0/1721075317 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40d2a20 con 0x5569743dbd20 2024-05-10T10:45:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721075317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 msgr2=0x5569743d8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743d8450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e090 tx=0x7f35c40c98f0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721075317 shutdown_connections 2024-05-10T10:45:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743dbd20 0x5569743d8450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743d6bb0 0x5569743d7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/1721075317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721075317 >> 172.21.15.112:0/1721075317 conn(0x5569743d1830 msgr2=0x5569743d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721075317 shutdown_connections 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 -- 172.21.15.112:0/1721075317 wait complete. 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.487+0000 7f35d4ba1740 1 -- start start 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 1 Processor -- start 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743d6bb0 0x5569743e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743dbd20 0x5569743e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1aa620 con 0x5569743d61c0 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1a9f50 con 0x5569743dbd20 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1a70d0 con 0x5569743d6bb0 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35d4ba1740 10 client.0 _ll_register_callbacks cb 0x7f35c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35caffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743d6bb0 0x5569743e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55142/0 (socket says 172.21.15.112:55142) 2024-05-10T10:45:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 -- 172.21.15.112:0/4010749399 learned_addr learned my addr 172.21.15.112:0/4010749399 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35caffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743d6bb0 0x5569743e4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:42830/0 (socket says 172.21.15.112:42830) 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cbfff640 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743dbd20 0x5569743e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 -- 172.21.15.112:0/4010749399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743d6bb0 msgr2=0x5569743e4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743d6bb0 0x5569743e4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 -- 172.21.15.112:0/4010749399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743dbd20 msgr2=0x5569743e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743dbd20 0x5569743e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cb7fe640 1 -- 172.21.15.112:0/4010749399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569743d5e00 con 0x5569743d61c0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.488+0000 7f35cbfff640 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743dbd20 0x5569743e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35cb7fe640 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e4990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009ad0 tx=0x7f35bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb11056e48cf8f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35aaffd640 1 -- 172.21.15.112:0/4010749399 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc003ac0 con 0x5569743d61c0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35aaffd640 1 -- 172.21.15.112:0/4010749399 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f35bc003c60 con 0x5569743d61c0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c40d2eb0 con 0x5569743d61c0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35aaffd640 1 -- 172.21.15.112:0/4010749399 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f35bc03c050 con 0x5569743d61c0 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35d4ba1740 10 client.16689 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.489+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556974456c10 con 0x5569743d61c0 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35aaffd640 1 -- 172.21.15.112:0/4010749399 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35c40d2eb0 con 0x5569743d61c0 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35aaffd640 1 -- 172.21.15.112:0/4010749399 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f35bc033470 con 0x5569743d61c0 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35d4ba1740 10 client.16689 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35d4ba1740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35d4ba1740 20 client.16689 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556974457650 0x556974459a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35d4ba1740 4 client.16689 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55697433e050 con 0x556974457650 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.490+0000 7f35caffd640 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556974457650 0x556974459a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.491+0000 7f35aaffd640 10 client.16689 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.491+0000 7f35caffd640 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556974457650 0x556974459a70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a91d914103ebb28 server_cookie=e1e1319e077feb81 in_seq=0 out_seq=0 2024-05-10T10:45:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.503+0000 7f35aaffd640 1 -- 172.21.15.112:0/4010749399 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55697433e050 con 0x556974457650 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.503+0000 7f35aaffd640 10 client.16689 handle_command_reply: tid=0 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.503+0000 7f35d4ba1740 1 client.16689 shutdown 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 msgr2=0x5569743e4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e4990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009ad0 tx=0x7f35bc004730 comp rx=0 tx=0).stop 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 shutdown_connections 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x556974457650 0x556974459a70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35aaffd640 0 client.16689 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569743dbd20 0x5569743e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569743d6bb0 0x5569743e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 --2- 172.21.15.112:0/4010749399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569743d61c0 0x5569743e4990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 >> 172.21.15.112:0/4010749399 conn(0x5569743d1830 msgr2=0x5569743e04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 shutdown_connections 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 1 -- 172.21.15.112:0/4010749399 wait complete. 2024-05-10T10:45:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.504+0000 7f35d4ba1740 20 client.16689 trim_cache size 0 max 16384 2024-05-10T10:45:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:45:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:45:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:45:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f35b40b2920 msgr2=0x7f35b40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f35b40b2920 0x7f35b40b4de0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5569743d6fb0 tx=0x7f35bc009260 comp rx=0 tx=0).stop 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 msgr2=0x7f35cc19de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 0x7f35cc19de00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f35c402c950 tx=0x7f35c4033a40 comp rx=0 tx=0).stop 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 shutdown_connections 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f35b40b2920 0x7f35b40b4de0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35cc1034c0 0x7f35cc19de00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35cc102b00 0x7f35cc196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 --2- 172.21.15.112:0/39270598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc101910 0x7f35cc196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.505+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 >> 172.21.15.112:0/39270598 conn(0x7f35cc0fce80 msgr2=0x7f35cc105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.506+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 shutdown_connections 2024-05-10T10:45:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:36.506+0000 7f35d1e83640 1 -- 172.21.15.112:0/39270598 wait complete. 2024-05-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:37.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:37 smithi148 ceph-mon[33568]: pgmap v591: 129 pgs: 129 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 609 op/s 2024-05-10T10:45:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:37 smithi195 ceph-mon[33773]: pgmap v591: 129 pgs: 129 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 609 op/s 2024-05-10T10:45:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:37 smithi112 ceph-mon[30100]: pgmap v591: 129 pgs: 129 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 609 op/s 2024-05-10T10:45:38.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.747+0000 7f14553f0700 1 -- 172.21.15.112:0/3427370914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450108410 msgr2=0x7f14501087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:38.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.747+0000 7f14553f0700 1 --2- 172.21.15.112:0/3427370914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450108410 0x7f14501087f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1440009990 tx=0x7f1440009ca0 comp rx=0 tx=0).stop 2024-05-10T10:45:38.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.747+0000 7f14553f0700 1 -- 172.21.15.112:0/3427370914 shutdown_connections 2024-05-10T10:45:38.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.747+0000 7f14553f0700 1 --2- 172.21.15.112:0/3427370914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1450102d60 0x7f145010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:38.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.747+0000 7f14553f0700 1 --2- 172.21.15.112:0/3427370914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450108410 0x7f14501087f0 secure :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1440009990 tx=0x7f1440009ca0 comp rx=0 tx=0).stop 2024-05-10T10:45:38.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.747+0000 7f14553f0700 1 --2- 172.21.15.112:0/3427370914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14501023a0 0x7f1450102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:38.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.747+0000 7f14553f0700 1 -- 172.21.15.112:0/3427370914 >> 172.21.15.112:0/3427370914 conn(0x7f1450079350 msgr2=0x7f1450079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:38.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.748+0000 7f14553f0700 1 -- 172.21.15.112:0/3427370914 shutdown_connections 2024-05-10T10:45:38.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.748+0000 7f14553f0700 1 -- 172.21.15.112:0/3427370914 wait complete. 2024-05-10T10:45:38.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.748+0000 7f14553f0700 1 Processor -- start 2024-05-10T10:45:38.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f14553f0700 1 -- start start 2024-05-10T10:45:38.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f14553f0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14501023a0 0x7f145010fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:38.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f14553f0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450102d60 0x7f1450110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:38.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f14553f0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1450110760 0x7f14501a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f14553f0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14501135e0 con 0x7f1450102d60 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f14553f0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113460 con 0x7f1450110760 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f14553f0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113760 con 0x7f14501023a0 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450102d60 0x7f1450110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450102d60 0x7f1450110220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55168/0 (socket says 172.21.15.112:55168) 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144effd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14501023a0 0x7f145010fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144e7fc700 1 -- 172.21.15.112:0/999907619 learned_addr learned my addr 172.21.15.112:0/999907619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144f7fe700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1450110760 0x7f14501a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144effd700 1 -- 172.21.15.112:0/999907619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1450110760 msgr2=0x7f14501a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144effd700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1450110760 0x7f14501a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:38.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144effd700 1 -- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450102d60 msgr2=0x7f1450110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144effd700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450102d60 0x7f1450110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144effd700 1 -- 172.21.15.112:0/999907619 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14501a8670 con 0x7f14501023a0 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.749+0000 7f144e7fc700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450102d60 0x7f1450110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.750+0000 7f144effd700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14501023a0 0x7f145010fce0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1440008000 tx=0x7f14400047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.750+0000 7f1437fff700 1 -- 172.21.15.112:0/999907619 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440026070 con 0x7f14501023a0 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.750+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14501a8900 con 0x7f14501023a0 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.750+0000 7f1437fff700 1 -- 172.21.15.112:0/999907619 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1440021410 con 0x7f14501023a0 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.750+0000 7f1437fff700 1 -- 172.21.15.112:0/999907619 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440017650 con 0x7f14501023a0 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.750+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14501a8d00 con 0x7f14501023a0 2024-05-10T10:45:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.751+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145006a190 con 0x7f14501023a0 2024-05-10T10:45:38.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.753+0000 7f1437fff700 1 -- 172.21.15.112:0/999907619 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1440009d10 con 0x7f14501023a0 2024-05-10T10:45:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.753+0000 7f1437fff700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14380b28c0 0x7f14380b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.753+0000 7f144e7fc700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14380b28c0 0x7f14380b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.753+0000 7f1437fff700 1 -- 172.21.15.112:0/999907619 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14400e3420 con 0x7f14501023a0 2024-05-10T10:45:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.754+0000 7f144e7fc700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14380b28c0 0x7f14380b4d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1450100d30 tx=0x7f1444009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:38.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:38.757+0000 7f1437fff700 1 -- 172.21.15.112:0/999907619 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1440013070 con 0x7f14501023a0 2024-05-10T10:45:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.064+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f145002e510 con 0x7f14501023a0 2024-05-10T10:45:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.064+0000 7f1437fff700 1 -- 172.21.15.112:0/999907619 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11963 (secure 0 0 0) 0x7f14400aae40 con 0x7f14501023a0 2024-05-10T10:45:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:45:39.066 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:45:29.579578+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24428,24445,24446,24451,24469]},"id":1}]} 2024-05-10T10:45:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14380b28c0 msgr2=0x7f14380b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14380b28c0 0x7f14380b4d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1450100d30 tx=0x7f1444009260 comp rx=0 tx=0).stop 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14501023a0 msgr2=0x7f145010fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14501023a0 0x7f145010fce0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1440008000 tx=0x7f14400047b0 comp rx=0 tx=0).stop 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 shutdown_connections 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1450110760 0x7f14501a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14380b28c0 0x7f14380b4d80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14501023a0 0x7f145010fce0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 --2- 172.21.15.112:0/999907619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1450102d60 0x7f1450110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 >> 172.21.15.112:0/999907619 conn(0x7f1450079350 msgr2=0x7f1450101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 shutdown_connections 2024-05-10T10:45:39.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:39.067+0000 7f14553f0700 1 -- 172.21.15.112:0/999907619 wait complete. 2024-05-10T10:45:39.070 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 37 2024-05-10T10:45:39.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:39 smithi112 ceph-mon[30100]: pgmap v592: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.2 MiB/s wr, 493 op/s 2024-05-10T10:45:39.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:39 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/999907619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:45:39.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:39 smithi148 ceph-mon[33568]: pgmap v592: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.2 MiB/s wr, 493 op/s 2024-05-10T10:45:39.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:39 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/999907619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:45:39.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:39 smithi195 ceph-mon[33773]: pgmap v592: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.2 MiB/s wr, 493 op/s 2024-05-10T10:45:39.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:39 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/999907619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:45:39.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:45:40.109 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:41.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:41 smithi112 ceph-mon[30100]: pgmap v593: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.8 MiB/s wr, 431 op/s 2024-05-10T10:45:41.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:41 smithi148 ceph-mon[33568]: pgmap v593: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.8 MiB/s wr, 431 op/s 2024-05-10T10:45:41.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:41 smithi195 ceph-mon[33773]: pgmap v593: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.8 MiB/s wr, 431 op/s 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 -- 172.21.15.112:0/2283583660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae94104160 msgr2=0x7fae94108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 --2- 172.21.15.112:0/2283583660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae94104160 0x7fae94108150 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fae840099c0 tx=0x7fae84009cd0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 -- 172.21.15.112:0/2283583660 shutdown_connections 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 --2- 172.21.15.112:0/2283583660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae94108880 0x7fae94110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 --2- 172.21.15.112:0/2283583660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae941037b0 0x7fae94103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 --2- 172.21.15.112:0/2283583660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae94104160 0x7fae94108150 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 -- 172.21.15.112:0/2283583660 >> 172.21.15.112:0/2283583660 conn(0x7fae9407a580 msgr2=0x7fae9407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 -- 172.21.15.112:0/2283583660 shutdown_connections 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.912+0000 7fae9bb15700 1 -- 172.21.15.112:0/2283583660 wait complete. 2024-05-10T10:45:41.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 Processor -- start 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 -- start start 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae941037b0 0x7fae940757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae94104160 0x7fae94075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94108880 0x7fae94076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae941134f0 con 0x7fae941037b0 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae94113370 con 0x7fae94108880 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9bb15700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae94113670 con 0x7fae94104160 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae998b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae941037b0 0x7fae940757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae998b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae941037b0 0x7fae940757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55182/0 (socket says 172.21.15.112:55182) 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae998b1700 1 -- 172.21.15.112:0/1873112214 learned_addr learned my addr 172.21.15.112:0/1873112214 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae990b0700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae94104160 0x7fae94075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.913+0000 7fae9a0b2700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94108880 0x7fae94076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae998b1700 1 -- 172.21.15.112:0/1873112214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae94104160 msgr2=0x7fae94075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae998b1700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae94104160 0x7fae94075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae998b1700 1 -- 172.21.15.112:0/1873112214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94108880 msgr2=0x7fae94076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae998b1700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94108880 0x7fae94076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae998b1700 1 -- 172.21.15.112:0/1873112214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae941a7ef0 con 0x7fae941037b0 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae998b1700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae941037b0 0x7fae940757a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fae9000e9a0 tx=0x7fae9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae8affd700 1 -- 172.21.15.112:0/1873112214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000ca10 con 0x7fae941037b0 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae941a81e0 con 0x7fae941037b0 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae8affd700 1 -- 172.21.15.112:0/1873112214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae9000cbb0 con 0x7fae941037b0 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae8affd700 1 -- 172.21.15.112:0/1873112214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90018680 con 0x7fae941037b0 2024-05-10T10:45:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.914+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941a8740 con 0x7fae941037b0 2024-05-10T10:45:41.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.916+0000 7fae8affd700 1 -- 172.21.15.112:0/1873112214 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fae90010430 con 0x7fae941037b0 2024-05-10T10:45:41.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.917+0000 7fae8affd700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fae800b2880 0x7fae800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.917+0000 7fae990b0700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fae800b2880 0x7fae800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.917+0000 7fae8affd700 1 -- 172.21.15.112:0/1873112214 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fae900e2740 con 0x7fae941037b0 2024-05-10T10:45:41.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.918+0000 7fae990b0700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fae800b2880 0x7fae800b4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fae94077230 tx=0x7fae84005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:41.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.918+0000 7faea63a8b80 1 Processor -- start 2024-05-10T10:45:41.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.918+0000 7faea63a8b80 1 -- start start 2024-05-10T10:45:41.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.918+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d756488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7faea63a8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557d7559a860 con 0x557d756484d0 2024-05-10T10:45:41.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7faea63a8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557d755397c0 con 0x557d7564cc80 2024-05-10T10:45:41.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7faea63a8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557d75596820 con 0x557d75648df0 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d756488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d756488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55186/0 (socket says 172.21.15.112:55186) 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae990b0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 -- 172.21.15.112:0/1132968621 learned_addr learned my addr 172.21.15.112:0/1132968621 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae9a0b2700 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 -- 172.21.15.112:0/1132968621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 msgr2=0x557d7564c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 -- 172.21.15.112:0/1132968621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 msgr2=0x557d7564d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 -- 172.21.15.112:0/1132968621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d754e78d0 con 0x557d756484d0 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae990b0700 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae998b1700 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d756488b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fae900a6c40 tx=0x7fae900b2240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb44f3076a4de3f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.919+0000 7fae7ffff700 1 -- 172.21.15.112:0/1132968621 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900aa0d0 con 0x557d756484d0 2024-05-10T10:45:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7fae7ffff700 1 -- 172.21.15.112:0/1132968621 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae900a99d0 con 0x557d756484d0 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7fae7ffff700 1 -- 172.21.15.112:0/1132968621 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900cc330 con 0x557d756484d0 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7faea63a8b80 1 -- 172.21.15.112:0/1132968621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 msgr2=0x557d756488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d756488b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fae900a6c40 tx=0x7fae900b2240 comp rx=0 tx=0).stop 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7faea63a8b80 1 -- 172.21.15.112:0/1132968621 shutdown_connections 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d756488b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1132968621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.920+0000 7faea63a8b80 1 -- 172.21.15.112:0/1132968621 >> 172.21.15.112:0/1132968621 conn(0x557d75574100 msgr2=0x557d75642c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.921+0000 7faea63a8b80 1 -- 172.21.15.112:0/1132968621 shutdown_connections 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.921+0000 7faea63a8b80 1 -- 172.21.15.112:0/1132968621 wait complete. 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.921+0000 7faea63a8b80 1 -- start start 2024-05-10T10:45:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.921+0000 7faea63a8b80 1 Processor -- start 2024-05-10T10:45:41.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d756484d0 0x557d75654110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d75648df0 0x557d756514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d7564cc80 0x557d756519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7faea63a8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557d75654e90 con 0x557d75648df0 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7faea63a8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557d75500c30 con 0x557d756484d0 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7faea63a8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557d75655010 con 0x557d7564cc80 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7faea63a8b80 10 client.0 _ll_register_callbacks cb 0x557d75532920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae990b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d75648df0 0x557d756514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d756484d0 0x557d75654110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae9a0b2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d7564cc80 0x557d756519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d756484d0 0x557d75654110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48306/0 (socket says 172.21.15.112:48306) 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae990b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d75648df0 0x557d756514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55198/0 (socket says 172.21.15.112:55198) 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 -- 172.21.15.112:0/1279704714 learned_addr learned my addr 172.21.15.112:0/1279704714 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 -- 172.21.15.112:0/1279704714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d7564cc80 msgr2=0x557d756519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d7564cc80 0x557d756519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 -- 172.21.15.112:0/1279704714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d75648df0 msgr2=0x557d756514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d75648df0 0x557d756514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 -- 172.21.15.112:0/1279704714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d75536690 con 0x557d756484d0 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae9a0b2700 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d7564cc80 0x557d756519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae998b1700 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d756484d0 0x557d75654110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae900a5900 tx=0x7fae900a95c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21bc873139d7db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.922+0000 7fae7d7fa700 1 -- 172.21.15.112:0/1279704714 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900c2070 con 0x557d756484d0 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7fae7d7fa700 1 -- 172.21.15.112:0/1279704714 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fae900b25b0 con 0x557d756484d0 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae900a99d0 con 0x557d756484d0 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7fae7d7fa700 1 -- 172.21.15.112:0/1279704714 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fae900aa0d0 con 0x557d756484d0 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 10 client.26078 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:41.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557d7565f740 con 0x557d756484d0 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7fae7d7fa700 1 -- 172.21.15.112:0/1279704714 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae900a99d0 con 0x557d756484d0 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7fae7d7fa700 1 -- 172.21.15.112:0/1279704714 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7fae900a8cc0 con 0x557d756484d0 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 10 client.26078 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 10 client.26078 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 20 client.26078 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d75661c10 0x557d75664000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 4 client.26078 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d75614650 con 0x557d75661c10 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.923+0000 7fae990b0700 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d75661c10 0x557d75664000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.924+0000 7fae990b0700 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d75661c10 0x557d75664000 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83833b6bab598204 server_cookie=d879734b391e64dd in_seq=0 out_seq=0 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.924+0000 7fae7d7fa700 10 client.26078 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.924+0000 7fae7d7fa700 1 -- 172.21.15.112:0/1279704714 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d75614650 con 0x557d75661c10 2024-05-10T10:45:41.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.924+0000 7fae7d7fa700 10 client.26078 handle_command_reply: tid=0 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.924+0000 7faea63a8b80 1 client.26078 shutdown 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.924+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d756484d0 msgr2=0x557d75654110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.924+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d756484d0 0x557d75654110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae900a5900 tx=0x7fae900a95c0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 shutdown_connections 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d75661c10 0x557d75664000 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d7564cc80 0x557d756519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7fae7d7fa700 0 client.26078 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d756484d0 0x557d75654110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 --2- 172.21.15.112:0/1279704714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d75648df0 0x557d756514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 >> 172.21.15.112:0/1279704714 conn(0x557d75574100 msgr2=0x557d7563d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 shutdown_connections 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 1 -- 172.21.15.112:0/1279704714 wait complete. 2024-05-10T10:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.925+0000 7faea63a8b80 20 client.26078 trim_cache size 0 max 16384 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.940+0000 7faea63a8b80 1 Processor -- start 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.940+0000 7faea63a8b80 1 -- start start 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.940+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75650410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7faea63a8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae941b13b0 con 0x557d756484d0 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7faea63a8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae941b3010 con 0x557d7564cc80 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7faea63a8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae941b1a80 con 0x557d75648df0 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75650410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75650410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55202/0 (socket says 172.21.15.112:55202) 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 -- 172.21.15.112:0/2106396974 learned_addr learned my addr 172.21.15.112:0/2106396974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae990b0700 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae9a0b2700 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 -- 172.21.15.112:0/2106396974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 msgr2=0x557d7564a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 -- 172.21.15.112:0/2106396974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 msgr2=0x557d7564ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 -- 172.21.15.112:0/2106396974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d75651d70 con 0x557d756484d0 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae998b1700 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75650410 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fae900a6c40 tx=0x7fae90009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1cd218bfdaa670 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae7f7fe700 1 -- 172.21.15.112:0/2106396974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900c2050 con 0x557d756484d0 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae7f7fe700 1 -- 172.21.15.112:0/2106396974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae900a99d0 con 0x557d756484d0 2024-05-10T10:45:41.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.941+0000 7fae7f7fe700 1 -- 172.21.15.112:0/2106396974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9006f260 con 0x557d756484d0 2024-05-10T10:45:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.942+0000 7faea63a8b80 1 -- 172.21.15.112:0/2106396974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 msgr2=0x557d75650410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.942+0000 7faea63a8b80 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75650410 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fae900a6c40 tx=0x7fae90009790 comp rx=0 tx=0).stop 2024-05-10T10:45:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.942+0000 7faea63a8b80 1 -- 172.21.15.112:0/2106396974 shutdown_connections 2024-05-10T10:45:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.942+0000 7faea63a8b80 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d7564ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.942+0000 7faea63a8b80 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75650410 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.942+0000 7faea63a8b80 1 --2- 172.21.15.112:0/2106396974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d7564a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.942+0000 7faea63a8b80 1 -- 172.21.15.112:0/2106396974 >> 172.21.15.112:0/2106396974 conn(0x557d75574100 msgr2=0x557d7563d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.943+0000 7faea63a8b80 1 -- 172.21.15.112:0/2106396974 shutdown_connections 2024-05-10T10:45:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.943+0000 7faea63a8b80 1 -- 172.21.15.112:0/2106396974 wait complete. 2024-05-10T10:45:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.944+0000 7faea63a8b80 1 -- start start 2024-05-10T10:45:41.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.944+0000 7faea63a8b80 1 Processor -- start 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75664440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d75664980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7faea63a8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d756c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7faea63a8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae941b28e0 con 0x557d756484d0 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7faea63a8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae941b2210 con 0x557d7564cc80 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7faea63a8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae941af350 con 0x557d75648df0 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7faea63a8b80 10 client.0 _ll_register_callbacks cb 0x557d75532920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d756c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d756c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:48316/0 (socket says 172.21.15.112:48316) 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 -- 172.21.15.112:0/3647935596 learned_addr learned my addr 172.21.15.112:0/3647935596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae990b0700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d75664980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae998b1700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75664440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 -- 172.21.15.112:0/3647935596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 msgr2=0x557d75664980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d75664980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 -- 172.21.15.112:0/3647935596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 msgr2=0x557d75664440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75664440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae9a0b2700 1 -- 172.21.15.112:0/3647935596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d756c4730 con 0x557d7564cc80 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.945+0000 7fae990b0700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d75664980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.946+0000 7fae9a0b2700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d756c40f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae8c012f10 tx=0x7fae8c0077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6f91b385d1635e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.946+0000 7fae7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.946+0000 7fae7d7fa700 1 -- 172.21.15.112:0/3647935596 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c022da0 con 0x557d7564cc80 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.946+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae900a99d0 con 0x557d7564cc80 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.946+0000 7fae7d7fa700 1 -- 172.21.15.112:0/3647935596 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fae8c02b420 con 0x557d7564cc80 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7fae7d7fa700 1 -- 172.21.15.112:0/3647935596 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fae8c033050 con 0x557d7564cc80 2024-05-10T10:45:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 10 client.26084 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557d756cbb20 con 0x557d7564cc80 2024-05-10T10:45:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7fae7d7fa700 1 -- 172.21.15.112:0/3647935596 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae900a99d0 con 0x557d7564cc80 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7fae7d7fa700 1 -- 172.21.15.112:0/3647935596 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7fae8c016690 con 0x557d7564cc80 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 10 client.26084 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 20 client.26084 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d756cc2b0 0x557d756ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 4 client.26084 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d755a8240 con 0x557d756cc2b0 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.947+0000 7fae998b1700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d756cc2b0 0x557d756ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7fae7d7fa700 10 client.26084 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7fae998b1700 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d756cc2b0 0x557d756ce6d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3eeaef6c686e7eb3 server_cookie=5a8a0dac50e89cd5 in_seq=0 out_seq=0 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7fae7d7fa700 1 -- 172.21.15.112:0/3647935596 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d755a8240 con 0x557d756cc2b0 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7fae7d7fa700 10 client.26084 handle_command_reply: tid=0 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 client.26084 shutdown 2024-05-10T10:45:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 msgr2=0x557d756c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d756c40f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae8c012f10 tx=0x7fae8c0077e0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 shutdown_connections 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7564cc80 0x557d756c40f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d756484d0 0x557d75664440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x557d756cc2b0 0x557d756ce6d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 --2- 172.21.15.112:0/3647935596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d75648df0 0x557d75664980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 >> 172.21.15.112:0/3647935596 conn(0x557d75574100 msgr2=0x557d756472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.948+0000 7fae7d7fa700 0 client.26084 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 shutdown_connections 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7faea63a8b80 1 -- 172.21.15.112:0/3647935596 wait complete. 2024-05-10T10:45:41.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7faea63a8b80 20 client.26084 trim_cache size 0 max 16384 2024-05-10T10:45:41.954 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fae800b2880 msgr2=0x7fae800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fae800b2880 0x7fae800b4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fae94077230 tx=0x7fae84005fb0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae941037b0 msgr2=0x7fae940757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae941037b0 0x7fae940757a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fae9000e9a0 tx=0x7fae9000ecb0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 shutdown_connections 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94108880 0x7fae94076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae941037b0 0x7fae940757a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fae800b2880 0x7fae800b4d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 --2- 172.21.15.112:0/1873112214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae94104160 0x7fae94075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.949+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 >> 172.21.15.112:0/1873112214 conn(0x7fae9407a580 msgr2=0x7fae94101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.950+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 shutdown_connections 2024-05-10T10:45:41.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:41.950+0000 7fae9bb15700 1 -- 172.21.15.112:0/1873112214 wait complete. 2024-05-10T10:45:42.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:45:43.027 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:43.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:43 smithi112 ceph-mon[30100]: pgmap v594: 129 pgs: 129 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 634 op/s 2024-05-10T10:45:43.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:43 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:45:43.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:43 smithi148 ceph-mon[33568]: pgmap v594: 129 pgs: 129 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 634 op/s 2024-05-10T10:45:43.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:43 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:45:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:43 smithi195 ceph-mon[33773]: pgmap v594: 129 pgs: 129 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 634 op/s 2024-05-10T10:45:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:43 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:45:44.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:45:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:45:44.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:45:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:45:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:45:44.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:45:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T10:45:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:45:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:45:44.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:45:45.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/3483558310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18102540 msgr2=0x7f1e1810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/3483558310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18102540 0x7f1e1810e9b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b0d0 tx=0x7f1e1000b3e0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/3483558310 shutdown_connections 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/3483558310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18102540 0x7f1e1810e9b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/3483558310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18107be0 0x7f1e18107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/3483558310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e18101b80 0x7f1e18102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/3483558310 >> 172.21.15.112:0/3483558310 conn(0x7f1e18078a10 msgr2=0x7f1e180ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/3483558310 shutdown_connections 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.163+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/3483558310 wait complete. 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 Processor -- start 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 -- start start 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18101b80 0x7f1e1819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e18102540 0x7f1e1819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107be0 0x7f1e181a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18112470 con 0x7f1e18107be0 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181122f0 con 0x7f1e18102540 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1f3cb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181125f0 con 0x7f1e18101b80 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18101b80 0x7f1e1819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d968700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107be0 0x7f1e181a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18101b80 0x7f1e1819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:42938/0 (socket says 172.21.15.112:42938) 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d968700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107be0 0x7f1e181a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55226/0 (socket says 172.21.15.112:55226) 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d167700 1 -- 172.21.15.112:0/312397834 learned_addr learned my addr 172.21.15.112:0/312397834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1c966700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e18102540 0x7f1e1819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d968700 1 -- 172.21.15.112:0/312397834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18101b80 msgr2=0x7f1e1819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d968700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18101b80 0x7f1e1819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d968700 1 -- 172.21.15.112:0/312397834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e18102540 msgr2=0x7f1e1819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d968700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e18102540 0x7f1e1819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d968700 1 -- 172.21.15.112:0/312397834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e181a5d10 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.164+0000 7f1e1d167700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18101b80 0x7f1e1819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.165+0000 7f1e1d968700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107be0 0x7f1e181a57d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b0d0 tx=0x7f1e10007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.165+0000 7f1e0e7fc700 1 -- 172.21.15.112:0/312397834 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1001f070 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.165+0000 7f1e0e7fc700 1 -- 172.21.15.112:0/312397834 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e10004c50 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.165+0000 7f1e0e7fc700 1 -- 172.21.15.112:0/312397834 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1001b430 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.165+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e181a5ee0 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.165+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e181a6330 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.167+0000 7f1e0e7fc700 1 -- 172.21.15.112:0/312397834 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1e10022070 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.168+0000 7f1e0e7fc700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e040b2880 0x7f1e040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.169+0000 7f1e1d167700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e040b2880 0x7f1e040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.169+0000 7f1e0e7fc700 1 -- 172.21.15.112:0/312397834 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1e1000f020 con 0x7f1e18107be0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.169+0000 7f1e1d167700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e040b2880 0x7f1e040b4d40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1e14011550 tx=0x7f1e14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.169+0000 7f1e29c5eb80 1 Processor -- start 2024-05-10T10:45:45.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.169+0000 7f1e29c5eb80 1 -- start start 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe01b0 0x5605f3fe0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe0ad0 0x5605f3fda810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fdb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605f3f2c940 con 0x5605f3fdad50 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5605f3e91da0 con 0x5605f3fe01b0 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5605f3f28900 con 0x5605f3fe0ad0 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe0ad0 0x5605f3fda810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1d968700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fdb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe0ad0 0x5605f3fda810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:42944/0 (socket says 172.21.15.112:42944) 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 -- 172.21.15.112:0/164948832 learned_addr learned my addr 172.21.15.112:0/164948832 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1d167700 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe01b0 0x5605f3fe0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 -- 172.21.15.112:0/164948832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe01b0 msgr2=0x5605f3fe0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe01b0 0x5605f3fe0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 -- 172.21.15.112:0/164948832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 msgr2=0x5605f3fdb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fdb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 -- 172.21.15.112:0/164948832 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f3e798d0 con 0x5605f3fe0ad0 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1d968700 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fdb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e1c966700 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe0ad0 0x5605f3fda810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e08007c60 tx=0x7f1e08007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1af1bb297fa0d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.170+0000 7f1e037fe700 1 -- 172.21.15.112:0/164948832 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08010ca0 con 0x5605f3fe0ad0 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.171+0000 7f1e037fe700 1 -- 172.21.15.112:0/164948832 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e08015070 con 0x5605f3fe0ad0 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.171+0000 7f1e037fe700 1 -- 172.21.15.112:0/164948832 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08003c40 con 0x5605f3fe0ad0 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/164948832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe0ad0 msgr2=0x5605f3fda810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe0ad0 0x5605f3fda810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e08007c60 tx=0x7f1e08007f70 comp rx=0 tx=0).stop 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/164948832 shutdown_connections 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fdb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe01b0 0x5605f3fe0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/164948832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe0ad0 0x5605f3fda810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/164948832 >> 172.21.15.112:0/164948832 conn(0x5605f3f06b70 msgr2=0x5605f3fd52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/164948832 shutdown_connections 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/164948832 wait complete. 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 -- start start 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.172+0000 7f1e29c5eb80 1 Processor -- start 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fe66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fe3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.293 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:45:45.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fe66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fe66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:42960/0 (socket says 172.21.15.112:42960) 2024-05-10T10:45:45.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e1c966700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fe3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e1c966700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fe3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55252/0 (socket says 172.21.15.112:55252) 2024-05-10T10:45:45.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e1d167700 1 -- 172.21.15.112:0/1702957084 learned_addr learned my addr 172.21.15.112:0/1702957084 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:45.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605f3fe7450 con 0x5605f3fe01b0 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5605f3ee5ed0 con 0x5605f3fe0ad0 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5605f3fe75d0 con 0x5605f3fdad50 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e29c5eb80 10 client.0 _ll_register_callbacks cb 0x5605f3ec05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.173+0000 7f1e1d968700 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e1d167700 1 -- 172.21.15.112:0/1702957084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 msgr2=0x5605f3fe3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e1d167700 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e1d167700 1 -- 172.21.15.112:0/1702957084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 msgr2=0x5605f3fe3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e1d167700 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fe3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e1d167700 1 -- 172.21.15.112:0/1702957084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f3ee3760 con 0x5605f3fdad50 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e1d968700 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e1d167700 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fe66b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400eed0 tx=0x7f1e140047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1b6dad5f0fe74c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:45.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e00ff9700 1 -- 172.21.15.112:0/1702957084 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1401a5b0 con 0x5605f3fdad50 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e08015070 con 0x5605f3fdad50 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e00ff9700 1 -- 172.21.15.112:0/1702957084 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1e1401a750 con 0x5605f3fdad50 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e00ff9700 1 -- 172.21.15.112:0/1702957084 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1e1401a950 con 0x5605f3fdad50 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e29c5eb80 10 client.26122 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.174+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5605f3ff1d50 con 0x5605f3fdad50 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e00ff9700 1 -- 172.21.15.112:0/1702957084 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e08015070 con 0x5605f3fdad50 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e00ff9700 1 -- 172.21.15.112:0/1702957084 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f1e14026070 con 0x5605f3fdad50 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e29c5eb80 10 client.26122 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e29c5eb80 10 client.26122 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e29c5eb80 20 client.26122 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f3ff42e0 0x5605f3ff6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e29c5eb80 4 client.26122 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f3f3a320 con 0x5605f3ff42e0 2024-05-10T10:45:45.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.175+0000 7f1e1c966700 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f3ff42e0 0x5605f3ff6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.176+0000 7f1e00ff9700 10 client.26122 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.176+0000 7f1e1c966700 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f3ff42e0 0x5605f3ff6710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c96e8421c7aa0b5 server_cookie=151806cf646e7dc5 in_seq=0 out_seq=0 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.176+0000 7f1e00ff9700 1 -- 172.21.15.112:0/1702957084 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5605f3f3a320 con 0x5605f3ff42e0 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.176+0000 7f1e00ff9700 10 client.26122 handle_command_reply: tid=0 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.176+0000 7f1e29c5eb80 1 client.26122 shutdown 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 msgr2=0x5605f3fe66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fe66b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400eed0 tx=0x7f1e140047c0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 shutdown_connections 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f3ff42e0 0x5605f3ff6710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fe66b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1702957084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fe3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 >> 172.21.15.112:0/1702957084 conn(0x5605f3f06b70 msgr2=0x5605f3fcfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e00ff9700 0 client.26122 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 shutdown_connections 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1702957084 wait complete. 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.177+0000 7f1e29c5eb80 20 client.26122 trim_cache size 0 max 16384 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 Processor -- start 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 -- start start 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fe4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe01b0 0x5605f3ff0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181acdf0 con 0x5605f3fdad50 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181aea50 con 0x5605f3fe0ad0 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181ad4c0 con 0x5605f3fe01b0 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fe4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fe4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55264/0 (socket says 172.21.15.112:55264) 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 -- 172.21.15.112:0/1827287420 learned_addr learned my addr 172.21.15.112:0/1827287420 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:45.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1c966700 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe01b0 0x5605f3ff0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d968700 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 -- 172.21.15.112:0/1827287420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe01b0 msgr2=0x5605f3ff0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe01b0 0x5605f3ff0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 -- 172.21.15.112:0/1827287420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 msgr2=0x5605f3fe15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.195+0000 7f1e1d167700 1 -- 172.21.15.112:0/1827287420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f3fe1fc0 con 0x5605f3fdad50 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.196+0000 7f1e1d167700 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fe4670 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1e14000c00 tx=0x7f1e14028f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab30778d49684cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.196+0000 7f1e02ffd700 1 -- 172.21.15.112:0/1827287420 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1403b070 con 0x5605f3fdad50 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.196+0000 7f1e02ffd700 1 -- 172.21.15.112:0/1827287420 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1401faa0 con 0x5605f3fdad50 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.196+0000 7f1e02ffd700 1 -- 172.21.15.112:0/1827287420 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14028920 con 0x5605f3fdad50 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.196+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1827287420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 msgr2=0x5605f3fe4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.196+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fe4670 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1e14000c00 tx=0x7f1e14028f50 comp rx=0 tx=0).stop 2024-05-10T10:45:45.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1827287420 shutdown_connections 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3fe15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fdad50 0x5605f3fe4670 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/1827287420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fe01b0 0x5605f3ff0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1827287420 >> 172.21.15.112:0/1827287420 conn(0x5605f3f06b70 msgr2=0x5605f3fe0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1827287420 shutdown_connections 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/1827287420 wait complete. 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 -- start start 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.197+0000 7f1e29c5eb80 1 Processor -- start 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fecdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3ffafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181ae320 con 0x5605f3fe01b0 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181adc50 con 0x5605f3fe0ad0 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181aad90 con 0x5605f3fdad50 2024-05-10T10:45:45.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 10 client.0 _ll_register_callbacks cb 0x5605f3ec05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1c966700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fecdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d968700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3ffafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fec890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:42966/0 (socket says 172.21.15.112:42966) 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 -- 172.21.15.112:0/4115663230 learned_addr learned my addr 172.21.15.112:0/4115663230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 -- 172.21.15.112:0/4115663230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 msgr2=0x5605f3ffafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3ffafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 -- 172.21.15.112:0/4115663230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 msgr2=0x5605f3fecdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fecdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 -- 172.21.15.112:0/4115663230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f3ffb610 con 0x5605f3fdad50 2024-05-10T10:45:45.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d968700 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3ffafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e1d167700 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fec890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e14029ff0 tx=0x7f1e1400e450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51606419d941a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e00ff9700 1 -- 172.21.15.112:0/4115663230 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14043060 con 0x5605f3fdad50 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e00ff9700 1 -- 172.21.15.112:0/4115663230 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1e140285c0 con 0x5605f3fdad50 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.198+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e1401faa0 con 0x5605f3fdad50 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e00ff9700 1 -- 172.21.15.112:0/4115663230 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1e1404a050 con 0x5605f3fdad50 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 10 client.26128 fetch_fsmap learned FSMap version 37 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5605f405e470 con 0x5605f3fdad50 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e00ff9700 1 -- 172.21.15.112:0/4115663230 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1401faa0 con 0x5605f3fdad50 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e00ff9700 1 -- 172.21.15.112:0/4115663230 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f1e14026050 con 0x5605f3fdad50 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 10 client.26128 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T10:45:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 10 client.26128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 20 client.26128 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f405ea60 0x5605f4060e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 4 client.26128 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605f3f343c0 con 0x5605f405ea60 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.199+0000 7f1e1c966700 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f405ea60 0x5605f4060e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e00ff9700 10 client.26128 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e1c966700 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f405ea60 0x5605f4060e50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a5ec900c041a7edb server_cookie=3491d0404fb7ed5e in_seq=0 out_seq=0 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e00ff9700 1 -- 172.21.15.112:0/4115663230 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1e08015020 con 0x5605f405ea60 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e00ff9700 10 client.26128 handle_command_reply: tid=0 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 client.26128 shutdown 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 msgr2=0x5605f3fec890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fec890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e14029ff0 tx=0x7f1e1400e450 comp rx=0 tx=0).stop 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 shutdown_connections 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5605f405ea60 0x5605f4060e50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605f3fe0ad0 0x5605f3ffafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e00ff9700 0 client.26128 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605f3fdad50 0x5605f3fec890 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 --2- 172.21.15.112:0/4115663230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605f3fe01b0 0x5605f3fecdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.200+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 >> 172.21.15.112:0/4115663230 conn(0x5605f3f06b70 msgr2=0x5605f3fdebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 shutdown_connections 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e29c5eb80 1 -- 172.21.15.112:0/4115663230 wait complete. 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e29c5eb80 20 client.26128 trim_cache size 0 max 16384 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e040b2880 msgr2=0x7f1e040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e040b2880 0x7f1e040b4d40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1e14011550 tx=0x7f1e14009260 comp rx=0 tx=0).stop 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107be0 msgr2=0x7f1e181a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107be0 0x7f1e181a57d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b0d0 tx=0x7f1e10007a00 comp rx=0 tx=0).stop 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 shutdown_connections 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107be0 0x7f1e181a57d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f1e040b2880 0x7f1e040b4d40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e18101b80 0x7f1e1819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 --2- 172.21.15.112:0/312397834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e18102540 0x7f1e1819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.201+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 >> 172.21.15.112:0/312397834 conn(0x7f1e18078a10 msgr2=0x7f1e18100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.202+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 shutdown_connections 2024-05-10T10:45:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:45.202+0000 7f1e1f3cb700 1 -- 172.21.15.112:0/312397834 wait complete. 2024-05-10T10:45:45.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:45 smithi148 ceph-mon[33568]: pgmap v595: 129 pgs: 129 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 425 op/s 2024-05-10T10:45:45.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:45 smithi148 ceph-mon[33568]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-10T10:45:45.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:45 smithi148 ceph-mon[33568]: Unable to set osd_memory_target on smithi195 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-10T10:45:45.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:45 smithi195 ceph-mon[33773]: pgmap v595: 129 pgs: 129 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 425 op/s 2024-05-10T10:45:45.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:45 smithi195 ceph-mon[33773]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-10T10:45:45.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:45 smithi195 ceph-mon[33773]: Unable to set osd_memory_target on smithi195 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-10T10:45:45.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:45 smithi112 ceph-mon[30100]: pgmap v595: 129 pgs: 129 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 425 op/s 2024-05-10T10:45:45.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:45 smithi112 ceph-mon[30100]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-10T10:45:45.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:45 smithi112 ceph-mon[30100]: Unable to set osd_memory_target on smithi195 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-10T10:45:46.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:45:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:46.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:46 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:46.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:46 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:46.565 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:46 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:47 smithi148 ceph-mon[33568]: pgmap v596: 129 pgs: 129 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 595 op/s 2024-05-10T10:45:47.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:47 smithi195 ceph-mon[33773]: pgmap v596: 129 pgs: 129 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 595 op/s 2024-05-10T10:45:47.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:47 smithi112 ceph-mon[30100]: pgmap v596: 129 pgs: 129 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 595 op/s 2024-05-10T10:45:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 -- 172.21.15.112:0/2347287603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c108410 msgr2=0x7f293c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 --2- 172.21.15.112:0/2347287603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c108410 0x7f293c1087f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f292c009990 tx=0x7f292c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 -- 172.21.15.112:0/2347287603 shutdown_connections 2024-05-10T10:45:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 --2- 172.21.15.112:0/2347287603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c102d60 0x7f293c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 --2- 172.21.15.112:0/2347287603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c108410 0x7f293c1087f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 --2- 172.21.15.112:0/2347287603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f293c1023a0 0x7f293c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 -- 172.21.15.112:0/2347287603 >> 172.21.15.112:0/2347287603 conn(0x7f293c079350 msgr2=0x7f293c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 -- 172.21.15.112:0/2347287603 shutdown_connections 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.174+0000 7f29425ef700 1 -- 172.21.15.112:0/2347287603 wait complete. 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 Processor -- start 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 -- start start 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c1023a0 0x7f293c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c102d60 0x7f293c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f293c108410 0x7f293c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f293c1123e0 con 0x7f293c102d60 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f293c112260 con 0x7f293c108410 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.175+0000 7f29425ef700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f293c112560 con 0x7f293c1023a0 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c1023a0 0x7f293c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f2933fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c102d60 0x7f293c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c1023a0 0x7f293c104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:51660/0 (socket says 172.21.15.112:51660) 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f2933fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c102d60 0x7f293c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48002/0 (socket says 172.21.15.112:48002) 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 -- 172.21.15.112:0/4232148530 learned_addr learned my addr 172.21.15.112:0/4232148530 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f2940b8c700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f293c108410 0x7f293c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 -- 172.21.15.112:0/4232148530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f293c108410 msgr2=0x7f293c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f293c108410 0x7f293c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 -- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c102d60 msgr2=0x7f293c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c102d60 0x7f293c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 -- 172.21.15.112:0/4232148530 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c1a8030 con 0x7f293c1023a0 2024-05-10T10:45:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f2940b8c700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f293c108410 0x7f293c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:45:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f293bfff700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c1023a0 0x7f293c104080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f292c006010 tx=0x7f292c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f2939ffb700 1 -- 172.21.15.112:0/4232148530 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c027070 con 0x7f293c1023a0 2024-05-10T10:45:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c1a82c0 con 0x7f293c1023a0 2024-05-10T10:45:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f2939ffb700 1 -- 172.21.15.112:0/4232148530 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f292c022470 con 0x7f293c1023a0 2024-05-10T10:45:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f2939ffb700 1 -- 172.21.15.112:0/4232148530 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c017650 con 0x7f293c1023a0 2024-05-10T10:45:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.176+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c1a8770 con 0x7f293c1023a0 2024-05-10T10:45:48.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.180+0000 7f2939ffb700 1 -- 172.21.15.112:0/4232148530 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f292c009d10 con 0x7f293c1023a0 2024-05-10T10:45:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.180+0000 7f2939ffb700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f29280b2880 0x7f29280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 Processor -- start 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f2933fff700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f29280b2880 0x7f29280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f2939ffb700 1 -- 172.21.15.112:0/4232148530 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f292c0e3060 con 0x7f293c1023a0 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 -- start start 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430a5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f642ff6940 con 0x55f6430aa1b0 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f642f5bda0 con 0x55f6430aaad0 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f294ce82b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f642ff2900 con 0x55f6430a4d50 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f2933fff700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f29280b2880 0x7f29280b4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f293c079900 tx=0x7f2924009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f293bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f2933fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f293bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48012/0 (socket says 172.21.15.112:48012) 2024-05-10T10:45:48.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f293bfff700 1 -- 172.21.15.112:0/2699728650 learned_addr learned my addr 172.21.15.112:0/2699728650 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:48.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f2933fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56820/0 (socket says 172.21.15.112:56820) 2024-05-10T10:45:48.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.181+0000 7f2940b8c700 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f2933fff700 1 -- 172.21.15.112:0/2699728650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 msgr2=0x55f6430a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f2933fff700 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f2933fff700 1 -- 172.21.15.112:0/2699728650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 msgr2=0x55f6430aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f2933fff700 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f2933fff700 1 -- 172.21.15.112:0/2699728650 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f642f438d0 con 0x55f6430aaad0 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f2940b8c700 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f2933fff700 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430a4810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2924011c40 tx=0x7f2924014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97acd9fccb85e142 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f29327fc700 1 -- 172.21.15.112:0/2699728650 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292400eb00 con 0x55f6430aaad0 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f29327fc700 1 -- 172.21.15.112:0/2699728650 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2924004cc0 con 0x55f6430aaad0 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.182+0000 7f29327fc700 1 -- 172.21.15.112:0/2699728650 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924005510 con 0x55f6430aaad0 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.183+0000 7f294ce82b80 1 -- 172.21.15.112:0/2699728650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 msgr2=0x55f6430a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.183+0000 7f294ce82b80 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430a4810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2924011c40 tx=0x7f2924014000 comp rx=0 tx=0).stop 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 -- 172.21.15.112:0/2699728650 shutdown_connections 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 --2- 172.21.15.112:0/2699728650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430a4810 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 -- 172.21.15.112:0/2699728650 >> 172.21.15.112:0/2699728650 conn(0x55f642fd0b70 msgr2=0x55f64309f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 -- 172.21.15.112:0/2699728650 shutdown_connections 2024-05-10T10:45:48.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 -- 172.21.15.112:0/2699728650 wait complete. 2024-05-10T10:45:48.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 -- start start 2024-05-10T10:45:48.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.184+0000 7f294ce82b80 1 Processor -- start 2024-05-10T10:45:48.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f294ce82b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f6430b1450 con 0x55f6430aa1b0 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f294ce82b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f642fafed0 con 0x55f6430aaad0 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f294ce82b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f6430b15d0 con 0x55f6430a4d50 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f294ce82b80 10 client.0 _ll_register_callbacks cb 0x55f642f8a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f293bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f293bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:51670/0 (socket says 172.21.15.112:51670) 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f293bfff700 1 -- 172.21.15.112:0/3344599767 learned_addr learned my addr 172.21.15.112:0/3344599767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.185+0000 7f2940b8c700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.186+0000 7f2933fff700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.186+0000 7f293bfff700 1 -- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 msgr2=0x55f6430ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.186+0000 7f293bfff700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.186+0000 7f293bfff700 1 -- 172.21.15.112:0/3344599767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 msgr2=0x55f6430adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.186+0000 7f293bfff700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.187+0000 7f293bfff700 1 -- 172.21.15.112:0/3344599767 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f642fad760 con 0x55f6430a4d50 2024-05-10T10:45:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.187+0000 7f2933fff700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.187+0000 7f293bfff700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430b07a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f292c0e0820 tx=0x7f292c022c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8945428a9f1473e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.187+0000 7f291b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.187+0000 7f291b7fe700 1 -- 172.21.15.112:0/3344599767 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0aa380 con 0x55f6430a4d50 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.187+0000 7f291b7fe700 1 -- 172.21.15.112:0/3344599767 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f292c0b2ed0 con 0x55f6430a4d50 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.187+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2924004cc0 con 0x55f6430a4d50 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.188+0000 7f291b7fe700 1 -- 172.21.15.112:0/3344599767 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f292c0ae730 con 0x55f6430a4d50 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.188+0000 7f294ce82b80 10 client.26140 fetch_fsmap learned FSMap version 38 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.188+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f6430b1450 con 0x55f6430a4d50 2024-05-10T10:45:48.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.188+0000 7f291b7fe700 1 -- 172.21.15.112:0/3344599767 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f292c0b3150 con 0x55f6430a4d50 2024-05-10T10:45:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.188+0000 7f291b7fe700 1 -- 172.21.15.112:0/3344599767 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f292c0a5cf0 con 0x55f6430a4d50 2024-05-10T10:45:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.188+0000 7f294ce82b80 10 client.26140 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.189+0000 7f294ce82b80 10 client.26140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.189+0000 7f294ce82b80 20 client.26140 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.189+0000 7f294ce82b80 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f6430be3d0 0x55f6430c07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.189+0000 7f294ce82b80 4 client.26140 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.189+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2924004cc0 con 0x55f6430be3d0 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.189+0000 7f2933fff700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f6430be3d0 0x55f6430c07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.190+0000 7f291b7fe700 10 client.26140 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.190+0000 7f2933fff700 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f6430be3d0 0x55f6430c07f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3be7ac6c5e341d9c server_cookie=2979c4fdcd01579e in_seq=0 out_seq=0 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f291b7fe700 1 -- 172.21.15.112:0/3344599767 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2924004cc0 con 0x55f6430be3d0 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f291b7fe700 10 client.26140 handle_command_reply: tid=0 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 client.26140 shutdown 2024-05-10T10:45:48.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 msgr2=0x55f6430b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430b07a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f292c0e0820 tx=0x7f292c022c00 comp rx=0 tx=0).stop 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 shutdown_connections 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430aaad0 0x55f6430ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430a4d50 0x55f6430b07a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f6430be3d0 0x55f6430c07f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 --2- 172.21.15.112:0/3344599767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aa1b0 0x55f6430adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 >> 172.21.15.112:0/3344599767 conn(0x55f642fd0b70 msgr2=0x55f643099b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f291b7fe700 0 client.26140 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 shutdown_connections 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.191+0000 7f294ce82b80 1 -- 172.21.15.112:0/3344599767 wait complete. 2024-05-10T10:45:48.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.192+0000 7f294ce82b80 20 client.26140 trim_cache size 0 max 16384 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 Processor -- start 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 -- start start 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f643099c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f293c1b1380 con 0x55f6430aaad0 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f293c1b2fb0 con 0x55f6430a4d50 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f294ce82b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f293c1b1a50 con 0x55f6430aa1b0 2024-05-10T10:45:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f643099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f643099c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48036/0 (socket says 172.21.15.112:48036) 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 -- 172.21.15.112:0/527871922 learned_addr learned my addr 172.21.15.112:0/527871922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2933fff700 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f293bfff700 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 -- 172.21.15.112:0/527871922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 msgr2=0x55f6430ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 -- 172.21.15.112:0/527871922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 msgr2=0x55f6430b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 -- 172.21.15.112:0/527871922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64309a180 con 0x55f6430aaad0 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.209+0000 7f2940b8c700 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f643099c40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f29340076c0 tx=0x7f293400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670dc3f8d5448148 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.210+0000 7f2931ffb700 1 -- 172.21.15.112:0/527871922 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934003a00 con 0x55f6430aaad0 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.210+0000 7f2931ffb700 1 -- 172.21.15.112:0/527871922 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2934003ba0 con 0x55f6430aaad0 2024-05-10T10:45:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.210+0000 7f2931ffb700 1 -- 172.21.15.112:0/527871922 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934017670 con 0x55f6430aaad0 2024-05-10T10:45:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.210+0000 7f294ce82b80 1 -- 172.21.15.112:0/527871922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 msgr2=0x55f643099c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.210+0000 7f294ce82b80 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f643099c40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f29340076c0 tx=0x7f293400b040 comp rx=0 tx=0).stop 2024-05-10T10:45:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 -- 172.21.15.112:0/527871922 shutdown_connections 2024-05-10T10:45:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f643099c40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 --2- 172.21.15.112:0/527871922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 -- 172.21.15.112:0/527871922 >> 172.21.15.112:0/527871922 conn(0x55f642fd0b70 msgr2=0x55f6430aaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 -- 172.21.15.112:0/527871922 shutdown_connections 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 -- 172.21.15.112:0/527871922 wait complete. 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 -- start start 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 Processor -- start 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.211+0000 7f294ce82b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f6430c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f294ce82b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f293c1b28b0 con 0x55f6430aaad0 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f294ce82b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f293c1b21e0 con 0x55f6430a4d50 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f294ce82b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f293c1af320 con 0x55f6430aa1b0 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2940b8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f6430c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f293bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2933fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2940b8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f6430c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48050/0 (socket says 172.21.15.112:48050) 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2940b8c700 1 -- 172.21.15.112:0/358392511 learned_addr learned my addr 172.21.15.112:0/358392511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f294ce82b80 10 client.0 _ll_register_callbacks cb 0x55f642f8a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2933fff700 1 -- 172.21.15.112:0/358392511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 msgr2=0x55f6430b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2933fff700 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2933fff700 1 -- 172.21.15.112:0/358392511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 msgr2=0x55f6430c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2933fff700 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f6430c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2933fff700 1 -- 172.21.15.112:0/358392511 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6430c6ae0 con 0x55f6430aa1b0 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f2933fff700 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430b6dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55f6430aaeb0 tx=0x7f2924005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7960e8aaad14872e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f291b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f291b7fe700 1 -- 172.21.15.112:0/358392511 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924036070 con 0x55f6430aa1b0 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2934003ba0 con 0x55f6430aa1b0 2024-05-10T10:45:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.212+0000 7f291b7fe700 1 -- 172.21.15.112:0/358392511 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f292400e980 con 0x55f6430aa1b0 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f291b7fe700 1 -- 172.21.15.112:0/358392511 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2924042050 con 0x55f6430aa1b0 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 10 client.26146 fetch_fsmap learned FSMap version 38 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f6431281d0 con 0x55f6430aa1b0 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f291b7fe700 1 -- 172.21.15.112:0/358392511 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2934003ba0 con 0x55f6430aa1b0 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f291b7fe700 1 -- 172.21.15.112:0/358392511 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f29240324c0 con 0x55f6430aa1b0 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 10 client.26146 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 10 client.26146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 20 client.26146 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f643128600 0x55f64312aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 4 client.26146 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f643004320 con 0x55f643128600 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.213+0000 7f293bfff700 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f643128600 0x55f64312aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f293bfff700 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f643128600 0x55f64312aa20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b7c9355f168592d server_cookie=774377a9ffe92c in_seq=0 out_seq=0 2024-05-10T10:45:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f291b7fe700 10 client.26146 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f291b7fe700 1 -- 172.21.15.112:0/358392511 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f643004320 con 0x55f643128600 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f291b7fe700 10 client.26146 handle_command_reply: tid=0 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 client.26146 shutdown 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 msgr2=0x55f6430b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430b6dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55f6430aaeb0 tx=0x7f2924005d50 comp rx=0 tx=0).stop 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 shutdown_connections 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x55f643128600 0x55f64312aa20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6430aaad0 0x55f6430c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6430a4d50 0x55f6430b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 --2- 172.21.15.112:0/358392511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6430aa1b0 0x55f6430b6dd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f291b7fe700 0 client.26146 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.214+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 >> 172.21.15.112:0/358392511 conn(0x55f642fd0b70 msgr2=0x55f6430a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.215+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 shutdown_connections 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.215+0000 7f294ce82b80 1 -- 172.21.15.112:0/358392511 wait complete. 2024-05-10T10:45:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.215+0000 7f294ce82b80 20 client.26146 trim_cache size 0 max 16384 2024-05-10T10:45:48.221 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:45:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f29280b2880 msgr2=0x7f29280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f29280b2880 0x7f29280b4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f293c079900 tx=0x7f2924009260 comp rx=0 tx=0).stop 2024-05-10T10:45:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c1023a0 msgr2=0x7f293c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c1023a0 0x7f293c104080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f292c006010 tx=0x7f292c0047b0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 shutdown_connections 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f293c108410 0x7f293c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f29280b2880 0x7f29280b4d40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293c1023a0 0x7f293c104080 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 --2- 172.21.15.112:0/4232148530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f293c102d60 0x7f293c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 >> 172.21.15.112:0/4232148530 conn(0x7f293c079350 msgr2=0x7f293c06b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.216+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 shutdown_connections 2024-05-10T10:45:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:48.217+0000 7f29425ef700 1 -- 172.21.15.112:0/4232148530 wait complete. 2024-05-10T10:45:49.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:45:49.455 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:49.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:49 smithi148 ceph-mon[33568]: pgmap v597: 129 pgs: 129 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.1 MiB/s wr, 440 op/s 2024-05-10T10:45:49.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:49 smithi195 ceph-mon[33773]: pgmap v597: 129 pgs: 129 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.1 MiB/s wr, 440 op/s 2024-05-10T10:45:49.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:49 smithi112 ceph-mon[30100]: pgmap v597: 129 pgs: 129 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.1 MiB/s wr, 440 op/s 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 -- 172.21.15.112:0/3463173229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40101910 msgr2=0x7f5c40101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 --2- 172.21.15.112:0/3463173229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40101910 0x7f5c40101d30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002a410 comp rx=0 tx=0).stop 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 -- 172.21.15.112:0/3463173229 shutdown_connections 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 --2- 172.21.15.112:0/3463173229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c401034c0 0x7f5c40109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 --2- 172.21.15.112:0/3463173229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c40102b00 0x7f5c40102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 --2- 172.21.15.112:0/3463173229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40101910 0x7f5c40101d30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 -- 172.21.15.112:0/3463173229 >> 172.21.15.112:0/3463173229 conn(0x7f5c400fce80 msgr2=0x7f5c400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 -- 172.21.15.112:0/3463173229 shutdown_connections 2024-05-10T10:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.609+0000 7f5c44e64640 1 -- 172.21.15.112:0/3463173229 wait complete. 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 Processor -- start 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 -- start start 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c40101910 0x7f5c40198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40102b00 0x7f5c40198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c401034c0 0x7f5c401a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4010d2f0 con 0x7f5c40102b00 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4010d170 con 0x7f5c401034c0 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c44e64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4010d470 con 0x7f5c40101910 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40102b00 0x7f5c40198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40102b00 0x7f5c40198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48058/0 (socket says 172.21.15.112:48058) 2024-05-10T10:45:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/1059433799 learned_addr learned my addr 172.21.15.112:0/1059433799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c40101910 0x7f5c40198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.610+0000 7f5c3ed9d640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c401034c0 0x7f5c401a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/1059433799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c40101910 msgr2=0x7f5c40198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c40101910 0x7f5c40198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/1059433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c401034c0 msgr2=0x7f5c401a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c401034c0 0x7f5c401a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/1059433799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c401a06c0 con 0x7f5c40102b00 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40102b00 0x7f5c40198fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9f0 tx=0x7f5c3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c2f7fe640 1 -- 172.21.15.112:0/1059433799 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400cd80 con 0x7f5c40102b00 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c401a09b0 con 0x7f5c40102b00 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c2f7fe640 1 -- 172.21.15.112:0/1059433799 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c340108c0 con 0x7f5c40102b00 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c2f7fe640 1 -- 172.21.15.112:0/1059433799 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c340187c0 con 0x7f5c40102b00 2024-05-10T10:45:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.611+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c401a0f00 con 0x7f5c40102b00 2024-05-10T10:45:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c2f7fe640 1 -- 172.21.15.112:0/1059433799 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5c34010430 con 0x7f5c40102b00 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 Processor -- start 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 -- start start 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee051061c0 0x55ee051065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee0510b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee0510bd20 0x55ee0510e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee050665d0 con 0x55ee051061c0 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ee0501f890 con 0x55ee0510bd20 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.614+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ee05062590 con 0x55ee05106bb0 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee0510bd20 0x55ee0510e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee051061c0 0x55ee051065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee0510bd20 0x55ee0510e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56862/0 (socket says 172.21.15.112:56862) 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee051061c0 0x55ee051065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48064/0 (socket says 172.21.15.112:48064) 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2160558254 learned_addr learned my addr 172.21.15.112:0/2160558254 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee0510b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c2f7fe640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c280b28a0 0x7f5c280b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2160558254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 msgr2=0x55ee0510b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee0510b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2160558254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee051061c0 msgr2=0x55ee051065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee051061c0 0x55ee051065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2160558254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee04d78210 con 0x55ee0510bd20 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3ed9d640 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee051061c0 0x55ee051065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3ed9d640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c280b28a0 0x7f5c280b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c2f7fe640 1 -- 172.21.15.112:0/1059433799 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c340e2910 con 0x7f5c40102b00 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.615+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee0510bd20 0x55ee0510e1b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5c34023a10 tx=0x7f5c34024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5daa7cbd5145aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.616+0000 7f5c3ed9d640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c280b28a0 0x7f5c280b4d60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55ee05107410 tx=0x7f5c38004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.616+0000 7f5c2cff9640 1 -- 172.21.15.112:0/2160558254 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c340a9a20 con 0x55ee0510bd20 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.616+0000 7f5c2cff9640 1 -- 172.21.15.112:0/2160558254 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c340cae50 con 0x55ee0510bd20 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.616+0000 7f5c2cff9640 1 -- 172.21.15.112:0/2160558254 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34010720 con 0x55ee0510bd20 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2160558254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee0510bd20 msgr2=0x55ee0510e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee0510bd20 0x55ee0510e1b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5c34023a10 tx=0x7f5c34024000 comp rx=0 tx=0).stop 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2160558254 shutdown_connections 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee0510bd20 0x55ee0510e1b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee0510b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2160558254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee051061c0 0x55ee051065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2160558254 >> 172.21.15.112:0/2160558254 conn(0x55ee050fd420 msgr2=0x55ee050fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2160558254 shutdown_connections 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2160558254 wait complete. 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- start start 2024-05-10T10:45:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 Processor -- start 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee051102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee05106bb0 0x55ee05110800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee0510bd20 0x55ee05117840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee0510ec70 con 0x55ee05106bb0 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ee05023370 con 0x55ee051061c0 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.617+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ee0510edf0 con 0x55ee0510bd20 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c47b7d740 10 client.0 _ll_register_callbacks cb 0x55ee04d77a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee05106bb0 0x55ee05110800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee051102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee05106bb0 0x55ee05110800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48068/0 (socket says 172.21.15.112:48068) 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2364758537 learned_addr learned my addr 172.21.15.112:0/2364758537 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3ed9d640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee0510bd20 0x55ee05117840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee0510bd20 msgr2=0x55ee05117840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee0510bd20 0x55ee05117840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2364758537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 msgr2=0x55ee051102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee051102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2364758537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee04e44620 con 0x55ee05106bb0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee051102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.618+0000 7f5c3ed9d640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee0510bd20 0x55ee05117840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee05106bb0 0x55ee05110800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5c340a9be0 tx=0x7f5c34010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae783889f3d38fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c25ffb640 1 -- 172.21.15.112:0/2364758537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34023c60 con 0x55ee05106bb0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c25ffb640 1 -- 172.21.15.112:0/2364758537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c340a5640 con 0x55ee05106bb0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c340cae50 con 0x55ee05106bb0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c25ffb640 1 -- 172.21.15.112:0/2364758537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5c34011000 con 0x55ee05106bb0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c47b7d740 10 client.16758 fetch_fsmap learned FSMap version 38 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ee0511a820 con 0x55ee05106bb0 2024-05-10T10:45:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.619+0000 7f5c25ffb640 1 -- 172.21.15.112:0/2364758537 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c340cae50 con 0x55ee05106bb0 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c25ffb640 1 -- 172.21.15.112:0/2364758537 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f5c340c40c0 con 0x55ee05106bb0 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c47b7d740 10 client.16758 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c47b7d740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c47b7d740 20 client.16758 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee0511bbc0 0x55ee0511dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c47b7d740 4 client.16758 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee050741e0 con 0x55ee0511bbc0 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.620+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee0511bbc0 0x55ee0511dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.621+0000 7f5c25ffb640 10 client.16758 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.621+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee0511bbc0 0x55ee0511dfb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954e94d1dee0df66 server_cookie=d579f3ea9d14ecbf in_seq=0 out_seq=0 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.621+0000 7f5c25ffb640 1 -- 172.21.15.112:0/2364758537 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee050741e0 con 0x55ee0511bbc0 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.621+0000 7f5c25ffb640 10 client.16758 handle_command_reply: tid=0 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.621+0000 7f5c47b7d740 1 client.16758 shutdown 2024-05-10T10:45:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee05106bb0 msgr2=0x55ee05110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee05106bb0 0x55ee05110800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5c340a9be0 tx=0x7f5c34010cc0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 shutdown_connections 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee0510bd20 0x55ee05117840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee05106bb0 0x55ee05110800 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee0511bbc0 0x55ee0511dfb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2364758537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee051102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c25ffb640 0 client.16758 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 >> 172.21.15.112:0/2364758537 conn(0x55ee050fd420 msgr2=0x55ee050fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 shutdown_connections 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2364758537 wait complete. 2024-05-10T10:45:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.622+0000 7f5c47b7d740 20 client.16758 trim_cache size 0 max 16384 2024-05-10T10:45:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 Processor -- start 2024-05-10T10:45:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 -- start start 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee050f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee05117800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee050fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a9220 con 0x55ee0510bd20 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401aae50 con 0x55ee051061c0 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a98f0 con 0x55ee05106bb0 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee05117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee05117800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:51708/0 (socket says 172.21.15.112:51708) 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2140019941 learned_addr learned my addr 172.21.15.112:0/2140019941 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee050f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.639+0000 7f5c3ed9d640 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee050fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2140019941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 msgr2=0x55ee050f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee050f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2140019941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 msgr2=0x55ee050fbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee050fbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c3dd9b640 1 -- 172.21.15.112:0/2140019941 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee050fd250 con 0x55ee05106bb0 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee050f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee05117800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9c0 tx=0x7f5c34009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee96126542cec70f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c27fff640 1 -- 172.21.15.112:0/2140019941 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c340a93c0 con 0x55ee05106bb0 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c27fff640 1 -- 172.21.15.112:0/2140019941 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c340cae50 con 0x55ee05106bb0 2024-05-10T10:45:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.640+0000 7f5c27fff640 1 -- 172.21.15.112:0/2140019941 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c340a95a0 con 0x55ee05106bb0 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2140019941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 msgr2=0x55ee05117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee05117800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9c0 tx=0x7f5c34009310 comp rx=0 tx=0).stop 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2140019941 shutdown_connections 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee050fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee05117800 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/2140019941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee050f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2140019941 >> 172.21.15.112:0/2140019941 conn(0x55ee050fd420 msgr2=0x55ee05107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2140019941 shutdown_connections 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 -- 172.21.15.112:0/2140019941 wait complete. 2024-05-10T10:45:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 -- start start 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.641+0000 7f5c47b7d740 1 Processor -- start 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee05115260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee051157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c47b7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee051167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401aa750 con 0x55ee0510bd20 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401aa080 con 0x55ee051061c0 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c47b7d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401a7200 con 0x55ee05106bb0 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c47b7d740 10 client.0 _ll_register_callbacks cb 0x7f5c34018f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee05115260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee051157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee051167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee05115260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56890/0 (socket says 172.21.15.112:56890) 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 -- 172.21.15.112:0/1002977474 learned_addr learned my addr 172.21.15.112:0/1002977474 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 -- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 msgr2=0x55ee051157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee051157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 -- 172.21.15.112:0/1002977474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 msgr2=0x55ee051167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee051167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 -- 172.21.15.112:0/1002977474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee05116da0 con 0x55ee051061c0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee051157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c3e59c640 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee05115260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5c300057c0 tx=0x7f5c300042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f91ca8f3b081e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.642+0000 7f5c25ffb640 1 -- 172.21.15.112:0/1002977474 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30035040 con 0x55ee051061c0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c25ffb640 1 -- 172.21.15.112:0/1002977474 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c30031940 con 0x55ee051061c0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c340cae50 con 0x55ee051061c0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c25ffb640 1 -- 172.21.15.112:0/1002977474 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5c30057050 con 0x55ee051061c0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 10 client.26138 fetch_fsmap learned FSMap version 38 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ee05186800 con 0x55ee051061c0 2024-05-10T10:45:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c25ffb640 1 -- 172.21.15.112:0/1002977474 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c340cae50 con 0x55ee051061c0 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c25ffb640 1 -- 172.21.15.112:0/1002977474 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f5c300035f0 con 0x55ee051061c0 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 10 client.26138 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 20 client.26138 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee051871d0 0x55ee051895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 4 client.26138 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee0506e050 con 0x55ee051871d0 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.643+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee051871d0 0x55ee051895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.644+0000 7f5c25ffb640 10 client.26138 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.644+0000 7f5c3dd9b640 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee051871d0 0x55ee051895f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c843a1be82eaea server_cookie=75c6fb955e43fac5 in_seq=0 out_seq=0 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c25ffb640 1 -- 172.21.15.112:0/1002977474 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee0506e050 con 0x55ee051871d0 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c25ffb640 10 client.26138 handle_command_reply: tid=0 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 client.26138 shutdown 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 msgr2=0x55ee05115260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee05115260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5c300057c0 tx=0x7f5c300042c0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 shutdown_connections 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0510bd20 0x55ee051167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ee05106bb0 0x55ee051157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ee051871d0 0x55ee051895f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 --2- 172.21.15.112:0/1002977474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee051061c0 0x55ee05115260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c25ffb640 0 client.26138 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 >> 172.21.15.112:0/1002977474 conn(0x55ee050fd420 msgr2=0x55ee05100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 shutdown_connections 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 1 -- 172.21.15.112:0/1002977474 wait complete. 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.659+0000 7f5c47b7d740 20 client.26138 trim_cache size 0 max 16384 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:45:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:45:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c280b28a0 msgr2=0x7f5c280b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c280b28a0 0x7f5c280b4d60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55ee05107410 tx=0x7f5c38004a40 comp rx=0 tx=0).stop 2024-05-10T10:45:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40102b00 msgr2=0x7f5c40198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40102b00 0x7f5c40198fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9f0 tx=0x7f5c3400eeb0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 shutdown_connections 2024-05-10T10:45:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f5c280b28a0 0x7f5c280b4d60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c401034c0 0x7f5c401a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40102b00 0x7f5c40198fe0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 --2- 172.21.15.112:0/1059433799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c40101910 0x7f5c40198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 >> 172.21.15.112:0/1059433799 conn(0x7f5c400fce80 msgr2=0x7f5c400fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 shutdown_connections 2024-05-10T10:45:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:45:49.660+0000 7f5c44e64640 1 -- 172.21.15.112:0/1059433799 wait complete. 2024-05-10T10:45:51.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:51 smithi148 ceph-mon[33568]: pgmap v598: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-05-10T10:45:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:51 smithi195 ceph-mon[33773]: pgmap v598: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-05-10T10:45:51.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:51 smithi112 ceph-mon[30100]: pgmap v598: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-05-10T10:45:52.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:52 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:52.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:52 smithi112 ceph-mon[30100]: pgmap v599: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 601 op/s 2024-05-10T10:45:52.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:52 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:52.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:52 smithi148 ceph-mon[33568]: pgmap v599: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 601 op/s 2024-05-10T10:45:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:52 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:52.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:52 smithi195 ceph-mon[33773]: pgmap v599: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 601 op/s 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 -- 172.21.15.112:0/71765326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56901037b0 msgr2=0x7f5690103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 --2- 172.21.15.112:0/71765326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56901037b0 0x7f5690103b90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f56800099c0 tx=0x7f5680009cd0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 -- 172.21.15.112:0/71765326 shutdown_connections 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 --2- 172.21.15.112:0/71765326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5690108880 0x7f5690110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 --2- 172.21.15.112:0/71765326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56901037b0 0x7f5690103b90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 --2- 172.21.15.112:0/71765326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 0x7f5690108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 -- 172.21.15.112:0/71765326 >> 172.21.15.112:0/71765326 conn(0x7f569007a580 msgr2=0x7f569007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 -- 172.21.15.112:0/71765326 shutdown_connections 2024-05-10T10:45:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.657+0000 7f56967b2700 1 -- 172.21.15.112:0/71765326 wait complete. 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 Processor -- start 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 -- start start 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56901037b0 0x7f569019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 0x7f569019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5690108880 0x7f56901997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5690113520 con 0x7f5690108880 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56901133a0 con 0x7f56901037b0 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f56967b2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56901136a0 con 0x7f5690104160 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f5694d4f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5690108880 0x7f56901997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f568f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 0x7f569019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f568ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56901037b0 0x7f569019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f568f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 0x7f569019fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:51752/0 (socket says 172.21.15.112:51752) 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f568ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56901037b0 0x7f569019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56922/0 (socket says 172.21.15.112:56922) 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.658+0000 7f568f7fe700 1 -- 172.21.15.112:0/1591349088 learned_addr learned my addr 172.21.15.112:0/1591349088 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f5694d4f700 1 -- 172.21.15.112:0/1591349088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 msgr2=0x7f569019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f5694d4f700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 0x7f569019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f5694d4f700 1 -- 172.21.15.112:0/1591349088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56901037b0 msgr2=0x7f569019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f5694d4f700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56901037b0 0x7f569019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f5694d4f700 1 -- 172.21.15.112:0/1591349088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5690199fc0 con 0x7f5690108880 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f568ffff700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56901037b0 0x7f569019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f568f7fe700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 0x7f569019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f5694d4f700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5690108880 0x7f56901997e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f568800efc0 tx=0x7f568800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f568d7fa700 1 -- 172.21.15.112:0/1591349088 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880092e0 con 0x7f5690108880 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569019a250 con 0x7f5690108880 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f568d7fa700 1 -- 172.21.15.112:0/1591349088 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5688004bb0 con 0x7f5690108880 2024-05-10T10:45:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a64c0 con 0x7f5690108880 2024-05-10T10:45:52.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.659+0000 7f568d7fa700 1 -- 172.21.15.112:0/1591349088 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688004020 con 0x7f5690108880 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.662+0000 7f568d7fa700 1 -- 172.21.15.112:0/1591349088 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f568801b030 con 0x7f5690108880 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f568d7fa700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f56780b2900 0x7f56780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f568ffff700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f56780b2900 0x7f56780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f568d7fa700 1 -- 172.21.15.112:0/1591349088 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f56880e2000 con 0x7f5690108880 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f56a1045b80 1 Processor -- start 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f56a1045b80 1 -- start start 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f568ffff700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f56780b2900 0x7f56780b4dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5680003760 tx=0x7f5680005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f58e0 0x55c2b33f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.663+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f58e0 0x55c2b33f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568ffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f58e0 0x55c2b33f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:51766/0 (socket says 172.21.15.112:51766) 2024-05-10T10:45:52.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f56a1045b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c2b3343860 con 0x55c2b33f6200 2024-05-10T10:45:52.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568ffff700 1 -- 172.21.15.112:0/3288211941 learned_addr learned my addr 172.21.15.112:0/3288211941 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:52.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f56a1045b80 1 -- 172.21.15.112:0/3288211941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2b32e27c0 con 0x55c2b33fa040 2024-05-10T10:45:52.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f56a1045b80 1 -- 172.21.15.112:0/3288211941 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2b333f820 con 0x55c2b33f58e0 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f5694d4f700 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568f7fe700 1 -- 172.21.15.112:0/3288211941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f58e0 msgr2=0x55c2b33f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568f7fe700 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f58e0 0x55c2b33f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568f7fe700 1 -- 172.21.15.112:0/3288211941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 msgr2=0x55c2b33fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568f7fe700 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568f7fe700 1 -- 172.21.15.112:0/3288211941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b32908d0 con 0x55c2b33f6200 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568ffff700 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f58e0 0x55c2b33f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f568f7fe700 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f9b00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5690103100 tx=0x7f56840118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89816fee7bb3bf08 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f56767fc700 1 -- 172.21.15.112:0/3288211941 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568401e070 con 0x55c2b33f6200 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f56767fc700 1 -- 172.21.15.112:0/3288211941 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5684019ac0 con 0x55c2b33f6200 2024-05-10T10:45:52.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.664+0000 7f56767fc700 1 -- 172.21.15.112:0/3288211941 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568400fa90 con 0x55c2b33f6200 2024-05-10T10:45:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 -- 172.21.15.112:0/3288211941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 msgr2=0x55c2b33f9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f9b00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5690103100 tx=0x7f56840118a0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 -- 172.21.15.112:0/3288211941 shutdown_connections 2024-05-10T10:45:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f58e0 0x55c2b33f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3288211941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f9b00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 -- 172.21.15.112:0/3288211941 >> 172.21.15.112:0/3288211941 conn(0x55c2b331d100 msgr2=0x55c2b33ebc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 -- 172.21.15.112:0/3288211941 shutdown_connections 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.665+0000 7f56a1045b80 1 -- 172.21.15.112:0/3288211941 wait complete. 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 -- start start 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 Processor -- start 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f6200 0x55c2b33fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c2b3402270 con 0x55c2b33f58e0 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2b32a9c30 con 0x55c2b33fa040 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2b34023f0 con 0x55c2b33f6200 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f56a1045b80 10 client.0 _ll_register_callbacks cb 0x55c2b32db920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f568ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f568ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48132/0 (socket says 172.21.15.112:48132) 2024-05-10T10:45:52.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f568ffff700 1 -- 172.21.15.112:0/1794487454 learned_addr learned my addr 172.21.15.112:0/1794487454 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f568f7fe700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f6200 0x55c2b33fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.666+0000 7f5694d4f700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f568ffff700 1 -- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f6200 msgr2=0x55c2b33fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f568ffff700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f6200 0x55c2b33fe850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f568ffff700 1 -- 172.21.15.112:0/1794487454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 msgr2=0x55c2b33fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f568ffff700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f568ffff700 1 -- 172.21.15.112:0/1794487454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b32df690 con 0x55c2b33f58e0 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f568f7fe700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f6200 0x55c2b33fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f5694d4f700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f568ffff700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34014b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f56800098e0 tx=0x7f568001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf905a813e0d3f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f566f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f566f7fe700 1 -- 172.21.15.112:0/1794487454 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680024820 con 0x55c2b33f58e0 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f566f7fe700 1 -- 172.21.15.112:0/1794487454 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f56800249c0 con 0x55c2b33f58e0 2024-05-10T10:45:52.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684019ac0 con 0x55c2b33f58e0 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f566f7fe700 1 -- 172.21.15.112:0/1794487454 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f568003e050 con 0x55c2b33f58e0 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f56a1045b80 10 client.16776 fetch_fsmap learned FSMap version 39 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2b340cb10 con 0x55c2b33f58e0 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.667+0000 7f566f7fe700 1 -- 172.21.15.112:0/1794487454 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5684019ac0 con 0x55c2b33f58e0 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f566f7fe700 1 -- 172.21.15.112:0/1794487454 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f5680030920 con 0x55c2b33f58e0 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f56a1045b80 10 client.16776 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f56a1045b80 10 client.16776 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f56a1045b80 20 client.16776 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f56a1045b80 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b340f150 0x55c2b3411570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f56a1045b80 4 client.16776 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2b33bd650 con 0x55c2b340f150 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f568f7fe700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b340f150 0x55c2b3411570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f568f7fe700 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b340f150 0x55c2b3411570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c3b36a2f6149952 server_cookie=bf7cf9f8ce0d7e96 in_seq=0 out_seq=0 2024-05-10T10:45:52.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.668+0000 7f566f7fe700 10 client.16776 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f566f7fe700 1 -- 172.21.15.112:0/1794487454 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2b33bd650 con 0x55c2b340f150 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f566f7fe700 10 client.16776 handle_command_reply: tid=0 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 client.16776 shutdown 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 msgr2=0x55c2b34014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34014b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f56800098e0 tx=0x7f568001a050 comp rx=0 tx=0).stop 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 shutdown_connections 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b340f150 0x55c2b3411570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33fa040 0x55c2b33fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34014b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 --2- 172.21.15.112:0/1794487454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33f6200 0x55c2b33fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 >> 172.21.15.112:0/1794487454 conn(0x55c2b331d100 msgr2=0x55c2b33e6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f566f7fe700 0 client.16776 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.669+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 shutdown_connections 2024-05-10T10:45:52.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.670+0000 7f56a1045b80 1 -- 172.21.15.112:0/1794487454 wait complete. 2024-05-10T10:45:52.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.670+0000 7f56a1045b80 20 client.16776 trim_cache size 0 max 16384 2024-05-10T10:45:52.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 Processor -- start 2024-05-10T10:45:52.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 -- start start 2024-05-10T10:45:52.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f58e0 0x55c2b33fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56901acef0 con 0x55c2b33f6200 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56901aeaf0 con 0x55c2b33f58e0 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f56a1045b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56901ad5c0 con 0x55c2b33fa040 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f568f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f568ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f58e0 0x55c2b33fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f568f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48140/0 (socket says 172.21.15.112:48140) 2024-05-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f568f7fe700 1 -- 172.21.15.112:0/3313473480 learned_addr learned my addr 172.21.15.112:0/3313473480 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.686+0000 7f5694d4f700 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f568f7fe700 1 -- 172.21.15.112:0/3313473480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 msgr2=0x55c2b33f0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f568f7fe700 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f568f7fe700 1 -- 172.21.15.112:0/3313473480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f58e0 msgr2=0x55c2b33fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f568f7fe700 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f58e0 0x55c2b33fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f568f7fe700 1 -- 172.21.15.112:0/3313473480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b33fcdc0 con 0x55c2b33f6200 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f568f7fe700 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f02e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f56840098a0 tx=0x7f5684009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2db1742b03e74f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f5675ffb700 1 -- 172.21.15.112:0/3313473480 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684030020 con 0x55c2b33f6200 2024-05-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f5675ffb700 1 -- 172.21.15.112:0/3313473480 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5684005040 con 0x55c2b33f6200 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.687+0000 7f5675ffb700 1 -- 172.21.15.112:0/3313473480 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568401e040 con 0x55c2b33f6200 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 -- 172.21.15.112:0/3313473480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 msgr2=0x55c2b33f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f02e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f56840098a0 tx=0x7f5684009670 comp rx=0 tx=0).stop 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 -- 172.21.15.112:0/3313473480 shutdown_connections 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f58e0 0x55c2b33fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 --2- 172.21.15.112:0/3313473480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f6200 0x55c2b33f02e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 -- 172.21.15.112:0/3313473480 >> 172.21.15.112:0/3313473480 conn(0x55c2b331d100 msgr2=0x55c2b33e6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 -- 172.21.15.112:0/3313473480 shutdown_connections 2024-05-10T10:45:52.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 -- 172.21.15.112:0/3313473480 wait complete. 2024-05-10T10:45:52.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 -- start start 2024-05-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.688+0000 7f56a1045b80 1 Processor -- start 2024-05-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f6200 0x55c2b3407c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f56a1045b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f56a1045b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56901ae3f0 con 0x55c2b33f58e0 2024-05-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f56a1045b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56901add20 con 0x55c2b33f6200 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f56a1045b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56901aae90 con 0x55c2b33fa040 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f56a1045b80 10 client.0 _ll_register_callbacks cb 0x55c2b32db920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f6200 0x55c2b3407c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f6200 0x55c2b3407c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56950/0 (socket says 172.21.15.112:56950) 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 -- 172.21.15.112:0/848804539 learned_addr learned my addr 172.21.15.112:0/848804539 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34076e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48146/0 (socket says 172.21.15.112:48146) 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f5694d4f700 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 -- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 msgr2=0x55c2b33f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 -- 172.21.15.112:0/848804539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 msgr2=0x55c2b34076e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34076e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 -- 172.21.15.112:0/848804539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b33f81b0 con 0x55c2b33f6200 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568ffff700 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f568f7fe700 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f6200 0x55c2b3407c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55c2b340bfa0 tx=0x7f5684009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724bddc8dc6db496 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.689+0000 7f566f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.690+0000 7f566f7fe700 1 -- 172.21.15.112:0/848804539 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684030030 con 0x55c2b33f6200 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.690+0000 7f566f7fe700 1 -- 172.21.15.112:0/848804539 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f56840196d0 con 0x55c2b33f6200 2024-05-10T10:45:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.690+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684005040 con 0x55c2b33f6200 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.690+0000 7f566f7fe700 1 -- 172.21.15.112:0/848804539 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5684044050 con 0x55c2b33f6200 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.690+0000 7f56a1045b80 10 client.26156 fetch_fsmap learned FSMap version 39 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.690+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2b3416890 con 0x55c2b33f6200 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f566f7fe700 1 -- 172.21.15.112:0/848804539 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56840198d0 con 0x55c2b33f6200 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f566f7fe700 1 -- 172.21.15.112:0/848804539 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f568402cce0 con 0x55c2b33f6200 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f56a1045b80 10 client.26156 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f56a1045b80 10 client.26156 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f56a1045b80 20 client.26156 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f56a1045b80 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b34799d0 0x55c2b347bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f56a1045b80 4 client.26156 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.691+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5684005040 con 0x55c2b34799d0 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.692+0000 7f568ffff700 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b34799d0 0x55c2b347bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.692+0000 7f566f7fe700 10 client.26156 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.692+0000 7f568ffff700 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b34799d0 0x55c2b347bdc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91e51715670b0de9 server_cookie=c79c92a1d24e646c in_seq=0 out_seq=0 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.692+0000 7f566f7fe700 1 -- 172.21.15.112:0/848804539 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5684005040 con 0x55c2b34799d0 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.692+0000 7f566f7fe700 10 client.26156 handle_command_reply: tid=0 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.693+0000 7f56a1045b80 1 client.26156 shutdown 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.693+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f6200 msgr2=0x55c2b3407c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.693+0000 7f56a1045b80 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f6200 0x55c2b3407c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55c2b340bfa0 tx=0x7f5684009260 comp rx=0 tx=0).stop 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 shutdown_connections 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55c2b34799d0 0x55c2b347bdc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2b33fa040 0x55c2b33f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2b33f58e0 0x55c2b34076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f566f7fe700 0 client.26156 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 --2- 172.21.15.112:0/848804539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2b33f6200 0x55c2b3407c20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 >> 172.21.15.112:0/848804539 conn(0x55c2b331d100 msgr2=0x55c2b33f02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 shutdown_connections 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 1 -- 172.21.15.112:0/848804539 wait complete. 2024-05-10T10:45:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.694+0000 7f56a1045b80 20 client.26156 trim_cache size 0 max 16384 2024-05-10T10:45:52.704 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f56780b2900 msgr2=0x7f56780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f56780b2900 0x7f56780b4dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5680003760 tx=0x7f5680005fb0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5690108880 msgr2=0x7f56901997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5690108880 0x7f56901997e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f568800efc0 tx=0x7f568800c3a0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 shutdown_connections 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5690108880 0x7f56901997e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56901037b0 0x7f569019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f56780b2900 0x7f56780b4dc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 --2- 172.21.15.112:0/1591349088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5690104160 0x7f569019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 >> 172.21.15.112:0/1591349088 conn(0x7f569007a580 msgr2=0x7f5690101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 shutdown_connections 2024-05-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:52.695+0000 7f56967b2700 1 -- 172.21.15.112:0/1591349088 wait complete. 2024-05-10T10:45:53.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:45:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:54.557 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-05-10T10:45:54.558 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-05-10T10:45:54.558 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-05-10T10:45:54.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:54 smithi148 ceph-mon[33568]: pgmap v600: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 397 op/s 2024-05-10T10:45:54.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:54 smithi195 ceph-mon[33773]: pgmap v600: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 397 op/s 2024-05-10T10:45:54.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:54 smithi112 ceph-mon[30100]: pgmap v600: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 397 op/s 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 -- 172.21.15.112:0/1076328408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 msgr2=0x7f96e4110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 --2- 172.21.15.112:0/1076328408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e4110400 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00b0d0 tx=0x7f96dc00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 -- 172.21.15.112:0/1076328408 shutdown_connections 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 --2- 172.21.15.112:0/1076328408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e4110400 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 --2- 172.21.15.112:0/1076328408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96e41037b0 0x7f96e4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 --2- 172.21.15.112:0/1076328408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e4104160 0x7f96e4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 -- 172.21.15.112:0/1076328408 >> 172.21.15.112:0/1076328408 conn(0x7f96e407a580 msgr2=0x7f96e407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 -- 172.21.15.112:0/1076328408 shutdown_connections 2024-05-10T10:45:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 -- 172.21.15.112:0/1076328408 wait complete. 2024-05-10T10:45:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 Processor -- start 2024-05-10T10:45:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.570+0000 7f96ea593700 1 -- start start 2024-05-10T10:45:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96ea593700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e41037b0 0x7f96e419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96ea593700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96e4104160 0x7f96e419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96ea593700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96ea593700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4113520 con 0x7f96e4108880 2024-05-10T10:45:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96ea593700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41133a0 con 0x7f96e41037b0 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96ea593700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41136a0 con 0x7f96e4104160 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96e8b30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96e8b30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e41997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44478/0 (socket says 172.21.15.112:44478) 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96e8b30700 1 -- 172.21.15.112:0/2106413560 learned_addr learned my addr 172.21.15.112:0/2106413560 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96e3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e41037b0 0x7f96e419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.571+0000 7f96e37fe700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96e4104160 0x7f96e419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e8b30700 1 -- 172.21.15.112:0/2106413560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96e4104160 msgr2=0x7f96e419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e8b30700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96e4104160 0x7f96e419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e8b30700 1 -- 172.21.15.112:0/2106413560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e41037b0 msgr2=0x7f96e419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e8b30700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e41037b0 0x7f96e419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e8b30700 1 -- 172.21.15.112:0/2106413560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e4199f80 con 0x7f96e4108880 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e8b30700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e41997a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009850 tx=0x7f96dc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e17fa700 1 -- 172.21.15.112:0/2106413560 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc01f070 con 0x7f96e4108880 2024-05-10T10:45:55.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e419a210 con 0x7f96e4108880 2024-05-10T10:45:55.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e17fa700 1 -- 172.21.15.112:0/2106413560 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96dc005500 con 0x7f96e4108880 2024-05-10T10:45:55.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41a6480 con 0x7f96e4108880 2024-05-10T10:45:55.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.572+0000 7f96e17fa700 1 -- 172.21.15.112:0/2106413560 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc029060 con 0x7f96e4108880 2024-05-10T10:45:55.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.575+0000 7f96e17fa700 1 -- 172.21.15.112:0/2106413560 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f96dc022070 con 0x7f96e4108880 2024-05-10T10:45:55.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96e17fa700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f96d00b2880 0x7f96d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96e3fff700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f96d00b2880 0x7f96d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96e17fa700 1 -- 172.21.15.112:0/2106413560 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f96dc0e3430 con 0x7f96e4108880 2024-05-10T10:45:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 Processor -- start 2024-05-10T10:45:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 -- start start 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96e3fff700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f96d00b2880 0x7f96d00b4d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f96d4004630 tx=0x7f96d400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f6cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565430eba940 con 0x565430f72ee0 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x565430e1fda0 con 0x565430f725c0 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x565430eb6900 con 0x565430f6d160 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.576+0000 7f96e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f6cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f6cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44480/0 (socket says 172.21.15.112:44480) 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 -- 172.21.15.112:0/1182442876 learned_addr learned my addr 172.21.15.112:0/1182442876 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e3fff700 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e8b30700 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 -- 172.21.15.112:0/1182442876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 msgr2=0x565430f6d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 -- 172.21.15.112:0/1182442876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 msgr2=0x565430f729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 -- 172.21.15.112:0/1182442876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565430e078d0 con 0x565430f72ee0 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96e37fe700 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f6cc20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ac80 tx=0x7f96cc00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382f7d5b0ae08a3c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96da7fc700 1 -- 172.21.15.112:0/1182442876 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc01d070 con 0x565430f72ee0 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96da7fc700 1 -- 172.21.15.112:0/1182442876 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96cc00fde0 con 0x565430f72ee0 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.577+0000 7f96da7fc700 1 -- 172.21.15.112:0/1182442876 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0106c0 con 0x565430f72ee0 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1182442876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 msgr2=0x565430f6cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f6cc20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ac80 tx=0x7f96cc00af90 comp rx=0 tx=0).stop 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1182442876 shutdown_connections 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1182442876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f6cc20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1182442876 >> 172.21.15.112:0/1182442876 conn(0x565430e94b70 msgr2=0x565430f632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1182442876 shutdown_connections 2024-05-10T10:45:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1182442876 wait complete. 2024-05-10T10:45:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.578+0000 7f96f4e26b80 1 -- start start 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 1 Processor -- start 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f78b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565430f79860 con 0x565430f72ee0 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x565430e73ed0 con 0x565430f725c0 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x565430f799e0 con 0x565430f6d160 2024-05-10T10:45:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96f4e26b80 10 client.0 _ll_register_callbacks cb 0x565430e4e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e8b30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f78b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e8b30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f76430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44490/0 (socket says 172.21.15.112:44490) 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e8b30700 1 -- 172.21.15.112:0/1253659206 learned_addr learned my addr 172.21.15.112:0/1253659206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e37fe700 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e3fff700 1 -- 172.21.15.112:0/1253659206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 msgr2=0x565430f75ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e3fff700 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f75ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e3fff700 1 -- 172.21.15.112:0/1253659206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 msgr2=0x565430f76430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e3fff700 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f76430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e3fff700 1 -- 172.21.15.112:0/1253659206 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565430e71760 con 0x565430f6d160 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.579+0000 7f96e8b30700 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96e3fff700 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f78b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96d400bd70 tx=0x7f96d40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd234ca3bc6fe933 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:45:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96c37fe700 1 -- 172.21.15.112:0/1253659206 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4011ce0 con 0x565430f6d160 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96cc00fde0 con 0x565430f6d160 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96c37fe700 1 -- 172.21.15.112:0/1253659206 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f96d4028470 con 0x565430f6d160 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96c37fe700 1 -- 172.21.15.112:0/1253659206 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f96d4031050 con 0x565430f6d160 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96f4e26b80 10 client.26191 fetch_fsmap learned FSMap version 39 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565430f84110 con 0x565430f6d160 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.580+0000 7f96c37fe700 1 -- 172.21.15.112:0/1253659206 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96cc00fde0 con 0x565430f6d160 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96c37fe700 1 -- 172.21.15.112:0/1253659206 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f96d401e430 con 0x565430f6d160 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96f4e26b80 10 client.26191 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96f4e26b80 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96f4e26b80 20 client.26191 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430f86750 0x565430f88b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96f4e26b80 4 client.26191 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565430ec8320 con 0x565430f86750 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96e37fe700 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430f86750 0x565430f88b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96e37fe700 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430f86750 0x565430f88b60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf70ed682047907 server_cookie=795ba20df86bc6c6 in_seq=0 out_seq=0 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.581+0000 7f96c37fe700 10 client.26191 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96c37fe700 1 -- 172.21.15.112:0/1253659206 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565430ec8320 con 0x565430f86750 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96c37fe700 10 client.26191 handle_command_reply: tid=0 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 client.26191 shutdown 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 msgr2=0x565430f78b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f78b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96d400bd70 tx=0x7f96d40047c0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 shutdown_connections 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430f86750 0x565430f88b60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f72ee0 0x565430f76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f78b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/1253659206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f75ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 >> 172.21.15.112:0/1253659206 conn(0x565430e94b70 msgr2=0x565430f5db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.582+0000 7f96c37fe700 0 client.26191 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.583+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 shutdown_connections 2024-05-10T10:45:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.583+0000 7f96f4e26b80 1 -- 172.21.15.112:0/1253659206 wait complete. 2024-05-10T10:45:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.583+0000 7f96f4e26b80 20 client.26191 trim_cache size 0 max 16384 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.597+0000 7f96f4e26b80 1 Processor -- start 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.597+0000 7f96f4e26b80 1 -- start start 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f725c0 0x565430f76460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f72ee0 0x565430f82920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41ace30 con 0x565430f725c0 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41aea30 con 0x565430f72ee0 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41ad500 con 0x565430f6d160 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f725c0 0x565430f76460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f725c0 0x565430f76460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44506/0 (socket says 172.21.15.112:44506) 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 -- 172.21.15.112:0/3782124351 learned_addr learned my addr 172.21.15.112:0/3782124351 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:55.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e3fff700 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 -- 172.21.15.112:0/3782124351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 msgr2=0x565430f6cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 -- 172.21.15.112:0/3782124351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f72ee0 msgr2=0x565430f82920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f72ee0 0x565430f82920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 -- 172.21.15.112:0/3782124351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565430f64d00 con 0x565430f725c0 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e3fff700 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96e37fe700 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f725c0 0x565430f76460 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ac80 tx=0x7f96cc018ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eeba3b464e50dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96d9ffb700 1 -- 172.21.15.112:0/3782124351 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc01d040 con 0x565430f725c0 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96d9ffb700 1 -- 172.21.15.112:0/3782124351 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96cc005d70 con 0x565430f725c0 2024-05-10T10:45:55.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.598+0000 7f96d9ffb700 1 -- 172.21.15.112:0/3782124351 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0046f0 con 0x565430f725c0 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 -- 172.21.15.112:0/3782124351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f725c0 msgr2=0x565430f76460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f725c0 0x565430f76460 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ac80 tx=0x7f96cc018ea0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 -- 172.21.15.112:0/3782124351 shutdown_connections 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f72ee0 0x565430f82920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f6d160 0x565430f6cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/3782124351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f725c0 0x565430f76460 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 -- 172.21.15.112:0/3782124351 >> 172.21.15.112:0/3782124351 conn(0x565430e94b70 msgr2=0x565430f732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 -- 172.21.15.112:0/3782124351 shutdown_connections 2024-05-10T10:45:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 -- 172.21.15.112:0/3782124351 wait complete. 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.599+0000 7f96f4e26b80 1 -- start start 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 1 Processor -- start 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f6d160 0x565430f8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f7eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f72ee0 0x565430f7f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41ae330 con 0x565430f6d160 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41adc60 con 0x565430f725c0 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41aadf0 con 0x565430f72ee0 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96f4e26b80 10 client.0 _ll_register_callbacks cb 0x565430e4e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f6d160 0x565430f8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f6d160 0x565430f8e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44518/0 (socket says 172.21.15.112:44518) 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 -- 172.21.15.112:0/516601736 learned_addr learned my addr 172.21.15.112:0/516601736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:45:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e8b30700 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f72ee0 0x565430f7f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e37fe700 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f7eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 -- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f72ee0 msgr2=0x565430f7f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f72ee0 0x565430f7f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 -- 172.21.15.112:0/516601736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 msgr2=0x565430f7eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f7eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.600+0000 7f96e3fff700 1 -- 172.21.15.112:0/516601736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565430f8d460 con 0x565430f6d160 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96e3fff700 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f6d160 0x565430f8e6c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f96d401e800 tx=0x7f96d4011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20428283dc03a8e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96c37fe700 1 -- 172.21.15.112:0/516601736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4003680 con 0x565430f6d160 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96c37fe700 1 -- 172.21.15.112:0/516601736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f96d4028470 con 0x565430f6d160 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96cc005d70 con 0x565430f6d160 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96c37fe700 1 -- 172.21.15.112:0/516601736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f96d4043050 con 0x565430f6d160 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 10 client.16800 fetch_fsmap learned FSMap version 39 2024-05-10T10:45:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565430ff00e0 con 0x565430f6d160 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96c37fe700 1 -- 172.21.15.112:0/516601736 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96cc005d70 con 0x565430f6d160 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96c37fe700 1 -- 172.21.15.112:0/516601736 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f96d4034070 con 0x565430f6d160 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 20 client.16800 populate_metadata read hostname 'smithi112' 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430ff0820 0x565430ff2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 4 client.16800 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.601+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565430ec23c0 con 0x565430ff0820 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.602+0000 7f96e37fe700 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430ff0820 0x565430ff2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.602+0000 7f96c37fe700 10 client.16800 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.602+0000 7f96e37fe700 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430ff0820 0x565430ff2c40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271dcae38d6fbbaf server_cookie=550dd469a9b2c0d6 in_seq=0 out_seq=0 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.602+0000 7f96c37fe700 1 -- 172.21.15.112:0/516601736 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565430ec23c0 con 0x565430ff0820 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.602+0000 7f96c37fe700 10 client.16800 handle_command_reply: tid=0 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.602+0000 7f96f4e26b80 1 client.16800 shutdown 2024-05-10T10:45:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f6d160 msgr2=0x565430f8e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f6d160 0x565430f8e6c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f96d401e800 tx=0x7f96d4011d90 comp rx=0 tx=0).stop 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 shutdown_connections 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x565430ff0820 0x565430ff2c40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565430f72ee0 0x565430f7f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565430f6d160 0x565430f8e6c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 --2- 172.21.15.112:0/516601736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565430f725c0 0x565430f7eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96c37fe700 0 client.16800 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 >> 172.21.15.112:0/516601736 conn(0x565430e94b70 msgr2=0x565430f71050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 shutdown_connections 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 1 -- 172.21.15.112:0/516601736 wait complete. 2024-05-10T10:45:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96f4e26b80 20 client.16800 trim_cache size 0 max 16384 2024-05-10T10:45:55.610 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:45:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f96d00b2880 msgr2=0x7f96d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96ea593700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f96d00b2880 0x7f96d00b4d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f96d4004630 tx=0x7f96d400b3c0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 msgr2=0x7f96e41997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.603+0000 7f96ea593700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e41997a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009850 tx=0x7f96dc007d90 comp rx=0 tx=0).stop 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 shutdown_connections 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e4108880 0x7f96e41997a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96e41037b0 0x7f96e419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f96d00b2880 0x7f96d00b4d40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 --2- 172.21.15.112:0/2106413560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96e4104160 0x7f96e419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 >> 172.21.15.112:0/2106413560 conn(0x7f96e407a580 msgr2=0x7f96e4101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 shutdown_connections 2024-05-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:45:55.604+0000 7f96ea593700 1 -- 172.21.15.112:0/2106413560 wait complete. 2024-05-10T10:45:55.715 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-10T10:45:55.715 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-10T10:45:55.715 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-05-10T10:45:55.715 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-10T10:45:55.719 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-10T10:45:55.730 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x1c37b90 2024-05-10T10:45:55.739 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x5bd330d 2024-05-10T10:45:55.753 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x637bf99 2024-05-10T10:45:56.033 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63d9b9c 2024-05-10T10:45:57.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:56 smithi112 ceph-mon[30100]: pgmap v601: 129 pgs: 129 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.9 MiB/s wr, 547 op/s 2024-05-10T10:45:57.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:56 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:57.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:56 smithi148 ceph-mon[33568]: pgmap v601: 129 pgs: 129 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.9 MiB/s wr, 547 op/s 2024-05-10T10:45:57.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:56 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:56 smithi195 ceph-mon[33773]: pgmap v601: 129 pgs: 129 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.9 MiB/s wr, 547 op/s 2024-05-10T10:45:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:56 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:45:57.456 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:45:57.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:45:57.791 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:45:58.985 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:45:58 smithi112 ceph-mon[30100]: pgmap v602: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.9 MiB/s wr, 398 op/s 2024-05-10T10:45:59.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:45:58 smithi148 ceph-mon[33568]: pgmap v602: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.9 MiB/s wr, 398 op/s 2024-05-10T10:45:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:45:58 smithi195 ceph-mon[33773]: pgmap v602: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.9 MiB/s wr, 398 op/s 2024-05-10T10:46:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.410+0000 7f6794a23700 1 -- 172.21.15.112:0/1437415851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790101b80 msgr2=0x7f6790102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.410+0000 7f6794a23700 1 --2- 172.21.15.112:0/1437415851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790101b80 0x7f6790102000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6784009920 tx=0x7f6784009c30 comp rx=0 tx=0).stop 2024-05-10T10:46:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 -- 172.21.15.112:0/1437415851 shutdown_connections 2024-05-10T10:46:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 --2- 172.21.15.112:0/1437415851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6790102540 0x7f679010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 --2- 172.21.15.112:0/1437415851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790107be0 0x7f6790107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 --2- 172.21.15.112:0/1437415851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790101b80 0x7f6790102000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 -- 172.21.15.112:0/1437415851 >> 172.21.15.112:0/1437415851 conn(0x7f6790078a10 msgr2=0x7f67900ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 -- 172.21.15.112:0/1437415851 shutdown_connections 2024-05-10T10:46:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 -- 172.21.15.112:0/1437415851 wait complete. 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.411+0000 7f6794a23700 1 Processor -- start 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f6794a23700 1 -- start start 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f6794a23700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6790101b80 0x7f67901037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f6794a23700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790102540 0x7f6790103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f6794a23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790107be0 0x7f6790104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f6794a23700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6790111300 con 0x7f6790107be0 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f6794a23700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6790111180 con 0x7f6790102540 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f6794a23700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6790111480 con 0x7f6790101b80 2024-05-10T10:46:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790102540 0x7f6790103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790102540 0x7f6790103ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51006/0 (socket says 172.21.15.112:51006) 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 -- 172.21.15.112:0/4152944799 learned_addr learned my addr 172.21.15.112:0/4152944799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678e59c700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6790101b80 0x7f67901037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678ed9d700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790107be0 0x7f6790104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 -- 172.21.15.112:0/4152944799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6790101b80 msgr2=0x7f67901037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6790101b80 0x7f67901037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 -- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790107be0 msgr2=0x7f6790104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790107be0 0x7f6790104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.412+0000 7f678dd9b700 1 -- 172.21.15.112:0/4152944799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67901ac440 con 0x7f6790102540 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.413+0000 7f678dd9b700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790102540 0x7f6790103ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6784000c00 tx=0x7f678400f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.413+0000 7f677f7fe700 1 -- 172.21.15.112:0/4152944799 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678400b560 con 0x7f6790102540 2024-05-10T10:46:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.413+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67901ac670 con 0x7f6790102540 2024-05-10T10:46:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.413+0000 7f677f7fe700 1 -- 172.21.15.112:0/4152944799 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f678400bb90 con 0x7f6790102540 2024-05-10T10:46:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.413+0000 7f677f7fe700 1 -- 172.21.15.112:0/4152944799 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784003970 con 0x7f6790102540 2024-05-10T10:46:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.413+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901acac0 con 0x7f6790102540 2024-05-10T10:46:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.416+0000 7f677f7fe700 1 -- 172.21.15.112:0/4152944799 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f67840052f0 con 0x7f6790102540 2024-05-10T10:46:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.417+0000 7f679f2b6b80 1 Processor -- start 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f679f2b6b80 1 -- start start 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f677f7fe700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67780b2880 0x7f67780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febdb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febdf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592feb2d8e0 con 0x5592febdfd00 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5592feacc840 con 0x5592febdb550 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592feb298a0 con 0x5592febdbe70 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f677f7fe700 1 -- 172.21.15.112:0/4152944799 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f67840e2fd0 con 0x7f6790102540 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f678ed9d700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67780b2880 0x7f67780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f678dd9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febdf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f678e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febdb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f678dd9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febdf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60526/0 (socket says 172.21.15.112:60526) 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f678dd9b700 1 -- 172.21.15.112:0/1457501322 learned_addr learned my addr 172.21.15.112:0/1457501322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.418+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678dd9b700 1 -- 172.21.15.112:0/1457501322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 msgr2=0x5592febdb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678dd9b700 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febdb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678dd9b700 1 -- 172.21.15.112:0/1457501322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 msgr2=0x5592febe0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678dd9b700 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678dd9b700 1 -- 172.21.15.112:0/1457501322 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592fea7a950 con 0x5592febdbe70 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678ed9d700 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678ed9d700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67780b2880 0x7f67780b4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6788009550 tx=0x7f67880092c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f678dd9b700 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febdf770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67840a6370 tx=0x7f6784009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a287534a3e4757 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f677cff9700 1 -- 172.21.15.112:0/1457501322 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784025120 con 0x5592febdbe70 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f677cff9700 1 -- 172.21.15.112:0/1457501322 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6784005120 con 0x5592febdbe70 2024-05-10T10:46:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.419+0000 7f677cff9700 1 -- 172.21.15.112:0/1457501322 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67840ae6e0 con 0x5592febdbe70 2024-05-10T10:46:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1457501322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 msgr2=0x5592febdf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febdf770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67840a6370 tx=0x7f6784009c30 comp rx=0 tx=0).stop 2024-05-10T10:46:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1457501322 shutdown_connections 2024-05-10T10:46:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febdb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1457501322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febdf770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1457501322 >> 172.21.15.112:0/1457501322 conn(0x5592feb07180 msgr2=0x5592febd5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.420+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1457501322 shutdown_connections 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1457501322 wait complete. 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 -- start start 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 Processor -- start 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febe7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febe44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592febe7cd0 con 0x5592febdfd00 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5592fea93cb0 con 0x5592febdb550 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592febe7e50 con 0x5592febdbe70 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f679f2b6b80 10 client.0 _ll_register_callbacks cb 0x5592feac59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:00.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f678e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febe7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.421+0000 7f678e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febe7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51026/0 (socket says 172.21.15.112:51026) 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678dd9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febe44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44552/0 (socket says 172.21.15.112:44552) 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678e59c700 1 -- 172.21.15.112:0/1546745747 learned_addr learned my addr 172.21.15.112:0/1546745747 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678dd9b700 1 -- 172.21.15.112:0/1546745747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 msgr2=0x5592febe7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678dd9b700 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febe7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678dd9b700 1 -- 172.21.15.112:0/1546745747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 msgr2=0x5592febe4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678dd9b700 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678dd9b700 1 -- 172.21.15.112:0/1546745747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592feac9710 con 0x5592febdbe70 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678e59c700 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febe7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f678dd9b700 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febe44d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67840e0ab0 tx=0x7f67840050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fbd41a2254c99b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f6775ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f6775ffb700 1 -- 172.21.15.112:0/1546745747 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67840aa9f0 con 0x5592febdbe70 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6784005120 con 0x5592febdbe70 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f6775ffb700 1 -- 172.21.15.112:0/1546745747 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f67840ae430 con 0x5592febdbe70 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f6775ffb700 1 -- 172.21.15.112:0/1546745747 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f67840ae6a0 con 0x5592febdbe70 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f679f2b6b80 10 client.26206 fetch_fsmap learned FSMap version 40 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.422+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5592febe7cd0 con 0x5592febdbe70 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f6775ffb700 1 -- 172.21.15.112:0/1546745747 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6784005120 con 0x5592febdbe70 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f6775ffb700 1 -- 172.21.15.112:0/1546745747 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f67840b2c30 con 0x5592febdbe70 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f679f2b6b80 10 client.26206 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f679f2b6b80 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f679f2b6b80 20 client.26206 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592febf6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f679f2b6b80 4 client.26206 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.423+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592feba76d0 con 0x5592febf4890 2024-05-10T10:46:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.424+0000 7f678e59c700 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592febf6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.424+0000 7f678e59c700 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592febf6cd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=911f8a3016d6e247 server_cookie=c8e3510664316cd in_seq=0 out_seq=0 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.424+0000 7f6775ffb700 10 client.26206 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f6775ffb700 1 -- 172.21.15.112:0/1546745747 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592feba76d0 con 0x5592febf4890 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f6775ffb700 10 client.26206 handle_command_reply: tid=0 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 client.26206 shutdown 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 msgr2=0x5592febe44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febe44d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67840e0ab0 tx=0x7f67840050b0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 shutdown_connections 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592febf6cd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febe4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f6775ffb700 0 client.26206 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdb550 0x5592febe7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/1546745747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdbe70 0x5592febe44d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 >> 172.21.15.112:0/1546745747 conn(0x5592feb07180 msgr2=0x5592febd07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 shutdown_connections 2024-05-10T10:46:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.425+0000 7f679f2b6b80 1 -- 172.21.15.112:0/1546745747 wait complete. 2024-05-10T10:46:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.426+0000 7f679f2b6b80 20 client.26206 trim_cache size 0 max 16384 2024-05-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 Processor -- start 2024-05-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 -- start start 2024-05-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febf0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdbe70 0x5592febfc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febfcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67901b5640 con 0x5592febdfd00 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67901b71e0 con 0x5592febdbe70 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67901b5d10 con 0x5592febdb550 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febfcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febfcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44564/0 (socket says 172.21.15.112:44564) 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f678ed9d700 1 -- 172.21.15.112:0/3312874833 learned_addr learned my addr 172.21.15.112:0/3312874833 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f678e59c700 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febf0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.450+0000 7f678dd9b700 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdbe70 0x5592febfc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f678e59c700 1 -- 172.21.15.112:0/3312874833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdbe70 msgr2=0x5592febfc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f678e59c700 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdbe70 0x5592febfc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f678e59c700 1 -- 172.21.15.112:0/3312874833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 msgr2=0x5592febfcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f678e59c700 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febfcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f678e59c700 1 -- 172.21.15.112:0/3312874833 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592febd75a0 con 0x5592febdb550 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f678ed9d700 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febfcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f678e59c700 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febf0d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5592febf74e0 tx=0x7f6780009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4f7ef6d2f6ec51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f6777fff700 1 -- 172.21.15.112:0/3312874833 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780004220 con 0x5592febdb550 2024-05-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f6777fff700 1 -- 172.21.15.112:0/3312874833 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6780025070 con 0x5592febdb550 2024-05-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.451+0000 7f6777fff700 1 -- 172.21.15.112:0/3312874833 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67800053a0 con 0x5592febdb550 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 -- 172.21.15.112:0/3312874833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 msgr2=0x5592febf0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febf0d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5592febf74e0 tx=0x7f6780009310 comp rx=0 tx=0).stop 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 -- 172.21.15.112:0/3312874833 shutdown_connections 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdfd00 0x5592febfcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febf0d20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/3312874833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdbe70 0x5592febfc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 -- 172.21.15.112:0/3312874833 >> 172.21.15.112:0/3312874833 conn(0x5592feb07180 msgr2=0x5592febd07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:00.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 -- 172.21.15.112:0/3312874833 shutdown_connections 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 -- 172.21.15.112:0/3312874833 wait complete. 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 -- start start 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 Processor -- start 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.452+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febe9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdbe70 0x5592febe98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f679f2b6b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdfd00 0x5592fec5e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f679f2b6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67901b6ae0 con 0x5592febdbe70 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdbe70 0x5592febe98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678e59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febe9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdbe70 0x5592febe98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44566/0 (socket says 172.21.15.112:44566) 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67901b6410 con 0x5592febdfd00 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67901b35e0 con 0x5592febdb550 2024-05-10T10:46:00.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678ed9d700 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdfd00 0x5592fec5e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678dd9b700 1 -- 172.21.15.112:0/2677272308 learned_addr learned my addr 172.21.15.112:0/2677272308 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f679f2b6b80 10 client.0 _ll_register_callbacks cb 0x5592feac59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678e59c700 1 -- 172.21.15.112:0/2677272308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdfd00 msgr2=0x5592fec5e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678e59c700 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdfd00 0x5592fec5e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678e59c700 1 -- 172.21.15.112:0/2677272308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdbe70 msgr2=0x5592febe98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678e59c700 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdbe70 0x5592febe98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678e59c700 1 -- 172.21.15.112:0/2677272308 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592fec5e690 con 0x5592febdb550 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.453+0000 7f678dd9b700 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdbe70 0x5592febe98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f6775ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f678e59c700 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febe9380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6780012410 tx=0x7f6780012180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5730134ecdc861de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f6775ffb700 1 -- 172.21.15.112:0/2677272308 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780004790 con 0x5592febdb550 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6780025070 con 0x5592febdb550 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f6775ffb700 1 -- 172.21.15.112:0/2677272308 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6780004930 con 0x5592febdb550 2024-05-10T10:46:00.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f6775ffb700 1 -- 172.21.15.112:0/2677272308 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f678003b050 con 0x5592febdb550 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f679f2b6b80 10 client.26212 fetch_fsmap learned FSMap version 40 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.454+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5592fec5f9a0 con 0x5592febdb550 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f6775ffb700 1 -- 172.21.15.112:0/2677272308 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6780025070 con 0x5592febdb550 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f6775ffb700 1 -- 172.21.15.112:0/2677272308 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f67800036a0 con 0x5592febdb550 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f679f2b6b80 10 client.26212 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f679f2b6b80 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f679f2b6b80 20 client.26212 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592fec62050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f679f2b6b80 4 client.26212 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5592feb3b2c0 con 0x5592febf4890 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.455+0000 7f678dd9b700 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592fec62050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.456+0000 7f678dd9b700 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592fec62050 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba254348e0392ac server_cookie=bb17863d9e815506 in_seq=0 out_seq=0 2024-05-10T10:46:00.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.456+0000 7f6775ffb700 10 client.26212 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f6775ffb700 1 -- 172.21.15.112:0/2677272308 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5592feb3b2c0 con 0x5592febf4890 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f6775ffb700 10 client.26212 handle_command_reply: tid=0 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 client.26212 shutdown 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 msgr2=0x5592febe9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febe9380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6780012410 tx=0x7f6780012180 comp rx=0 tx=0).stop 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 shutdown_connections 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5592febf4890 0x5592fec62050 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592febdfd00 0x5592fec5e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5592febdb550 0x5592febe9380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 --2- 172.21.15.112:0/2677272308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592febdbe70 0x5592febe98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 >> 172.21.15.112:0/2677272308 conn(0x5592feb07180 msgr2=0x5592febda360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.457+0000 7f6775ffb700 0 client.26212 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.458+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 shutdown_connections 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.458+0000 7f679f2b6b80 1 -- 172.21.15.112:0/2677272308 wait complete. 2024-05-10T10:46:00.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.458+0000 7f679f2b6b80 20 client.26212 trim_cache size 0 max 16384 2024-05-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "f30ce93b-670d-4eff-9cd5-0d32c06a901f", 2024-05-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.458+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67780b2880 msgr2=0x7f67780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.458+0000 7f6794a23700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67780b2880 0x7f67780b4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6788009550 tx=0x7f67880092c0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.458+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790102540 msgr2=0x7f6790103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.458+0000 7f6794a23700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790102540 0x7f6790103ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6784000c00 tx=0x7f678400f990 comp rx=0 tx=0).stop 2024-05-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 shutdown_connections 2024-05-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790107be0 0x7f6790104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f67780b2880 0x7f67780b4d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6790101b80 0x7f67901037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 --2- 172.21.15.112:0/4152944799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6790102540 0x7f6790103ce0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 >> 172.21.15.112:0/4152944799 conn(0x7f6790078a10 msgr2=0x7f6790106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 shutdown_connections 2024-05-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:00.459+0000 7f6794a23700 1 -- 172.21.15.112:0/4152944799 wait complete. 2024-05-10T10:46:00.816 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:46:00 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: 2024-05-10T10:46:00.485+0000 7ff3584cc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002bb [...2,head] /volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp/ auth{0=1,3=1,4=1} v5735 ap=2 f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1) n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2) (inest sync->mix g=0,3,4) caps={15294=pAsLsXsFs/-@1} | lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x5572a8cae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000002bb:[<0x10000000007/ltp v436>,<0x10000000006/xfstests-dev v72>,<0x10000000005/tmp v10>,<0x10000000003/client.0 v12>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v28>,<0x10000000001/sv_0 v52>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x100000002bb:[<0x10000000007/ltp v5735>,<0x10000000006/xfstests-dev v12369>,<0x10000000005/tmp v6802>,<0x10000000003/client.0 v537>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v545>,<0x10000000001/sv_0 v556>,<0x10000000000/qa v546>,<0x1/volumes v505>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T10:42:55.038905+0000 15=14+1)","ondisk_value.rstat":"n(v0 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","memory_value.dirstat":"f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1)","memory_value.rstat":"n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T10:46:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:00 smithi195 ceph-mon[33773]: pgmap v603: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 401 op/s 2024-05-10T10:46:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:00 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:46:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:00.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:00 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:00.817 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:00 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:00 smithi148 ceph-mon[33568]: pgmap v603: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 401 op/s 2024-05-10T10:46:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:00 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:46:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:00 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:00 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:00 smithi112 ceph-mon[30100]: pgmap v603: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 401 op/s 2024-05-10T10:46:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:00 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:46:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:00 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:00 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.436 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:46:01.662 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:01 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:01.774 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:46:02.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:01 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:02.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:46:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:46:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:01 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 -- 172.21.15.112:0/765027964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff350102a10 msgr2=0x7ff350102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 --2- 172.21.15.112:0/765027964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff350102a10 0x7ff350102e30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff344009900 tx=0x7ff34402a410 comp rx=0 tx=0).stop 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 -- 172.21.15.112:0/765027964 shutdown_connections 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 --2- 172.21.15.112:0/765027964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3501045c0 0x7ff35010ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 --2- 172.21.15.112:0/765027964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff350103c00 0x7ff350104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 --2- 172.21.15.112:0/765027964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff350102a10 0x7ff350102e30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 -- 172.21.15.112:0/765027964 >> 172.21.15.112:0/765027964 conn(0x7ff3500fdfa0 msgr2=0x7ff3501003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 -- 172.21.15.112:0/765027964 shutdown_connections 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 -- 172.21.15.112:0/765027964 wait complete. 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 Processor -- start 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.779+0000 7ff356eda640 1 -- start start 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff356eda640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff350102a10 0x7ff350198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff356eda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff350103c00 0x7ff350198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff356eda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3501045c0 0x7ff3501a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff356eda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff35010e3f0 con 0x7ff3501045c0 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff356eda640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff35010e270 con 0x7ff350103c00 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff356eda640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff35010e570 con 0x7ff350102a10 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3501045c0 0x7ff3501a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3501045c0 0x7ff3501a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44584/0 (socket says 172.21.15.112:44584) 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 -- 172.21.15.112:0/3110423206 learned_addr learned my addr 172.21.15.112:0/3110423206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff34ffff640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff350103c00 0x7ff350198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff354c76640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff350102a10 0x7ff350198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 -- 172.21.15.112:0/3110423206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff350102a10 msgr2=0x7ff350198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff350102a10 0x7ff350198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 -- 172.21.15.112:0/3110423206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff350103c00 msgr2=0x7ff350198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff350103c00 0x7ff350198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 -- 172.21.15.112:0/3110423206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3501a06d0 con 0x7ff3501045c0 2024-05-10T10:46:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff355477640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3501045c0 0x7ff3501a0060 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff34800eb10 tx=0x7ff34800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff34dffb640 1 -- 172.21.15.112:0/3110423206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348004b20 con 0x7ff3501045c0 2024-05-10T10:46:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff34dffb640 1 -- 172.21.15.112:0/3110423206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff348004cc0 con 0x7ff3501045c0 2024-05-10T10:46:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3501a09c0 con 0x7ff3501045c0 2024-05-10T10:46:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.780+0000 7ff34dffb640 1 -- 172.21.15.112:0/3110423206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348010690 con 0x7ff3501045c0 2024-05-10T10:46:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.781+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3501a0f10 con 0x7ff3501045c0 2024-05-10T10:46:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff34dffb640 1 -- 172.21.15.112:0/3110423206 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff348005020 con 0x7ff3501045c0 2024-05-10T10:46:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 Processor -- start 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 -- start start 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291691db0 0x55b2916921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b2916972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291697910 0x55b291699da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b2915f65d0 con 0x55b291691db0 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b2915af890 con 0x55b291697910 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff359bf9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b2915f2590 con 0x55b2916927a0 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291691db0 0x55b2916921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291691db0 0x55b2916921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44592/0 (socket says 172.21.15.112:44592) 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.783+0000 7ff34ffff640 1 -- 172.21.15.112:0/4175350259 learned_addr learned my addr 172.21.15.112:0/4175350259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff354c76640 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b2916972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34ffff640 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291697910 0x55b291699da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34dffb640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff3400b28a0 0x7ff3400b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34ffff640 1 -- 172.21.15.112:0/4175350259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 msgr2=0x55b2916972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34ffff640 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b2916972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff355477640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff3400b28a0 0x7ff3400b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34ffff640 1 -- 172.21.15.112:0/4175350259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291697910 msgr2=0x55b291699da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34ffff640 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291697910 0x55b291699da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34ffff640 1 -- 172.21.15.112:0/4175350259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b291308210 con 0x55b291691db0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34dffb640 1 -- 172.21.15.112:0/3110423206 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff3480e2bb0 con 0x7ff3501045c0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff34ffff640 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291691db0 0x55b2916921d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0137f0 tx=0x7ff33c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1462d4edf787ad5c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff33affd640 1 -- 172.21.15.112:0/4175350259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c013d70 con 0x55b291691db0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff355477640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff3400b28a0 0x7ff3400b4d60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3480049d0 tx=0x7ff34806f380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff33affd640 1 -- 172.21.15.112:0/4175350259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff33c0044c0 con 0x55b291691db0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.784+0000 7ff33affd640 1 -- 172.21.15.112:0/4175350259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c004a00 con 0x55b291691db0 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 -- 172.21.15.112:0/4175350259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291691db0 msgr2=0x55b2916921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291691db0 0x55b2916921d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0137f0 tx=0x7ff33c0097a0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 -- 172.21.15.112:0/4175350259 shutdown_connections 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291697910 0x55b291699da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b2916972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 --2- 172.21.15.112:0/4175350259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291691db0 0x55b2916921d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 -- 172.21.15.112:0/4175350259 >> 172.21.15.112:0/4175350259 conn(0x55b29168d420 msgr2=0x55b29168d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 -- 172.21.15.112:0/4175350259 shutdown_connections 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 -- 172.21.15.112:0/4175350259 wait complete. 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 -- start start 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.785+0000 7ff359bf9740 1 Processor -- start 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291691db0 0x55b29169bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b29169c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff359bf9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b29169a860 con 0x55b291697910 2024-05-10T10:46:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff359bf9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b2915b3370 con 0x55b291691db0 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff359bf9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b29169a9e0 con 0x55b2916927a0 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff359bf9740 10 client.0 _ll_register_callbacks cb 0x55b291307a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44600/0 (socket says 172.21.15.112:44600) 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 -- 172.21.15.112:0/3519994745 learned_addr learned my addr 172.21.15.112:0/3519994745 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291691db0 0x55b29169bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff354c76640 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b29169c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 -- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 msgr2=0x55b29169c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b29169c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 -- 172.21.15.112:0/3519994745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291691db0 msgr2=0x55b29169bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291691db0 0x55b29169bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 -- 172.21.15.112:0/3519994745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2913d4620 con 0x55b291697910 2024-05-10T10:46:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff34ffff640 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a34e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009920 tx=0x7ff33c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec786923f8cf07 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.786+0000 7ff32ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff32ffff640 1 -- 172.21.15.112:0/3519994745 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c011b00 con 0x55b291697910 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff32ffff640 1 -- 172.21.15.112:0/3519994745 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff33c011ca0 con 0x55b291697910 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2916041e0 con 0x55b291697910 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff32ffff640 1 -- 172.21.15.112:0/3519994745 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff33c033050 con 0x55b291697910 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff359bf9740 10 client.16830 fetch_fsmap learned FSMap version 40 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b2916a64a0 con 0x55b291697910 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff32ffff640 1 -- 172.21.15.112:0/3519994745 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b2916041e0 con 0x55b291697910 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.787+0000 7ff32ffff640 1 -- 172.21.15.112:0/3519994745 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7ff33c01f3d0 con 0x55b291697910 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff359bf9740 10 client.16830 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff359bf9740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff359bf9740 20 client.16830 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff359bf9740 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b2916a7a50 0x55b2916a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff359bf9740 4 client.16830 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2915fe050 con 0x55b2916a7a50 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff354c76640 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b2916a7a50 0x55b2916a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff354c76640 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b2916a7a50 0x55b2916a9e90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faea93d8117a17f3 server_cookie=ebb142e3d67bc5ce in_seq=0 out_seq=0 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.788+0000 7ff32ffff640 10 client.16830 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff32ffff640 1 -- 172.21.15.112:0/3519994745 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2915fe050 con 0x55b2916a7a50 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff32ffff640 10 client.16830 handle_command_reply: tid=0 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 client.16830 shutdown 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 msgr2=0x55b2916a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a34e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009920 tx=0x7ff33c01c040 comp rx=0 tx=0).stop 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 shutdown_connections 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a34e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b2916927a0 0x55b29169c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b2916a7a50 0x55b2916a9e90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff32ffff640 0 client.16830 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 --2- 172.21.15.112:0/3519994745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b291691db0 0x55b29169bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 >> 172.21.15.112:0/3519994745 conn(0x55b29168d420 msgr2=0x55b29168c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 shutdown_connections 2024-05-10T10:46:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.789+0000 7ff359bf9740 1 -- 172.21.15.112:0/3519994745 wait complete. 2024-05-10T10:46:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.790+0000 7ff359bf9740 20 client.16830 trim_cache size 0 max 16384 2024-05-10T10:46:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 Processor -- start 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 -- start start 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b29168cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b291690810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b291690d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ab470 con 0x55b291697910 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ad010 con 0x55b2916927a0 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff359bf9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501abb40 con 0x55b291691db0 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b291690d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b291690810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b29168cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b29168cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60590/0 (socket says 172.21.15.112:60590) 2024-05-10T10:46:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b291690810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51110/0 (socket says 172.21.15.112:51110) 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff34ffff640 1 -- 172.21.15.112:0/1808340828 learned_addr learned my addr 172.21.15.112:0/1808340828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 -- 172.21.15.112:0/1808340828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 msgr2=0x55b29168cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b29168cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 -- 172.21.15.112:0/1808340828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 msgr2=0x55b291690d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b291690d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 -- 172.21.15.112:0/1808340828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29168d240 con 0x55b2916927a0 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff34ffff640 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b29168cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.809+0000 7ff354c76640 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b291690810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff34402a930 tx=0x7ff344005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314fe44222e676f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff33a7fc640 1 -- 172.21.15.112:0/1808340828 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34404b070 con 0x55b2916927a0 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff33a7fc640 1 -- 172.21.15.112:0/1808340828 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff344031c50 con 0x55b2916927a0 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff33a7fc640 1 -- 172.21.15.112:0/1808340828 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344035040 con 0x55b2916927a0 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 -- 172.21.15.112:0/1808340828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 msgr2=0x55b291690810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b291690810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff34402a930 tx=0x7ff344005a20 comp rx=0 tx=0).stop 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 -- 172.21.15.112:0/1808340828 shutdown_connections 2024-05-10T10:46:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b291690d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b291690810 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 --2- 172.21.15.112:0/1808340828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b29168cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 -- 172.21.15.112:0/1808340828 >> 172.21.15.112:0/1808340828 conn(0x55b29168d420 msgr2=0x55b291690100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.810+0000 7ff359bf9740 1 -- 172.21.15.112:0/1808340828 shutdown_connections 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 -- 172.21.15.112:0/1808340828 wait complete. 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 -- start start 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 Processor -- start 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b2916a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b2916a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ac910 con 0x55b291697910 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ac240 con 0x55b2916927a0 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501a9450 con 0x55b291691db0 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff359bf9740 10 client.0 _ll_register_callbacks cb 0x7ff33c009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b2916a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff354c76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b2916a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44616/0 (socket says 172.21.15.112:44616) 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.811+0000 7ff34ffff640 1 -- 172.21.15.112:0/2531722736 learned_addr learned my addr 172.21.15.112:0/2531722736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff34ffff640 1 -- 172.21.15.112:0/2531722736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 msgr2=0x55b2916a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff34ffff640 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b2916a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff34ffff640 1 -- 172.21.15.112:0/2531722736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 msgr2=0x55b2916a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff34ffff640 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b2916a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff34ffff640 1 -- 172.21.15.112:0/2531722736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2916a1c80 con 0x55b291697910 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff354c76640 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b2916a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff34ffff640 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a1690 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff33c011820 tx=0x7ff33c013ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacc75437f1f32fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff32ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff32ffff640 1 -- 172.21.15.112:0/2531722736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c005500 con 0x55b291697910 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff32ffff640 1 -- 172.21.15.112:0/2531722736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff33c0056a0 con 0x55b291697910 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff344031c50 con 0x55b291697910 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff32ffff640 1 -- 172.21.15.112:0/2531722736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff33c013030 con 0x55b291697910 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff359bf9740 10 client.16836 fetch_fsmap learned FSMap version 40 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b291712310 con 0x55b291697910 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.812+0000 7ff32ffff640 1 -- 172.21.15.112:0/2531722736 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff344031c50 con 0x55b291697910 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff32ffff640 1 -- 172.21.15.112:0/2531722736 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7ff33c01fbe0 con 0x55b291697910 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff359bf9740 10 client.16836 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff359bf9740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff359bf9740 20 client.16836 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff359bf9740 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b291712b50 0x55b291714f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff359bf9740 4 client.16836 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:46:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2915795f0 con 0x55b291712b50 2024-05-10T10:46:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff354c76640 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b291712b50 0x55b291714f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff354c76640 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b291712b50 0x55b291714f70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29c2a5d6ad489de server_cookie=f93841c9d3f1342a in_seq=0 out_seq=0 2024-05-10T10:46:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.813+0000 7ff32ffff640 10 client.16836 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff32ffff640 1 -- 172.21.15.112:0/2531722736 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2915795f0 con 0x55b291712b50 2024-05-10T10:46:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff32ffff640 10 client.16836 handle_command_reply: tid=0 2024-05-10T10:46:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 client.16836 shutdown 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 msgr2=0x55b2916a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a1690 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff33c011820 tx=0x7ff33c013ed0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 shutdown_connections 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b291697910 0x55b2916a1690 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b291712b50 0x55b291714f70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2916927a0 0x55b2916a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 --2- 172.21.15.112:0/2531722736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b291691db0 0x55b2916a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff32ffff640 0 client.16836 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 >> 172.21.15.112:0/2531722736 conn(0x55b29168d420 msgr2=0x55b291690990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 shutdown_connections 2024-05-10T10:46:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.828+0000 7ff359bf9740 1 -- 172.21.15.112:0/2531722736 wait complete. 2024-05-10T10:46:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff359bf9740 20 client.16836 trim_cache size 0 max 16384 2024-05-10T10:46:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:46:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:46:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:46:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:46:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff3400b28a0 msgr2=0x7ff3400b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff3400b28a0 0x7ff3400b4d60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3480049d0 tx=0x7ff34806f380 comp rx=0 tx=0).stop 2024-05-10T10:46:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3501045c0 msgr2=0x7ff3501a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3501045c0 0x7ff3501a0060 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff34800eb10 tx=0x7ff34800c2d0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 shutdown_connections 2024-05-10T10:46:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff3400b28a0 0x7ff3400b4d60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3501045c0 0x7ff3501a0060 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff350103c00 0x7ff350198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 --2- 172.21.15.112:0/3110423206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff350102a10 0x7ff350198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 >> 172.21.15.112:0/3110423206 conn(0x7ff3500fdfa0 msgr2=0x7ff350106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 shutdown_connections 2024-05-10T10:46:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:02.829+0000 7ff356eda640 1 -- 172.21.15.112:0/3110423206 wait complete. 2024-05-10T10:46:02.924 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63ee1da 2024-05-10T10:46:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:02 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:02 smithi148 ceph-mon[33568]: pgmap v604: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.4 MiB/s wr, 612 op/s 2024-05-10T10:46:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:02 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:02 smithi195 ceph-mon[33773]: pgmap v604: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.4 MiB/s wr, 612 op/s 2024-05-10T10:46:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:02 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:46:03.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:02 smithi112 ceph-mon[30100]: pgmap v604: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.4 MiB/s wr, 612 op/s 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 -- 172.21.15.112:0/816157153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 msgr2=0x7ff7a41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 --2- 172.21.15.112:0/816157153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a41098e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009990 tx=0x7ff7a0009ca0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 -- 172.21.15.112:0/816157153 shutdown_connections 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 --2- 172.21.15.112:0/816157153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a406c3d0 0x7ff7a4110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 --2- 172.21.15.112:0/816157153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a41098e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 --2- 172.21.15.112:0/816157153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406ba10 0x7ff7a406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 -- 172.21.15.112:0/816157153 >> 172.21.15.112:0/816157153 conn(0x7ff7a4079310 msgr2=0x7ff7a4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 -- 172.21.15.112:0/816157153 shutdown_connections 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.747+0000 7ff7abfda700 1 -- 172.21.15.112:0/816157153 wait complete. 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 Processor -- start 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 -- start start 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a406ba10 0x7ff7a4102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406c3d0 0x7ff7a41029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a4102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7aa577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a4102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7a9575700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406c3d0 0x7ff7a41029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41134c0 con 0x7ff7a4109500 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4113340 con 0x7ff7a406c3d0 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7aa577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a4102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44636/0 (socket says 172.21.15.112:44636) 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7aa577700 1 -- 172.21.15.112:0/4086846037 learned_addr learned my addr 172.21.15.112:0/4086846037 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7a9575700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406c3d0 0x7ff7a41029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51140/0 (socket says 172.21.15.112:51140) 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.748+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4113640 con 0x7ff7a406ba10 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a406ba10 0x7ff7a4102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7a9575700 1 -- 172.21.15.112:0/4086846037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a406ba10 msgr2=0x7ff7a4102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7a9575700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a406ba10 0x7ff7a4102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7a9575700 1 -- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 msgr2=0x7ff7a4102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7a9575700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a4102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7a9575700 1 -- 172.21.15.112:0/4086846037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a41a7fa0 con 0x7ff7a406c3d0 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7aa577700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7a9575700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406c3d0 0x7ff7a41029a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9f0 tx=0x7ff79400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff79affd700 1 -- 172.21.15.112:0/4086846037 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79400cbd0 con 0x7ff7a406c3d0 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a41a8290 con 0x7ff7a406c3d0 2024-05-10T10:46:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff79affd700 1 -- 172.21.15.112:0/4086846037 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff79400cd70 con 0x7ff7a406c3d0 2024-05-10T10:46:03.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff79affd700 1 -- 172.21.15.112:0/4086846037 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794018680 con 0x7ff7a406c3d0 2024-05-10T10:46:03.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.749+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a8820 con 0x7ff7a406c3d0 2024-05-10T10:46:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.752+0000 7ff79affd700 1 -- 172.21.15.112:0/4086846037 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff794010430 con 0x7ff7a406c3d0 2024-05-10T10:46:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.753+0000 7ff79affd700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.753+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.753+0000 7ff79affd700 1 -- 172.21.15.112:0/4086846037 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff7940e2770 con 0x7ff7a406c3d0 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.753+0000 7ff7b686db80 1 Processor -- start 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7b686db80 1 -- start start 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006010 tx=0x7ff7a0005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7b686db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646c7401940 con 0x5646c74afd50 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7b686db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5646c7366da0 con 0x5646c74b51b0 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7b686db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5646c73fd900 con 0x5646c74b5ad0 2024-05-10T10:46:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44648/0 (socket says 172.21.15.112:44648) 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 -- 172.21.15.112:0/1204716701 learned_addr learned my addr 172.21.15.112:0/1204716701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 -- 172.21.15.112:0/1204716701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 msgr2=0x5646c74af810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74af810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 -- 172.21.15.112:0/1204716701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 msgr2=0x5646c74b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.754+0000 7ff7aa577700 1 -- 172.21.15.112:0/1204716701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c734e8d0 con 0x5646c74afd50 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.755+0000 7ff7aa577700 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74b0190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0078d0 tx=0x7ff79c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5a8ca7a62764d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.755+0000 7ff78ffff700 1 -- 172.21.15.112:0/1204716701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c019030 con 0x5646c74afd50 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.755+0000 7ff78ffff700 1 -- 172.21.15.112:0/1204716701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff79c015650 con 0x5646c74afd50 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.755+0000 7ff78ffff700 1 -- 172.21.15.112:0/1204716701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c004280 con 0x5646c74afd50 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 -- 172.21.15.112:0/1204716701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 msgr2=0x5646c74b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74b0190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0078d0 tx=0x7ff79c009230 comp rx=0 tx=0).stop 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 -- 172.21.15.112:0/1204716701 shutdown_connections 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74b0190 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1204716701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 -- 172.21.15.112:0/1204716701 >> 172.21.15.112:0/1204716701 conn(0x5646c73dbb70 msgr2=0x5646c74aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 -- 172.21.15.112:0/1204716701 shutdown_connections 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 -- 172.21.15.112:0/1204716701 wait complete. 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 -- start start 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.756+0000 7ff7b686db80 1 Processor -- start 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74afd50 0x5646c74bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7b686db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646c74bc450 con 0x5646c74b5ad0 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7b686db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5646c73baed0 con 0x5646c74b51b0 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7b686db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5646c74bc5d0 con 0x5646c74afd50 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7b686db80 10 client.0 _ll_register_callbacks cb 0x5646c73955b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7aa577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7a9575700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7a9575700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51160/0 (socket says 172.21.15.112:51160) 2024-05-10T10:46:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7a9575700 1 -- 172.21.15.112:0/980947528 learned_addr learned my addr 172.21.15.112:0/980947528 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74afd50 0x5646c74bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7aa577700 1 -- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74afd50 msgr2=0x5646c74bb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7aa577700 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74afd50 0x5646c74bb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7aa577700 1 -- 172.21.15.112:0/980947528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 msgr2=0x5646c74b8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7aa577700 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7aa577700 1 -- 172.21.15.112:0/980947528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c73b8760 con 0x5646c74b5ad0 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.757+0000 7ff7a9575700 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7aa577700 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74b9050 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff79c015f90 tx=0x7ff79c015a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705b01518bfcd18d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff78d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff78d7fa700 1 -- 172.21.15.112:0/980947528 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c022070 con 0x5646c74b5ad0 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c015650 con 0x5646c74b5ad0 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff78d7fa700 1 -- 172.21.15.112:0/980947528 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff79c01e880 con 0x5646c74b5ad0 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff78d7fa700 1 -- 172.21.15.112:0/980947528 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff79c02e050 con 0x5646c74b5ad0 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7b686db80 10 client.16848 fetch_fsmap learned FSMap version 40 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5646c74c6d30 con 0x5646c74b5ad0 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff78d7fa700 1 -- 172.21.15.112:0/980947528 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff79c015650 con 0x5646c74b5ad0 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff78d7fa700 1 -- 172.21.15.112:0/980947528 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7ff79c0053e0 con 0x5646c74b5ad0 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7b686db80 10 client.16848 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T10:46:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7b686db80 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7b686db80 20 client.16848 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.758+0000 7ff7b686db80 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c74c9260 0x5646c74cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.759+0000 7ff7b686db80 4 client.16848 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.759+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c740f320 con 0x5646c74c9260 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.759+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c74c9260 0x5646c74cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.759+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c74c9260 0x5646c74cb650 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0d8f7c8c588dea server_cookie=11aefaa9a6dc992 in_seq=0 out_seq=0 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.759+0000 7ff78d7fa700 10 client.16848 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.760+0000 7ff78d7fa700 1 -- 172.21.15.112:0/980947528 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646c740f320 con 0x5646c74c9260 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.760+0000 7ff78d7fa700 10 client.16848 handle_command_reply: tid=0 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.760+0000 7ff7b686db80 1 client.16848 shutdown 2024-05-10T10:46:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.760+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 msgr2=0x5646c74b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.760+0000 7ff7b686db80 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74b9050 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff79c015f90 tx=0x7ff79c015a30 comp rx=0 tx=0).stop 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.760+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 shutdown_connections 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.760+0000 7ff7b686db80 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c74c9260 0x5646c74cb650 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff7b686db80 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74b9050 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff7b686db80 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74afd50 0x5646c74bb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff7b686db80 1 --2- 172.21.15.112:0/980947528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 >> 172.21.15.112:0/980947528 conn(0x5646c73dbb70 msgr2=0x5646c74a4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff78d7fa700 0 client.16848 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 shutdown_connections 2024-05-10T10:46:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff7b686db80 1 -- 172.21.15.112:0/980947528 wait complete. 2024-05-10T10:46:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.761+0000 7ff7b686db80 20 client.16848 trim_cache size 0 max 16384 2024-05-10T10:46:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 Processor -- start 2024-05-10T10:46:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 -- start start 2024-05-10T10:46:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b1490 con 0x5646c74afd50 2024-05-10T10:46:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b3030 con 0x5646c74b51b0 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.783+0000 7ff7b686db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b1b60 con 0x5646c74b5ad0 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44660/0 (socket says 172.21.15.112:44660) 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 -- 172.21.15.112:0/3043477067 learned_addr learned my addr 172.21.15.112:0/3043477067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7aa577700 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9575700 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 -- 172.21.15.112:0/3043477067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 msgr2=0x5646c74a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 -- 172.21.15.112:0/3043477067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 msgr2=0x5646c74b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 -- 172.21.15.112:0/3043477067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c74bc010 con 0x5646c74afd50 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7aa577700 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9575700 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74af810 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0005850 tx=0x7ff7a0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5db039ac42a7599 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff78f7fe700 1 -- 172.21.15.112:0/3043477067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0003680 con 0x5646c74afd50 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff78f7fe700 1 -- 172.21.15.112:0/3043477067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7a0003cb0 con 0x5646c74afd50 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.784+0000 7ff78f7fe700 1 -- 172.21.15.112:0/3043477067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000fa00 con 0x5646c74afd50 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 -- 172.21.15.112:0/3043477067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 msgr2=0x5646c74af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74af810 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0005850 tx=0x7ff7a0020000 comp rx=0 tx=0).stop 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 -- 172.21.15.112:0/3043477067 shutdown_connections 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b5ad0 0x5646c74a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74afd50 0x5646c74af810 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 --2- 172.21.15.112:0/3043477067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74b51b0 0x5646c74b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 -- 172.21.15.112:0/3043477067 >> 172.21.15.112:0/3043477067 conn(0x5646c73dbb70 msgr2=0x5646c74b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 -- 172.21.15.112:0/3043477067 shutdown_connections 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 -- 172.21.15.112:0/3043477067 wait complete. 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.785+0000 7ff7b686db80 1 -- start start 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 1 Processor -- start 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74afd50 0x5646c74ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b51b0 0x5646c74cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b2930 con 0x5646c74b5ad0 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b2260 con 0x5646c74afd50 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41af430 con 0x5646c74b51b0 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7b686db80 10 client.0 _ll_register_callbacks cb 0x5646c73955b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7aa577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7a9575700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b51b0 0x5646c74cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7a9d76700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74afd50 0x5646c74ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7aa577700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44674/0 (socket says 172.21.15.112:44674) 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7aa577700 1 -- 172.21.15.112:0/1686223662 learned_addr learned my addr 172.21.15.112:0/1686223662 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:03.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7aa577700 1 -- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b51b0 msgr2=0x5646c74cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.786+0000 7ff7aa577700 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b51b0 0x5646c74cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7aa577700 1 -- 172.21.15.112:0/1686223662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74afd50 msgr2=0x5646c74ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7aa577700 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74afd50 0x5646c74ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7aa577700 1 -- 172.21.15.112:0/1686223662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c74cc480 con 0x5646c74b5ad0 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7a9575700 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b51b0 0x5646c74cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7aa577700 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74cbe90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000f80 tx=0x7ff79c009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41e601c7bf91346 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff78d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff78d7fa700 1 -- 172.21.15.112:0/1686223662 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c02b070 con 0x5646c74b5ad0 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff78d7fa700 1 -- 172.21.15.112:0/1686223662 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff79c005be0 con 0x5646c74b5ad0 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a0003cb0 con 0x5646c74b5ad0 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff78d7fa700 1 -- 172.21.15.112:0/1686223662 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff79c03a050 con 0x5646c74b5ad0 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7b686db80 10 client.16854 fetch_fsmap learned FSMap version 40 2024-05-10T10:46:03.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5646c7533570 con 0x5646c74b5ad0 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff78d7fa700 1 -- 172.21.15.112:0/1686223662 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7a0003cb0 con 0x5646c74b5ad0 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.787+0000 7ff78d7fa700 1 -- 172.21.15.112:0/1686223662 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7ff79c005000 con 0x5646c74b5ad0 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7b686db80 10 client.16854 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7b686db80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7b686db80 20 client.16854 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c7533b60 0x5646c7535f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7b686db80 4 client.16854 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646c74093c0 con 0x5646c7533b60 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c7533b60 0x5646c7535f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff78d7fa700 10 client.16854 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:03.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7a9d76700 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c7533b60 0x5646c7535f80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6ce376a5e0032f server_cookie=fceed5ac5464def2 in_seq=0 out_seq=0 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff78d7fa700 1 -- 172.21.15.112:0/1686223662 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5646c74093c0 con 0x5646c7533b60 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff78d7fa700 10 client.16854 handle_command_reply: tid=0 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.788+0000 7ff7b686db80 1 client.16854 shutdown 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 msgr2=0x5646c74cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74cbe90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000f80 tx=0x7ff79c009660 comp rx=0 tx=0).stop 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 shutdown_connections 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5646c7533b60 0x5646c7535f80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646c74b5ad0 0x5646c74cbe90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646c74afd50 0x5646c74ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 --2- 172.21.15.112:0/1686223662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5646c74b51b0 0x5646c74cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff78d7fa700 0 client.16854 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:03.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 >> 172.21.15.112:0/1686223662 conn(0x5646c73dbb70 msgr2=0x5646c74b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 shutdown_connections 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 1 -- 172.21.15.112:0/1686223662 wait complete. 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.789+0000 7ff7b686db80 20 client.16854 trim_cache size 0 max 16384 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stdout: "f30ce93b-670d-4eff-9cd5-0d32c06a901f": { 2024-05-10T10:46:03.800 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-10T10:46:03.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "f30ce93b-670d-4eff-9cd5-0d32c06a901f", 2024-05-10T10:46:03.801 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-10T10:46:03.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:46:03.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:46:03.801 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:46:03.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.792+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 msgr2=0x7ff7900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.792+0000 7ff7abfda700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006010 tx=0x7ff7a0005c40 comp rx=0 tx=0).stop 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.792+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406c3d0 msgr2=0x7ff7a41029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.792+0000 7ff7abfda700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406c3d0 0x7ff7a41029a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9f0 tx=0x7ff79400ed00 comp rx=0 tx=0).stop 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.793+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 shutdown_connections 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.793+0000 7ff7abfda700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7a4109500 0x7ff7a4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.793+0000 7ff7abfda700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7ff7900b2880 0x7ff7900b4d40 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006010 tx=0x7ff7a0005c40 comp rx=0 tx=0).stop 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.793+0000 7ff7abfda700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a406ba10 0x7ff7a4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.793+0000 7ff7abfda700 1 --2- 172.21.15.112:0/4086846037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a406c3d0 0x7ff7a41029a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.793+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 >> 172.21.15.112:0/4086846037 conn(0x7ff7a4079310 msgr2=0x7ff7a4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.795+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 shutdown_connections 2024-05-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:03.795+0000 7ff7abfda700 1 -- 172.21.15.112:0/4086846037 wait complete. 2024-05-10T10:46:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:04 smithi148 ceph-mon[33568]: pgmap v605: 129 pgs: 129 active+clean; 80 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 565 op/s 2024-05-10T10:46:05.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:04 smithi195 ceph-mon[33773]: pgmap v605: 129 pgs: 129 active+clean; 80 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 565 op/s 2024-05-10T10:46:05.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:04 smithi112 ceph-mon[30100]: pgmap v605: 129 pgs: 129 active+clean; 80 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 565 op/s 2024-05-10T10:46:05.331 INFO:tasks.cephfs.filesystem:scrub status for tag:f30ce93b-670d-4eff-9cd5-0d32c06a901f - {'path': '/', 'tag': 'f30ce93b-670d-4eff-9cd5-0d32c06a901f', 'options': 'recursive,force'} 2024-05-10T10:46:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:06 smithi148 ceph-mon[33568]: pgmap v606: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.1 MiB/s wr, 1.01k op/s 2024-05-10T10:46:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:06 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:46:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:06 smithi195 ceph-mon[33773]: pgmap v606: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.1 MiB/s wr, 1.01k op/s 2024-05-10T10:46:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:06 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:46:07.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:06 smithi112 ceph-mon[30100]: pgmap v606: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.1 MiB/s wr, 1.01k op/s 2024-05-10T10:46:07.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:06 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:46:09.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:08 smithi148 ceph-mon[33568]: pgmap v607: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 907 op/s 2024-05-10T10:46:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:08 smithi195 ceph-mon[33773]: pgmap v607: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 907 op/s 2024-05-10T10:46:09.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:08 smithi112 ceph-mon[30100]: pgmap v607: 129 pgs: 129 active+clean; 62 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 907 op/s 2024-05-10T10:46:11.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:10 smithi148 ceph-mon[33568]: pgmap v608: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 946 op/s 2024-05-10T10:46:11.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:10 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:46:11.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:10 smithi195 ceph-mon[33773]: pgmap v608: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 946 op/s 2024-05-10T10:46:11.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:10 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:46:11.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:10 smithi112 ceph-mon[30100]: pgmap v608: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 946 op/s 2024-05-10T10:46:11.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:10 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:46:13.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:12 smithi148 ceph-mon[33568]: pgmap v609: 129 pgs: 129 active+clean; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.2 MiB/s wr, 1.11k op/s 2024-05-10T10:46:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:12 smithi195 ceph-mon[33773]: pgmap v609: 129 pgs: 129 active+clean; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.2 MiB/s wr, 1.11k op/s 2024-05-10T10:46:13.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:12 smithi112 ceph-mon[30100]: pgmap v609: 129 pgs: 129 active+clean; 73 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.2 MiB/s wr, 1.11k op/s 2024-05-10T10:46:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:14 smithi148 ceph-mon[33568]: pgmap v610: 129 pgs: 129 active+clean; 72 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.7 MiB/s wr, 954 op/s 2024-05-10T10:46:15.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:14 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:46:15.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:14 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:46:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:14 smithi195 ceph-mon[33773]: pgmap v610: 129 pgs: 129 active+clean; 72 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.7 MiB/s wr, 954 op/s 2024-05-10T10:46:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:14 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:46:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:14 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:46:15.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:14 smithi112 ceph-mon[30100]: pgmap v610: 129 pgs: 129 active+clean; 72 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.7 MiB/s wr, 954 op/s 2024-05-10T10:46:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:14 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:46:15.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:14 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:46:15.599 INFO:tasks.workunit.client.0.smithi112.stdout:10000 write 0x4c35901 thru 0x4c3d40f (0x7b0f bytes) 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 -- 172.21.15.112:0/1695601315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65241034c0 msgr2=0x7f6524109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 --2- 172.21.15.112:0/1695601315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65241034c0 0x7f6524109d50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f651c009ca0 tx=0x7f651c02a8c0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 -- 172.21.15.112:0/1695601315 shutdown_connections 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 --2- 172.21.15.112:0/1695601315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65241034c0 0x7f6524109d50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 --2- 172.21.15.112:0/1695601315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6524102b00 0x7f6524102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 --2- 172.21.15.112:0/1695601315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6524101910 0x7f6524101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 -- 172.21.15.112:0/1695601315 >> 172.21.15.112:0/1695601315 conn(0x7f65240fce80 msgr2=0x7f65240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 -- 172.21.15.112:0/1695601315 shutdown_connections 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.940+0000 7f652b578640 1 -- 172.21.15.112:0/1695601315 wait complete. 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 Processor -- start 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 -- start start 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6524101910 0x7f6524197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6524102b00 0x7f6524197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65241034c0 0x7f652419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f652410d320 con 0x7f6524101910 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f652410d1a0 con 0x7f6524102b00 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f652b578640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f652410d4a0 con 0x7f65241034c0 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f6529314640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6524101910 0x7f6524197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f6529314640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6524101910 0x7f6524197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53650/0 (socket says 172.21.15.112:53650) 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f6529314640 1 -- 172.21.15.112:0/3437728756 learned_addr learned my addr 172.21.15.112:0/3437728756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f6529b15640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65241034c0 0x7f652419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.941+0000 7f6528b13640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6524102b00 0x7f6524197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f6529314640 1 -- 172.21.15.112:0/3437728756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65241034c0 msgr2=0x7f652419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f6529314640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65241034c0 0x7f652419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f6529314640 1 -- 172.21.15.112:0/3437728756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6524102b00 msgr2=0x7f6524197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f6529314640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6524102b00 0x7f6524197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f6529314640 1 -- 172.21.15.112:0/3437728756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652419f5b0 con 0x7f6524101910 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f6528b13640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6524102b00 0x7f6524197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f6529314640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6524101910 0x7f6524197990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f652000e9c0 tx=0x7f652000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f651a7fc640 1 -- 172.21.15.112:0/3437728756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652000cd80 con 0x7f6524101910 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f651a7fc640 1 -- 172.21.15.112:0/3437728756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65200044b0 con 0x7f6524101910 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652419f8a0 con 0x7f6524101910 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f651a7fc640 1 -- 172.21.15.112:0/3437728756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520010430 con 0x7f6524101910 2024-05-10T10:46:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.942+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652419fe40 con 0x7f6524101910 2024-05-10T10:46:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.944+0000 7f652e369740 1 Processor -- start 2024-05-10T10:46:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f652e369740 1 -- start start 2024-05-10T10:46:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f652e369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f652e369740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c08540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f652e369740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c144e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f652e369740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5640e0b6c5d0 con 0x5640e0c080c0 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f652e369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5640e0b25890 con 0x5640e0c09430 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f652e369740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5640e0b68590 con 0x5640e0c120f0 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f651a7fc640 1 -- 172.21.15.112:0/3437728756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6520003f00 con 0x7f6524101910 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c07b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51766/0 (socket says 172.21.15.112:51766) 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 -- 172.21.15.112:0/723074091 learned_addr learned my addr 172.21.15.112:0/723074091 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6528b13640 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529314640 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c08540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 -- 172.21.15.112:0/723074091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 msgr2=0x5640e0c144e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c144e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 -- 172.21.15.112:0/723074091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 msgr2=0x5640e0c08540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c08540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6529b15640 1 -- 172.21.15.112:0/723074091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e087e210 con 0x5640e0c09430 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.945+0000 7f6528b13640 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c144e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.946+0000 7f6529b15640 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c07b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f651c02ade0 tx=0x7f651c005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e412565f4f83508a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.946+0000 7f651a7fc640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65100b2920 0x7f65100b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.946+0000 7f650f7fe640 1 -- 172.21.15.112:0/723074091 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c031a80 con 0x5640e0c09430 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.946+0000 7f650f7fe640 1 -- 172.21.15.112:0/723074091 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f651c036030 con 0x5640e0c09430 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.946+0000 7f650f7fe640 1 -- 172.21.15.112:0/723074091 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0044e0 con 0x5640e0c09430 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.946+0000 7f6529314640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65100b2920 0x7f65100b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.946+0000 7f651a7fc640 1 -- 172.21.15.112:0/3437728756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f65200e29c0 con 0x7f6524101910 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 -- 172.21.15.112:0/723074091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 msgr2=0x5640e0c07b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f6529314640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65100b2920 0x7f65100b4de0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6520018000 tx=0x7f65200e8550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c07b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f651c02ade0 tx=0x7f651c005fa0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 -- 172.21.15.112:0/723074091 shutdown_connections 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c144e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c08540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 --2- 172.21.15.112:0/723074091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c07b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 -- 172.21.15.112:0/723074091 >> 172.21.15.112:0/723074091 conn(0x5640e0c03420 msgr2=0x5640e0c03820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 -- 172.21.15.112:0/723074091 shutdown_connections 2024-05-10T10:46:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 -- 172.21.15.112:0/723074091 wait complete. 2024-05-10T10:46:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 -- start start 2024-05-10T10:46:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.947+0000 7f652e369740 1 Processor -- start 2024-05-10T10:46:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.950+0000 7f652e369740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c080c0 0x5640e0c0d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.950+0000 7f652e369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c0da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.950+0000 7f652e369740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c120f0 0x5640e0c09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.950+0000 7f652e369740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5640e0c14ae0 con 0x5640e0c120f0 2024-05-10T10:46:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.950+0000 7f652e369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5640e0b29370 con 0x5640e0c09430 2024-05-10T10:46:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.950+0000 7f652e369740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5640e0c14c60 con 0x5640e0c080c0 2024-05-10T10:46:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.950+0000 7f652e369740 10 client.0 _ll_register_callbacks cb 0x5640e087da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.951+0000 7f6529b15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c120f0 0x5640e0c09ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.951+0000 7f6529b15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c120f0 0x5640e0c09ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53664/0 (socket says 172.21.15.112:53664) 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.951+0000 7f6529b15640 1 -- 172.21.15.112:0/2125418541 learned_addr learned my addr 172.21.15.112:0/2125418541 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.951+0000 7f6528b13640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c0da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.951+0000 7f6529b15640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c080c0 0x5640e0c0d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.952+0000 7f6529b15640 1 -- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c080c0 msgr2=0x5640e0c0d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.952+0000 7f6529b15640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c080c0 0x5640e0c0d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.952+0000 7f6529b15640 1 -- 172.21.15.112:0/2125418541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 msgr2=0x5640e0c0da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.952+0000 7f6529b15640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c0da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.952+0000 7f6529b15640 1 -- 172.21.15.112:0/2125418541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e094a620 con 0x5640e0c120f0 2024-05-10T10:46:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.952+0000 7f6528b13640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c0da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.953+0000 7f6529b15640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c120f0 0x5640e0c09ff0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f651c009960 tx=0x7f651c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f62cd9881fae4e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.953+0000 7f650cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.955+0000 7f650cff9640 1 -- 172.21.15.112:0/2125418541 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c058060 con 0x5640e0c120f0 2024-05-10T10:46:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.955+0000 7f650cff9640 1 -- 172.21.15.112:0/2125418541 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f651c004480 con 0x5640e0c120f0 2024-05-10T10:46:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.955+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f651c036030 con 0x5640e0c120f0 2024-05-10T10:46:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.956+0000 7f650cff9640 1 -- 172.21.15.112:0/2125418541 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f651c003e30 con 0x5640e0c120f0 2024-05-10T10:46:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.956+0000 7f652e369740 10 client.16866 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.956+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640e0c14ae0 con 0x5640e0c120f0 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f650cff9640 1 -- 172.21.15.112:0/2125418541 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f651c036030 con 0x5640e0c120f0 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f650cff9640 1 -- 172.21.15.112:0/2125418541 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f651c043070 con 0x5640e0c120f0 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f652e369740 10 client.16866 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f652e369740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f652e369740 20 client.16866 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f652e369740 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c26190 0x5640e0c285b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f652e369740 4 client.16866 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.957+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640e0b7a1e0 con 0x5640e0c26190 2024-05-10T10:46:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.958+0000 7f6528b13640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c26190 0x5640e0c285b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.959+0000 7f6528b13640 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c26190 0x5640e0c285b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7707b70b5b19d15 server_cookie=9f62cf27229386f4 in_seq=0 out_seq=0 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.959+0000 7f650cff9640 10 client.16866 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f650cff9640 1 -- 172.21.15.112:0/2125418541 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5640e0b7a1e0 con 0x5640e0c26190 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f650cff9640 10 client.16866 handle_command_reply: tid=0 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 client.16866 shutdown 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c120f0 msgr2=0x5640e0c09ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c120f0 0x5640e0c09ff0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f651c009960 tx=0x7f651c0095c0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 shutdown_connections 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c120f0 0x5640e0c09ff0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c26190 0x5640e0c285b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c09430 0x5640e0c0da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 --2- 172.21.15.112:0/2125418541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c080c0 0x5640e0c0d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 >> 172.21.15.112:0/2125418541 conn(0x5640e0c03420 msgr2=0x5640e0c02580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f650cff9640 0 client.16866 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 shutdown_connections 2024-05-10T10:46:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 1 -- 172.21.15.112:0/2125418541 wait complete. 2024-05-10T10:46:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.960+0000 7f652e369740 20 client.16866 trim_cache size 0 max 16384 2024-05-10T10:46:15.966 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:15 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:46:15.966 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:15 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:46:15.966 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:15 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.980+0000 7f652e369740 1 Processor -- start 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.980+0000 7f652e369740 1 -- start start 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.980+0000 7f652e369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c080c0 0x5640e0c281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f652e369740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c09430 0x5640e0c0d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f652e369740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f652e369740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65241a91f0 con 0x5640e0c09430 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f652e369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65241aad30 con 0x5640e0c080c0 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f652e369740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65241a98c0 con 0x5640e0c120f0 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6529b15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c080c0 0x5640e0c281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6529b15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c080c0 0x5640e0c281a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:51794/0 (socket says 172.21.15.112:51794) 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6529b15640 1 -- 172.21.15.112:0/391626251 learned_addr learned my addr 172.21.15.112:0/391626251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6528b13640 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c09430 0x5640e0c0d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6528b13640 1 -- 172.21.15.112:0/391626251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 msgr2=0x5640e0c0d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6528b13640 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c0d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6528b13640 1 -- 172.21.15.112:0/391626251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c080c0 msgr2=0x5640e0c281a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6528b13640 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c080c0 0x5640e0c281a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6528b13640 1 -- 172.21.15.112:0/391626251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e0c03250 con 0x5640e0c09430 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f6528b13640 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c09430 0x5640e0c0d470 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5640e0c28e40 tx=0x7f6514013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36ecbf0b0665df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f650effd640 1 -- 172.21.15.112:0/391626251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65140042f0 con 0x5640e0c09430 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f650effd640 1 -- 172.21.15.112:0/391626251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6514025070 con 0x5640e0c09430 2024-05-10T10:46:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.981+0000 7f650effd640 1 -- 172.21.15.112:0/391626251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514003a50 con 0x5640e0c09430 2024-05-10T10:46:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 -- 172.21.15.112:0/391626251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c09430 msgr2=0x5640e0c0d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c09430 0x5640e0c0d470 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5640e0c28e40 tx=0x7f6514013950 comp rx=0 tx=0).stop 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 -- 172.21.15.112:0/391626251 shutdown_connections 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c120f0 0x5640e0c0d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c09430 0x5640e0c0d470 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 --2- 172.21.15.112:0/391626251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c080c0 0x5640e0c281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 -- 172.21.15.112:0/391626251 >> 172.21.15.112:0/391626251 conn(0x5640e0c03420 msgr2=0x5640e0bfe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 -- 172.21.15.112:0/391626251 shutdown_connections 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.982+0000 7f652e369740 1 -- 172.21.15.112:0/391626251 wait complete. 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 -- start start 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 Processor -- start 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c09430 0x5640e0c37530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c120f0 0x5640e0c8e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65241aa660 con 0x5640e0c080c0 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65241a9f90 con 0x5640e0c120f0 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65241a71d0 con 0x5640e0c09430 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f652e369740 10 client.0 _ll_register_callbacks cb 0x7f65200e8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f6529b15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f6529b15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c36ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53676/0 (socket says 172.21.15.112:53676) 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f6529b15640 1 -- 172.21.15.112:0/3889699838 learned_addr learned my addr 172.21.15.112:0/3889699838 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.983+0000 7f6528b13640 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c09430 0x5640e0c37530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f6529b15640 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c120f0 0x5640e0c8e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f6529b15640 1 -- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c09430 msgr2=0x5640e0c37530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f6529b15640 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c09430 0x5640e0c37530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f6529b15640 1 -- 172.21.15.112:0/3889699838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c120f0 msgr2=0x5640e0c8e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f6529b15640 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c120f0 0x5640e0c8e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f6529b15640 1 -- 172.21.15.112:0/3889699838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e0c8ec70 con 0x5640e0c080c0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f6529b15640 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c36ff0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f651c02aeb0 tx=0x7f651c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa4addb0a9413a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f650cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f650cff9640 1 -- 172.21.15.112:0/3889699838 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c058040 con 0x5640e0c080c0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6514025070 con 0x5640e0c080c0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f650cff9640 1 -- 172.21.15.112:0/3889699838 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f651c004b40 con 0x5640e0c080c0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f650cff9640 1 -- 172.21.15.112:0/3889699838 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f651c004d40 con 0x5640e0c080c0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f652e369740 10 client.16872 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.984+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640e0c8ff80 con 0x5640e0c080c0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f650cff9640 1 -- 172.21.15.112:0/3889699838 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6514025070 con 0x5640e0c080c0 2024-05-10T10:46:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f650cff9640 1 -- 172.21.15.112:0/3889699838 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f651c063070 con 0x5640e0c080c0 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f652e369740 10 client.16872 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f652e369740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f652e369740 20 client.16872 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f652e369740 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c90820 0x5640e0c92c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f652e369740 4 client.16872 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640e0b74050 con 0x5640e0c90820 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.985+0000 7f6528b13640 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c90820 0x5640e0c92c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.986+0000 7f650cff9640 10 client.16872 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:15.986+0000 7f6528b13640 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c90820 0x5640e0c92c40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ca1aa0ee0e79b4 server_cookie=ae1e514dd8612a57 in_seq=0 out_seq=0 2024-05-10T10:46:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.004+0000 7f650cff9640 1 -- 172.21.15.112:0/3889699838 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640e0b74050 con 0x5640e0c90820 2024-05-10T10:46:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.004+0000 7f650cff9640 10 client.16872 handle_command_reply: tid=0 2024-05-10T10:46:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 client.16872 shutdown 2024-05-10T10:46:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 msgr2=0x5640e0c36ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c36ff0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f651c02aeb0 tx=0x7f651c02a8c0 comp rx=0 tx=0).stop 2024-05-10T10:46:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 shutdown_connections 2024-05-10T10:46:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5640e0c90820 0x5640e0c92c40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640e0c120f0 0x5640e0c8e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640e0c09430 0x5640e0c37530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 --2- 172.21.15.112:0/3889699838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5640e0c080c0 0x5640e0c36ff0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f650cff9640 0 client.16872 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.005+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 >> 172.21.15.112:0/3889699838 conn(0x5640e0c03420 msgr2=0x5640e0c281a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.006+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 shutdown_connections 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.006+0000 7f652e369740 1 -- 172.21.15.112:0/3889699838 wait complete. 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.006+0000 7f652e369740 20 client.16872 trim_cache size 0 max 16384 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:46:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:46:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:46:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65100b2920 msgr2=0x7f65100b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65100b2920 0x7f65100b4de0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6520018000 tx=0x7f65200e8550 comp rx=0 tx=0).stop 2024-05-10T10:46:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6524101910 msgr2=0x7f6524197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6524101910 0x7f6524197990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f652000e9c0 tx=0x7f652000ee80 comp rx=0 tx=0).stop 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 shutdown_connections 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65100b2920 0x7f65100b4de0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6520018000 tx=0x7f65200e8550 comp rx=0 tx=0).stop 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65241034c0 0x7f652419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6524102b00 0x7f6524197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 --2- 172.21.15.112:0/3437728756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6524101910 0x7f6524197990 secure :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f652000e9c0 tx=0x7f652000ee80 comp rx=0 tx=0).stop 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.007+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 >> 172.21.15.112:0/3437728756 conn(0x7f65240fce80 msgr2=0x7f6524105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.008+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 shutdown_connections 2024-05-10T10:46:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:16.008+0000 7f652b578640 1 -- 172.21.15.112:0/3437728756 wait complete. 2024-05-10T10:46:16.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:15 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:46:16.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:15 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:46:16.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:15 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:46:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:15 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:46:16.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:15 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:46:16.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:15 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:46:17.046 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:16 smithi148 ceph-mon[33568]: pgmap v611: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.5 MiB/s wr, 1.02k op/s 2024-05-10T10:46:17.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:16 smithi195 ceph-mon[33773]: pgmap v611: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.5 MiB/s wr, 1.02k op/s 2024-05-10T10:46:17.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:16 smithi112 ceph-mon[30100]: pgmap v611: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.5 MiB/s wr, 1.02k op/s 2024-05-10T10:46:19.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:18 smithi148 ceph-mon[33568]: pgmap v612: 129 pgs: 129 active+clean; 69 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 626 op/s 2024-05-10T10:46:19.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:18 smithi195 ceph-mon[33773]: pgmap v612: 129 pgs: 129 active+clean; 69 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 626 op/s 2024-05-10T10:46:19.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:18 smithi112 ceph-mon[30100]: pgmap v612: 129 pgs: 129 active+clean; 69 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 626 op/s 2024-05-10T10:46:21.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:20 smithi148 ceph-mon[33568]: pgmap v613: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 639 op/s 2024-05-10T10:46:21.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:20 smithi195 ceph-mon[33773]: pgmap v613: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 639 op/s 2024-05-10T10:46:21.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:20 smithi112 ceph-mon[30100]: pgmap v613: 129 pgs: 129 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 639 op/s 2024-05-10T10:46:23.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:22 smithi148 ceph-mon[33568]: pgmap v614: 129 pgs: 129 active+clean; 71 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 783 op/s 2024-05-10T10:46:23.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:22 smithi195 ceph-mon[33773]: pgmap v614: 129 pgs: 129 active+clean; 71 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 783 op/s 2024-05-10T10:46:23.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:22 smithi112 ceph-mon[30100]: pgmap v614: 129 pgs: 129 active+clean; 71 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 783 op/s 2024-05-10T10:46:25.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:24 smithi148 ceph-mon[33568]: pgmap v615: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 624 op/s 2024-05-10T10:46:25.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:24 smithi195 ceph-mon[33773]: pgmap v615: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 624 op/s 2024-05-10T10:46:25.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:24 smithi112 ceph-mon[30100]: pgmap v615: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 624 op/s 2024-05-10T10:46:27.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:26 smithi148 ceph-mon[33568]: pgmap v616: 129 pgs: 129 active+clean; 89 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 813 op/s 2024-05-10T10:46:27.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:26 smithi195 ceph-mon[33773]: pgmap v616: 129 pgs: 129 active+clean; 89 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 813 op/s 2024-05-10T10:46:27.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:26 smithi112 ceph-mon[30100]: pgmap v616: 129 pgs: 129 active+clean; 89 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 813 op/s 2024-05-10T10:46:29.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:28 smithi148 ceph-mon[33568]: pgmap v617: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 608 op/s 2024-05-10T10:46:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:28 smithi195 ceph-mon[33773]: pgmap v617: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 608 op/s 2024-05-10T10:46:29.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:28 smithi112 ceph-mon[30100]: pgmap v617: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 608 op/s 2024-05-10T10:46:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 -- 172.21.15.112:0/4027895291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 msgr2=0x7f24bc109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 --2- 172.21.15.112:0/4027895291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc109d70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009ca0 tx=0x7f24b402a8c0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 -- 172.21.15.112:0/4027895291 shutdown_connections 2024-05-10T10:46:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 --2- 172.21.15.112:0/4027895291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc109d70 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 --2- 172.21.15.112:0/4027895291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc102b20 0x7f24bc102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 --2- 172.21.15.112:0/4027895291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24bc101930 0x7f24bc101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 -- 172.21.15.112:0/4027895291 >> 172.21.15.112:0/4027895291 conn(0x7f24bc0fcea0 msgr2=0x7f24bc0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.140+0000 7f24c3207640 1 -- 172.21.15.112:0/4027895291 shutdown_connections 2024-05-10T10:46:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.141+0000 7f24c3207640 1 -- 172.21.15.112:0/4027895291 wait complete. 2024-05-10T10:46:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.141+0000 7f24c3207640 1 Processor -- start 2024-05-10T10:46:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.141+0000 7f24c3207640 1 -- start start 2024-05-10T10:46:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.141+0000 7f24c3207640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24bc101930 0x7f24bc06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c3207640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc102b20 0x7f24bc19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c3207640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c3207640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc10d340 con 0x7f24bc1034e0 2024-05-10T10:46:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c3207640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc10d1c0 con 0x7f24bc101930 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c3207640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc10d4c0 con 0x7f24bc102b20 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48270/0 (socket says 172.21.15.112:48270) 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 -- 172.21.15.112:0/217949909 learned_addr learned my addr 172.21.15.112:0/217949909 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24bc101930 0x7f24bc06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 -- 172.21.15.112:0/217949909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc102b20 msgr2=0x7f24bc19d780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc102b20 0x7f24bc19d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 -- 172.21.15.112:0/217949909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24bc101930 msgr2=0x7f24bc06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24bc101930 0x7f24bc06bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 -- 172.21.15.112:0/217949909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc1a0680 con 0x7f24bc1034e0 2024-05-10T10:46:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c17a4640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc19fdc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f24b402aeb0 tx=0x7f24b4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24b9ffb640 1 -- 172.21.15.112:0/217949909 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003e10 con 0x7f24bc1034e0 2024-05-10T10:46:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc1a0910 con 0x7f24bc1034e0 2024-05-10T10:46:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24b9ffb640 1 -- 172.21.15.112:0/217949909 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24b40044d0 con 0x7f24bc1034e0 2024-05-10T10:46:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.142+0000 7f24b9ffb640 1 -- 172.21.15.112:0/217949909 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4030c90 con 0x7f24bc1034e0 2024-05-10T10:46:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.143+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc1a0e60 con 0x7f24bc1034e0 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.146+0000 7f24c5ff7740 1 Processor -- start 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.146+0000 7f24b9ffb640 1 -- 172.21.15.112:0/217949909 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f24b4004680 con 0x7f24bc1034e0 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.146+0000 7f24c5ff7740 1 -- start start 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a7a0910 0x55ed4a7a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a6ff5d0 con 0x55ed4a7a0910 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a6b8890 con 0x55ed4a79b7a0 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:59720/0 (socket says 172.21.15.112:59720) 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24bbfff640 1 -- 172.21.15.112:0/691226116 learned_addr learned my addr 172.21.15.112:0/691226116 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24c5ff7740 1 -- 172.21.15.112:0/691226116 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a6fb590 con 0x55ed4a79adb0 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24bbfff640 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a7a0910 0x55ed4a7a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.147+0000 7f24b9ffb640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f24ac0b2920 0x7f24ac0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c0fa3640 1 -- 172.21.15.112:0/691226116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 msgr2=0x55ed4a79b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c0fa3640 1 -- 172.21.15.112:0/691226116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a7a0910 msgr2=0x55ed4a7a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c17a4640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f24ac0b2920 0x7f24ac0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a7a0910 0x55ed4a7a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c0fa3640 1 -- 172.21.15.112:0/691226116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4a411210 con 0x55ed4a79b7a0 2024-05-10T10:46:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24bbfff640 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24b9ffb640 1 -- 172.21.15.112:0/217949909 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f24b4103df0 con 0x7f24bc1034e0 2024-05-10T10:46:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a02a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009bf0 tx=0x7f24b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a776a3ea1e4048c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24c17a4640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f24ac0b2920 0x7f24ac0b4de0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009430 tx=0x7f24b40fa630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24a6ffd640 1 -- 172.21.15.112:0/691226116 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b001e070 con 0x55ed4a79b7a0 2024-05-10T10:46:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24a6ffd640 1 -- 172.21.15.112:0/691226116 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24b0009da0 con 0x55ed4a79b7a0 2024-05-10T10:46:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.148+0000 7f24a6ffd640 1 -- 172.21.15.112:0/691226116 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0003d90 con 0x55ed4a79b7a0 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 -- 172.21.15.112:0/691226116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 msgr2=0x55ed4a7a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a02a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009bf0 tx=0x7f24b0011810 comp rx=0 tx=0).stop 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 -- 172.21.15.112:0/691226116 shutdown_connections 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a7a0910 0x55ed4a7a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a02a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/691226116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 -- 172.21.15.112:0/691226116 >> 172.21.15.112:0/691226116 conn(0x55ed4a796420 msgr2=0x55ed4a796820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 -- 172.21.15.112:0/691226116 shutdown_connections 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 -- 172.21.15.112:0/691226116 wait complete. 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 -- start start 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.149+0000 7f24c5ff7740 1 Processor -- start 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a7a0910 0x55ed4a7ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48290/0 (socket says 172.21.15.112:48290) 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 -- 172.21.15.112:0/1146215521 learned_addr learned my addr 172.21.15.112:0/1146215521 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a7a38c0 con 0x55ed4a79adb0 2024-05-10T10:46:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a6bc370 con 0x55ed4a7a0910 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a7a3a40 con 0x55ed4a79b7a0 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c5ff7740 10 client.0 _ll_register_callbacks cb 0x55ed4a410a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a7a0910 0x55ed4a7ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 -- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79b7a0 msgr2=0x55ed4a7a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 -- 172.21.15.112:0/1146215521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a79adb0 msgr2=0x55ed4a7a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.150+0000 7f24bbfff640 1 -- 172.21.15.112:0/1146215521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4a4dd620 con 0x55ed4a7a0910 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a7a0910 0x55ed4a7ac5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f24a8014e70 tx=0x7f24a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bdfc48f3045b53b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f249bfff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f249bfff640 1 -- 172.21.15.112:0/1146215521 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8011e80 con 0x55ed4a7a0910 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b0009da0 con 0x55ed4a7a0910 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f249bfff640 1 -- 172.21.15.112:0/1146215521 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f24a8004040 con 0x55ed4a7a0910 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f249bfff640 1 -- 172.21.15.112:0/1146215521 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f24a8026050 con 0x55ed4a7a0910 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 10 client.26252 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ed4a7af5f0 con 0x55ed4a7a0910 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f249bfff640 1 -- 172.21.15.112:0/1146215521 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24b0009da0 con 0x55ed4a7a0910 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f249bfff640 1 -- 172.21.15.112:0/1146215521 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f24a8003ba0 con 0x55ed4a7a0910 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 10 client.26252 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 20 client.26252 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7b0c10 0x55ed4a7b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 4 client.26252 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.151+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed4a70d1e0 con 0x55ed4a7b0c10 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.152+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7b0c10 0x55ed4a7b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7b0c10 0x55ed4a7b3080 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79970b7f708d5652 server_cookie=8cfca60f0a04097f in_seq=0 out_seq=0 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f249bfff640 10 client.26252 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f249bfff640 1 -- 172.21.15.112:0/1146215521 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed4a70d1e0 con 0x55ed4a7b0c10 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f249bfff640 10 client.26252 handle_command_reply: tid=0 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f24c5ff7740 1 client.26252 shutdown 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a7a0910 msgr2=0x55ed4a7ac5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a7a0910 0x55ed4a7ac5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f24a8014e70 tx=0x7f24a80046c0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 shutdown_connections 2024-05-10T10:46:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.153+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7b0c10 0x55ed4a7b3080 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a7a0910 0x55ed4a7ac5a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1146215521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f249bfff640 0 client.26252 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 >> 172.21.15.112:0/1146215521 conn(0x55ed4a796420 msgr2=0x55ed4a795580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 shutdown_connections 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1146215521 wait complete. 2024-05-10T10:46:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.154+0000 7f24c5ff7740 20 client.26252 trim_cache size 0 max 16384 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 Processor -- start 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 -- start start 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a79c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1a9180 con 0x55ed4a796420 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1aacc0 con 0x55ed4a79b7a0 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.173+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1a9850 con 0x55ed4a79adb0 2024-05-10T10:46:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a79c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a79c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48298/0 (socket says 172.21.15.112:48298) 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24bbfff640 1 -- 172.21.15.112:0/1585904511 learned_addr learned my addr 172.21.15.112:0/1585904511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24c0fa3640 1 -- 172.21.15.112:0/1585904511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 msgr2=0x55ed4a7ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24c0fa3640 1 -- 172.21.15.112:0/1585904511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 msgr2=0x55ed4a79c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a79c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24c0fa3640 1 -- 172.21.15.112:0/1585904511 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4a79b500 con 0x55ed4a79adb0 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79cec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ed4a7972f0 tx=0x7f24b0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd1a0a68fcde931b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.174+0000 7f24a67fc640 1 -- 172.21.15.112:0/1585904511 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b00097a0 con 0x55ed4a79adb0 2024-05-10T10:46:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24a67fc640 1 -- 172.21.15.112:0/1585904511 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24b002ee50 con 0x55ed4a79adb0 2024-05-10T10:46:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24a67fc640 1 -- 172.21.15.112:0/1585904511 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0005620 con 0x55ed4a79adb0 2024-05-10T10:46:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1585904511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 msgr2=0x55ed4a79cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79cec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ed4a7972f0 tx=0x7f24b0014000 comp rx=0 tx=0).stop 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1585904511 shutdown_connections 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a79cec0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1585904511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a79c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1585904511 >> 172.21.15.112:0/1585904511 conn(0x55ed4a791860 msgr2=0x55ed4a79fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1585904511 shutdown_connections 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.175+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1585904511 wait complete. 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 -- start start 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 Processor -- start 2024-05-10T10:46:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a7a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1aa5f0 con 0x55ed4a796420 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1a9f20 con 0x55ed4a79b7a0 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1a7160 con 0x55ed4a79adb0 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c5ff7740 10 client.0 _ll_register_callbacks cb 0x7f24a8014620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a7a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24c0fa3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a7a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48312/0 (socket says 172.21.15.112:48312) 2024-05-10T10:46:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24bbfff640 1 -- 172.21.15.112:0/1430709032 learned_addr learned my addr 172.21.15.112:0/1430709032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.176+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c0fa3640 1 -- 172.21.15.112:0/1430709032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 msgr2=0x55ed4a7aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c0fa3640 1 -- 172.21.15.112:0/1430709032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 msgr2=0x55ed4a7a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a7a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c0fa3640 1 -- 172.21.15.112:0/1430709032 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4a7ab5a0 con 0x55ed4a79adb0 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a7a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c0fa3640 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a9ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55ed4a796ef0 tx=0x7f24b0009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=353c7f6f58b39ea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f249bfff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f249bfff640 1 -- 172.21.15.112:0/1430709032 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b00053a0 con 0x55ed4a79adb0 2024-05-10T10:46:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b002ee50 con 0x55ed4a79adb0 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f249bfff640 1 -- 172.21.15.112:0/1430709032 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f24b0005540 con 0x55ed4a79adb0 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f249bfff640 1 -- 172.21.15.112:0/1430709032 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f24b0003a00 con 0x55ed4a79adb0 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c5ff7740 10 client.26272 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.177+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ed4a7ac8b0 con 0x55ed4a79adb0 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f249bfff640 1 -- 172.21.15.112:0/1430709032 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24b002ee50 con 0x55ed4a79adb0 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f249bfff640 1 -- 172.21.15.112:0/1430709032 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f24b002b070 con 0x55ed4a79adb0 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f24c5ff7740 10 client.26272 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f24c5ff7740 10 client.26272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f24c5ff7740 20 client.26272 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7a0910 0x55ed4a81da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f24c5ff7740 4 client.26272 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:46:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed4a707050 con 0x55ed4a7a0910 2024-05-10T10:46:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.178+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7a0910 0x55ed4a81da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.179+0000 7f24bbfff640 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7a0910 0x55ed4a81da00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34725ab4ad794ec5 server_cookie=137e7fd45dfb0693 in_seq=0 out_seq=0 2024-05-10T10:46:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.179+0000 7f249bfff640 10 client.26272 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.192+0000 7f249bfff640 1 -- 172.21.15.112:0/1430709032 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed4a707050 con 0x55ed4a7a0910 2024-05-10T10:46:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.192+0000 7f249bfff640 10 client.26272 handle_command_reply: tid=0 2024-05-10T10:46:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.192+0000 7f24c5ff7740 1 client.26272 shutdown 2024-05-10T10:46:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 msgr2=0x55ed4a7a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a9ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55ed4a796ef0 tx=0x7f24b0009da0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 shutdown_connections 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55ed4a7a0910 0x55ed4a81da00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ed4a79b7a0 0x55ed4a7aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f249bfff640 0 client.26272 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ed4a79adb0 0x55ed4a7a9ff0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 --2- 172.21.15.112:0/1430709032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed4a796420 0x55ed4a7a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 >> 172.21.15.112:0/1430709032 conn(0x55ed4a791860 msgr2=0x55ed4a79c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 shutdown_connections 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 1 -- 172.21.15.112:0/1430709032 wait complete. 2024-05-10T10:46:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.193+0000 7f24c5ff7740 20 client.26272 trim_cache size 0 max 16384 2024-05-10T10:46:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:46:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:46:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:46:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:46:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f24ac0b2920 msgr2=0x7f24ac0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f24ac0b2920 0x7f24ac0b4de0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009430 tx=0x7f24b40fa630 comp rx=0 tx=0).stop 2024-05-10T10:46:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 msgr2=0x7f24bc19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc19fdc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f24b402aeb0 tx=0x7f24b4003c20 comp rx=0 tx=0).stop 2024-05-10T10:46:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 shutdown_connections 2024-05-10T10:46:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f24ac0b2920 0x7f24ac0b4de0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24bc1034e0 0x7f24bc19fdc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24bc102b20 0x7f24bc19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 --2- 172.21.15.112:0/217949909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24bc101930 0x7f24bc06bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 >> 172.21.15.112:0/217949909 conn(0x7f24bc0fcea0 msgr2=0x7f24bc105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 shutdown_connections 2024-05-10T10:46:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:29.194+0000 7f24c3207640 1 -- 172.21.15.112:0/217949909 wait complete. 2024-05-10T10:46:31.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:30 smithi148 ceph-mon[33568]: pgmap v618: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 618 op/s 2024-05-10T10:46:31.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:30 smithi195 ceph-mon[33773]: pgmap v618: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 618 op/s 2024-05-10T10:46:31.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:30 smithi112 ceph-mon[30100]: pgmap v618: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 618 op/s 2024-05-10T10:46:33.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:32 smithi148 ceph-mon[33568]: pgmap v619: 129 pgs: 129 active+clean; 83 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 776 op/s 2024-05-10T10:46:33.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:32 smithi195 ceph-mon[33773]: pgmap v619: 129 pgs: 129 active+clean; 83 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 776 op/s 2024-05-10T10:46:33.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:32 smithi112 ceph-mon[30100]: pgmap v619: 129 pgs: 129 active+clean; 83 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 776 op/s 2024-05-10T10:46:35.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:34 smithi148 ceph-mon[33568]: pgmap v620: 129 pgs: 129 active+clean; 76 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 597 op/s 2024-05-10T10:46:35.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:34 smithi195 ceph-mon[33773]: pgmap v620: 129 pgs: 129 active+clean; 76 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 597 op/s 2024-05-10T10:46:35.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:34 smithi112 ceph-mon[30100]: pgmap v620: 129 pgs: 129 active+clean; 76 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 597 op/s 2024-05-10T10:46:35.333 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:46:35.648 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:46:36.653 INFO:tasks.workunit.client.0.smithi112.stdout:20000 trunc from 0x62a09d3 to 0x111d204 2024-05-10T10:46:37.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:36 smithi112 ceph-mon[30100]: pgmap v621: 129 pgs: 129 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.5 MiB/s wr, 788 op/s 2024-05-10T10:46:37.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:36 smithi148 ceph-mon[33568]: pgmap v621: 129 pgs: 129 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.5 MiB/s wr, 788 op/s 2024-05-10T10:46:37.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:36 smithi195 ceph-mon[33773]: pgmap v621: 129 pgs: 129 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.5 MiB/s wr, 788 op/s 2024-05-10T10:46:38.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:38 smithi112 ceph-mon[30100]: pgmap v622: 129 pgs: 129 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 551 op/s 2024-05-10T10:46:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 -- 172.21.15.112:0/2784118848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c102d60 msgr2=0x7efe5c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 --2- 172.21.15.112:0/2784118848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c102d60 0x7efe5c10f250 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe5400b0a0 tx=0x7efe5400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 -- 172.21.15.112:0/2784118848 shutdown_connections 2024-05-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 --2- 172.21.15.112:0/2784118848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c102d60 0x7efe5c10f250 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 --2- 172.21.15.112:0/2784118848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c108410 0x7efe5c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 --2- 172.21.15.112:0/2784118848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c1023a0 0x7efe5c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 -- 172.21.15.112:0/2784118848 >> 172.21.15.112:0/2784118848 conn(0x7efe5c079350 msgr2=0x7efe5c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 -- 172.21.15.112:0/2784118848 shutdown_connections 2024-05-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 -- 172.21.15.112:0/2784118848 wait complete. 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.007+0000 7efe61d80700 1 Processor -- start 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe61d80700 1 -- start start 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe61d80700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c1023a0 0x7efe5c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe61d80700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c102d60 0x7efe5c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe61d80700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c108410 0x7efe5c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe61d80700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c112400 con 0x7efe5c102d60 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe61d80700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c112280 con 0x7efe5c108410 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe61d80700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c112580 con 0x7efe5c1023a0 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c102d60 0x7efe5c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c1023a0 0x7efe5c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c102d60 0x7efe5c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43248/0 (socket says 172.21.15.112:43248) 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5affd700 1 -- 172.21.15.112:0/1275240713 learned_addr learned my addr 172.21.15.112:0/1275240713 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5bfff700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c108410 0x7efe5c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5affd700 1 -- 172.21.15.112:0/1275240713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c1023a0 msgr2=0x7efe5c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5affd700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c1023a0 0x7efe5c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.008+0000 7efe5affd700 1 -- 172.21.15.112:0/1275240713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c108410 msgr2=0x7efe5c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.009+0000 7efe5affd700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c108410 0x7efe5c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.009+0000 7efe5affd700 1 -- 172.21.15.112:0/1275240713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c1a7f60 con 0x7efe5c102d60 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.009+0000 7efe5affd700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c102d60 0x7efe5c1045c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efe5000eb30 tx=0x7efe5000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.009+0000 7efe58ff9700 1 -- 172.21.15.112:0/1275240713 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5000cbd0 con 0x7efe5c102d60 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.009+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1a81f0 con 0x7efe5c102d60 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.009+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a8720 con 0x7efe5c102d60 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.009+0000 7efe58ff9700 1 -- 172.21.15.112:0/1275240713 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe500044b0 con 0x7efe5c102d60 2024-05-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.010+0000 7efe58ff9700 1 -- 172.21.15.112:0/1275240713 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50003c80 con 0x7efe5c102d60 2024-05-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.013+0000 7efe58ff9700 1 -- 172.21.15.112:0/1275240713 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7efe50018430 con 0x7efe5c102d60 2024-05-10T10:46:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.014+0000 7efe58ff9700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.014+0000 7efe58ff9700 1 -- 172.21.15.112:0/1275240713 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe500e2800 con 0x7efe5c102d60 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.014+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 Processor -- start 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 -- start start 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009be0 tx=0x7efe4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e2c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e2d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564881d7a860 con 0x564881e2cc80 2024-05-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564881d197c0 con 0x564881e28df0 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe6c613b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564881d76820 con 0x564881e284d0 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e2d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe5affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e2c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e2d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43252/0 (socket says 172.21.15.112:43252) 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe5bfff700 1 -- 172.21.15.112:0/564548247 learned_addr learned my addr 172.21.15.112:0/564548247 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.015+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.016+0000 7efe5bfff700 1 -- 172.21.15.112:0/564548247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 msgr2=0x564881e288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.016+0000 7efe5bfff700 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.016+0000 7efe5bfff700 1 -- 172.21.15.112:0/564548247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 msgr2=0x564881e2c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.016+0000 7efe5bfff700 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e2c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.016+0000 7efe5bfff700 1 -- 172.21.15.112:0/564548247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564881cc78d0 con 0x564881e2cc80 2024-05-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.016+0000 7efe5affd700 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e2c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.017+0000 7efe5bfff700 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e2d100 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efe54009e20 tx=0x7efe540092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ddea998932b90e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.017+0000 7efe41ffb700 1 -- 172.21.15.112:0/564548247 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54020070 con 0x564881e2cc80 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.017+0000 7efe41ffb700 1 -- 172.21.15.112:0/564548247 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5400bcb0 con 0x564881e2cc80 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.017+0000 7efe41ffb700 1 -- 172.21.15.112:0/564548247 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54007c70 con 0x564881e2cc80 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 -- 172.21.15.112:0/564548247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 msgr2=0x564881e2d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e2d100 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efe54009e20 tx=0x7efe540092e0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 -- 172.21.15.112:0/564548247 shutdown_connections 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e2d100 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 --2- 172.21.15.112:0/564548247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e2c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 -- 172.21.15.112:0/564548247 >> 172.21.15.112:0/564548247 conn(0x564881d54100 msgr2=0x564881e22c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 -- 172.21.15.112:0/564548247 shutdown_connections 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 -- 172.21.15.112:0/564548247 wait complete. 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.018+0000 7efe6c613b80 1 -- start start 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 1 Processor -- start 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e34060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e31400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e31940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564881e34e30 con 0x564881e2cc80 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564881ce0c30 con 0x564881e28df0 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564881e34fb0 con 0x564881e284d0 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe5b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e34060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe5affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e31400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe6c613b80 10 client.0 _ll_register_callbacks cb 0x564881d12920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e31940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe5b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e34060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:37984/0 (socket says 172.21.15.112:37984) 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.019+0000 7efe5b7fe700 1 -- 172.21.15.112:0/3249192107 learned_addr learned my addr 172.21.15.112:0/3249192107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe5affd700 1 -- 172.21.15.112:0/3249192107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 msgr2=0x564881e34060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe5affd700 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e34060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe5affd700 1 -- 172.21.15.112:0/3249192107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 msgr2=0x564881e31940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe5affd700 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e31940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe5affd700 1 -- 172.21.15.112:0/3249192107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564881d16690 con 0x564881e28df0 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe5bfff700 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e31940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe5affd700 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e31400 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe500dff10 tx=0x7efe500a9540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b909c0237ddda2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe3affd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe3affd700 1 -- 172.21.15.112:0/3249192107 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe500aa130 con 0x564881e28df0 2024-05-10T10:46:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe5400bcb0 con 0x564881e28df0 2024-05-10T10:46:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.020+0000 7efe3affd700 1 -- 172.21.15.112:0/3249192107 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe500a6020 con 0x564881e28df0 2024-05-10T10:46:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe3affd700 1 -- 172.21.15.112:0/3249192107 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efe500a9a30 con 0x564881e28df0 2024-05-10T10:46:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 10 client.26270 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564881e3f6c0 con 0x564881e28df0 2024-05-10T10:46:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe3affd700 1 -- 172.21.15.112:0/3249192107 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5400bcb0 con 0x564881e28df0 2024-05-10T10:46:39.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe3affd700 1 -- 172.21.15.112:0/3249192107 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7efe500cace0 con 0x564881e28df0 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 10 client.26270 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 10 client.26270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 20 client.26270 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881e41be0 0x564881e44060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 4 client.26270 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.021+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564881df4650 con 0x564881e41be0 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.022+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881e41be0 0x564881e44060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.022+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881e41be0 0x564881e44060 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d112af10ff11250 server_cookie=d41b5093ca2470d7 in_seq=0 out_seq=0 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.022+0000 7efe3affd700 10 client.26270 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe3affd700 1 -- 172.21.15.112:0/3249192107 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564881df4650 con 0x564881e41be0 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe3affd700 10 client.26270 handle_command_reply: tid=0 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 client.26270 shutdown 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 msgr2=0x564881e31400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e31400 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe500dff10 tx=0x7efe500a9540 comp rx=0 tx=0).stop 2024-05-10T10:46:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 shutdown_connections 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881e41be0 0x564881e44060 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e2cc80 0x564881e31940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e34060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 --2- 172.21.15.112:0/3249192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e28df0 0x564881e31400 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 >> 172.21.15.112:0/3249192107 conn(0x564881d54100 msgr2=0x564881e2d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe3affd700 0 client.26270 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 shutdown_connections 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 1 -- 172.21.15.112:0/3249192107 wait complete. 2024-05-10T10:46:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.023+0000 7efe6c613b80 20 client.26270 trim_cache size 0 max 16384 2024-05-10T10:46:39.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:38 smithi148 ceph-mon[33568]: pgmap v622: 129 pgs: 129 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 551 op/s 2024-05-10T10:46:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 Processor -- start 2024-05-10T10:46:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 -- start start 2024-05-10T10:46:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e284d0 0x564881e3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e2cc80 0x564881e1d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e31a30 0x564881e49970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe4800eac0 con 0x564881e284d0 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe48010600 con 0x564881e31a30 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe6c613b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe4800f190 con 0x564881e2cc80 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe5b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e284d0 0x564881e3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe5b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e284d0 0x564881e3df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43260/0 (socket says 172.21.15.112:43260) 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe5b7fe700 1 -- 172.21.15.112:0/1323876245 learned_addr learned my addr 172.21.15.112:0/1323876245 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.052+0000 7efe5bfff700 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e31a30 0x564881e49970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe5b7fe700 1 -- 172.21.15.112:0/1323876245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e2cc80 msgr2=0x564881e1d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e2cc80 0x564881e1d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe5b7fe700 1 -- 172.21.15.112:0/1323876245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e31a30 msgr2=0x564881e49970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e31a30 0x564881e49970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe5b7fe700 1 -- 172.21.15.112:0/1323876245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564881e2f990 con 0x564881e284d0 2024-05-10T10:46:39.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e284d0 0x564881e3df50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efe4c011ba0 tx=0x7efe4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8e5dc14722e464 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:39.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe417fa700 1 -- 172.21.15.112:0/1323876245 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c02e070 con 0x564881e284d0 2024-05-10T10:46:39.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe417fa700 1 -- 172.21.15.112:0/1323876245 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe4c029850 con 0x564881e284d0 2024-05-10T10:46:39.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.053+0000 7efe417fa700 1 -- 172.21.15.112:0/1323876245 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c003dc0 con 0x564881e284d0 2024-05-10T10:46:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 -- 172.21.15.112:0/1323876245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e284d0 msgr2=0x564881e3df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e284d0 0x564881e3df50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efe4c011ba0 tx=0x7efe4c014000 comp rx=0 tx=0).stop 2024-05-10T10:46:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 -- 172.21.15.112:0/1323876245 shutdown_connections 2024-05-10T10:46:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e31a30 0x564881e49970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e284d0 0x564881e3df50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 --2- 172.21.15.112:0/1323876245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e2cc80 0x564881e1d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 -- 172.21.15.112:0/1323876245 >> 172.21.15.112:0/1323876245 conn(0x564881d54100 msgr2=0x564881e1ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 -- 172.21.15.112:0/1323876245 shutdown_connections 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.054+0000 7efe6c613b80 1 -- 172.21.15.112:0/1323876245 wait complete. 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.055+0000 7efe6c613b80 1 -- start start 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.055+0000 7efe6c613b80 1 Processor -- start 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.055+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.055+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e2cc80 0x564881e36730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.055+0000 7efe6c613b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e31a30 0x564881eab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e2cc80 0x564881e36730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e2cc80 0x564881e36730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:45642/0 (socket says 172.21.15.112:45642) 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5affd700 1 -- 172.21.15.112:0/677522316 learned_addr learned my addr 172.21.15.112:0/677522316 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe4800ff30 con 0x564881e31a30 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5bfff700 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e31a30 0x564881eab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe4800f860 con 0x564881e2cc80 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe4800ca60 con 0x564881e284d0 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe6c613b80 10 client.0 _ll_register_callbacks cb 0x564881d12920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5bfff700 1 -- 172.21.15.112:0/677522316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 msgr2=0x564881e361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5bfff700 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5bfff700 1 -- 172.21.15.112:0/677522316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e2cc80 msgr2=0x564881e36730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5bfff700 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e2cc80 0x564881e36730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.056+0000 7efe5bfff700 1 -- 172.21.15.112:0/677522316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564881eaba60 con 0x564881e31a30 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe5bfff700 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e31a30 0x564881eab520 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efe54009fd0 tx=0x7efe54007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db285442fcd55e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe3affd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe3affd700 1 -- 172.21.15.112:0/677522316 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54020070 con 0x564881e31a30 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe3affd700 1 -- 172.21.15.112:0/677522316 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7efe54007cd0 con 0x564881e31a30 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4c029850 con 0x564881e31a30 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe3affd700 1 -- 172.21.15.112:0/677522316 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efe5400be60 con 0x564881e31a30 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe6c613b80 10 client.16908 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.057+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564881eacbb0 con 0x564881e31a30 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe3affd700 1 -- 172.21.15.112:0/677522316 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5402d070 con 0x564881e31a30 2024-05-10T10:46:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe3affd700 1 -- 172.21.15.112:0/677522316 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7efe54011070 con 0x564881e31a30 2024-05-10T10:46:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe6c613b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe6c613b80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe6c613b80 20 client.16908 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe6c613b80 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881eace90 0x564881eaf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe6c613b80 4 client.16908 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:46:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7efe4c029850 con 0x564881eace90 2024-05-10T10:46:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.058+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881eace90 0x564881eaf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.060+0000 7efe3affd700 10 client.16908 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.060+0000 7efe5b7fe700 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881eace90 0x564881eaf280 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d63ce7db185a13a server_cookie=c2f1f7982ba7b264 in_seq=0 out_seq=0 2024-05-10T10:46:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.060+0000 7efe3affd700 1 -- 172.21.15.112:0/677522316 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7efe4c029850 con 0x564881eace90 2024-05-10T10:46:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.060+0000 7efe3affd700 10 client.16908 handle_command_reply: tid=0 2024-05-10T10:46:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.060+0000 7efe6c613b80 1 client.16908 shutdown 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e31a30 msgr2=0x564881eab520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e31a30 0x564881eab520 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efe54009fd0 tx=0x7efe54007850 comp rx=0 tx=0).stop 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 shutdown_connections 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x564881eace90 0x564881eaf280 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564881e31a30 0x564881eab520 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564881e284d0 0x564881e361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 --2- 172.21.15.112:0/677522316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564881e2cc80 0x564881e36730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 >> 172.21.15.112:0/677522316 conn(0x564881d54100 msgr2=0x564881e1d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe3affd700 0 client.16908 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 shutdown_connections 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 1 -- 172.21.15.112:0/677522316 wait complete. 2024-05-10T10:46:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.061+0000 7efe6c613b80 20 client.16908 trim_cache size 0 max 16384 2024-05-10T10:46:39.065 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:46:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:46:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:46:39.065 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:46:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 msgr2=0x7efe440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009be0 tx=0x7efe4c009260 comp rx=0 tx=0).stop 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c102d60 msgr2=0x7efe5c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c102d60 0x7efe5c1045c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efe5000eb30 tx=0x7efe5000ee40 comp rx=0 tx=0).stop 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 shutdown_connections 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe5c108410 0x7efe5c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7efe440b2900 0x7efe440b4dc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe5c1023a0 0x7efe5c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 --2- 172.21.15.112:0/1275240713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe5c102d60 0x7efe5c1045c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 >> 172.21.15.112:0/1275240713 conn(0x7efe5c079350 msgr2=0x7efe5c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 shutdown_connections 2024-05-10T10:46:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:39.062+0000 7efe61d80700 1 -- 172.21.15.112:0/1275240713 wait complete. 2024-05-10T10:46:39.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:38 smithi195 ceph-mon[33773]: pgmap v622: 129 pgs: 129 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 551 op/s 2024-05-10T10:46:39.928 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:46:39.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:46:40.278 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:46:41.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:40 smithi148 ceph-mon[33568]: pgmap v623: 129 pgs: 129 active+clean; 72 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 593 op/s 2024-05-10T10:46:41.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:40 smithi195 ceph-mon[33773]: pgmap v623: 129 pgs: 129 active+clean; 72 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 593 op/s 2024-05-10T10:46:41.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:40 smithi112 ceph-mon[30100]: pgmap v623: 129 pgs: 129 active+clean; 72 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 593 op/s 2024-05-10T10:46:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.317+0000 7f994fd38640 1 -- 172.21.15.112:0/571156960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f994810c320 msgr2=0x7f994810c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.317+0000 7f994fd38640 1 --2- 172.21.15.112:0/571156960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f994810c320 0x7f994810c780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9938009900 tx=0x7f993802a400 comp rx=0 tx=0).stop 2024-05-10T10:46:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.317+0000 7f994fd38640 1 -- 172.21.15.112:0/571156960 shutdown_connections 2024-05-10T10:46:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.317+0000 7f994fd38640 1 --2- 172.21.15.112:0/571156960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99481130f0 0x7f9948115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.317+0000 7f994fd38640 1 --2- 172.21.15.112:0/571156960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f994810c320 0x7f994810c780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.317+0000 7f994fd38640 1 --2- 172.21.15.112:0/571156960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99481099b0 0x7f994810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.317+0000 7f994fd38640 1 -- 172.21.15.112:0/571156960 >> 172.21.15.112:0/571156960 conn(0x7f994806fa20 msgr2=0x7f9948071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.318+0000 7f994fd38640 1 -- 172.21.15.112:0/571156960 shutdown_connections 2024-05-10T10:46:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.318+0000 7f994fd38640 1 -- 172.21.15.112:0/571156960 wait complete. 2024-05-10T10:46:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.319+0000 7f994fd38640 1 Processor -- start 2024-05-10T10:46:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.319+0000 7f994fd38640 1 -- start start 2024-05-10T10:46:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994fd38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99481099b0 0x7f994819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994fd38640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f994810c320 0x7f994819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994fd38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99481130f0 0x7f99481a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994fd38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f994810f6b0 con 0x7f99481099b0 2024-05-10T10:46:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994fd38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f994810f530 con 0x7f99481130f0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994fd38640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f994810f830 con 0x7f994810c320 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994d2d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f994810c320 0x7f994819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994d2d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f994810c320 0x7f994819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:38012/0 (socket says 172.21.15.112:38012) 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994d2d3640 1 -- 172.21.15.112:0/2113741973 learned_addr learned my addr 172.21.15.112:0/2113741973 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994dad4640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99481099b0 0x7f994819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.320+0000 7f994e2d5640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99481130f0 0x7f99481a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f994e2d5640 1 -- 172.21.15.112:0/2113741973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f994810c320 msgr2=0x7f994819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f994e2d5640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f994810c320 0x7f994819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f994e2d5640 1 -- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99481099b0 msgr2=0x7f994819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f994e2d5640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99481099b0 0x7f994819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f994e2d5640 1 -- 172.21.15.112:0/2113741973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99481a6ce0 con 0x7f99481130f0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f994e2d5640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99481130f0 0x7f99481a6670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f994400fd70 tx=0x7f9944015f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f993effd640 1 -- 172.21.15.112:0/2113741973 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944005020 con 0x7f99481130f0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f993effd640 1 -- 172.21.15.112:0/2113741973 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99440047a0 con 0x7f99481130f0 2024-05-10T10:46:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f993effd640 1 -- 172.21.15.112:0/2113741973 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994401d660 con 0x7f99481130f0 2024-05-10T10:46:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.321+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99481a6fd0 con 0x7f99481130f0 2024-05-10T10:46:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.322+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99481a73d0 con 0x7f99481130f0 2024-05-10T10:46:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.324+0000 7f993effd640 1 -- 172.21.15.112:0/2113741973 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9944004950 con 0x7f99481130f0 2024-05-10T10:46:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.325+0000 7f993effd640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99340b2920 0x7f99340b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.325+0000 7f993effd640 1 -- 172.21.15.112:0/2113741973 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f99440ea170 con 0x7f99481130f0 2024-05-10T10:46:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.325+0000 7f994dad4640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99340b2920 0x7f99340b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 Processor -- start 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99340b2920 0x7f99340b4de0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9940005fd0 tx=0x7f9940009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 -- start start 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b1756871d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1756877a0 0x55b17568c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b17568c910 0x55b17568eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1755eb5d0 con 0x55b175686db0 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b1755a4890 con 0x55b1756877a0 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f9952a55740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b1755e7590 con 0x55b17568c910 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b1756871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b1756871d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43290/0 (socket says 172.21.15.112:43290) 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994d2d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1756877a0 0x55b17568c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 -- 172.21.15.112:0/2614193472 learned_addr learned my addr 172.21.15.112:0/2614193472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994e2d5640 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b17568c910 0x55b17568eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 -- 172.21.15.112:0/2614193472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b17568c910 msgr2=0x55b17568eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b17568c910 0x55b17568eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 -- 172.21.15.112:0/2614193472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1756877a0 msgr2=0x55b17568c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1756877a0 0x55b17568c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994dad4640 1 -- 172.21.15.112:0/2614193472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1752fd210 con 0x55b175686db0 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.326+0000 7f994d2d3640 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1756877a0 0x55b17568c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.327+0000 7f994dad4640 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b1756871d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9940003d10 tx=0x7f9940014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2faf21f8673d691 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.327+0000 7f9933fff640 1 -- 172.21.15.112:0/2614193472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940027670 con 0x55b175686db0 2024-05-10T10:46:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.327+0000 7f9933fff640 1 -- 172.21.15.112:0/2614193472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9940022070 con 0x55b175686db0 2024-05-10T10:46:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.327+0000 7f9933fff640 1 -- 172.21.15.112:0/2614193472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994001d990 con 0x55b175686db0 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 -- 172.21.15.112:0/2614193472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 msgr2=0x55b1756871d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b1756871d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9940003d10 tx=0x7f9940014000 comp rx=0 tx=0).stop 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 -- 172.21.15.112:0/2614193472 shutdown_connections 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b17568c910 0x55b17568eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1756877a0 0x55b17568c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 --2- 172.21.15.112:0/2614193472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b1756871d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 -- 172.21.15.112:0/2614193472 >> 172.21.15.112:0/2614193472 conn(0x55b175682420 msgr2=0x55b175682820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 -- 172.21.15.112:0/2614193472 shutdown_connections 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 -- 172.21.15.112:0/2614193472 wait complete. 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.328+0000 7f9952a55740 1 -- start start 2024-05-10T10:46:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 1 Processor -- start 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175690ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b175691530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f994dad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175690ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f994dad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175690ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43306/0 (socket says 172.21.15.112:43306) 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175698570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f994dad4640 1 -- 172.21.15.112:0/289178765 learned_addr learned my addr 172.21.15.112:0/289178765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b17568f8c0 con 0x55b175686db0 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b1755a8370 con 0x55b17568c910 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b17568fa40 con 0x55b1756877a0 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f994d2d3640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b175691530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994dad4640 1 -- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 msgr2=0x55b175691530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994e2d5640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175698570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.329+0000 7f9952a55740 10 client.0 _ll_register_callbacks cb 0x55b1752fca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994dad4640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b175691530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994dad4640 1 -- 172.21.15.112:0/289178765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 msgr2=0x55b175698570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994dad4640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175698570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994dad4640 1 -- 172.21.15.112:0/289178765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1753c9620 con 0x55b175686db0 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994d2d3640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b175691530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994e2d5640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175698570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9940022070 con 0x55b175686db0 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f994dad4640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175690ff0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9940011ca0 tx=0x7f9940009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d1bb986f9ff8f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f99317fa640 10 client.16920 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f99317fa640 1 -- 172.21.15.112:0/289178765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940004c50 con 0x55b175686db0 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f99317fa640 1 -- 172.21.15.112:0/289178765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9940004df0 con 0x55b175686db0 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f99317fa640 1 -- 172.21.15.112:0/289178765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9940005050 con 0x55b175686db0 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f9952a55740 10 client.16920 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.330+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55b17569b240 con 0x55b175686db0 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f99317fa640 1 -- 172.21.15.112:0/289178765 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f99400051f0 con 0x55b175686db0 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f9952a55740 10 client.16920 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f9952a55740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f9952a55740 20 client.16920 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f9952a55740 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b17569e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f9952a55740 4 client.16920 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1755f91e0 con 0x55b17569c590 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f994d2d3640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b17569e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f99317fa640 10 client.16920 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.331+0000 7f994d2d3640 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b17569e980 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6b20ba06b964e4 server_cookie=77ca3c5f9d5c1539 in_seq=0 out_seq=0 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f99317fa640 1 -- 172.21.15.112:0/289178765 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1755f91e0 con 0x55b17569c590 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f99317fa640 10 client.16920 handle_command_reply: tid=0 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 client.16920 shutdown 2024-05-10T10:46:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 msgr2=0x55b175690ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175690ff0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9940011ca0 tx=0x7f9940009e00 comp rx=0 tx=0).stop 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 shutdown_connections 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b17569e980 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f99317fa640 0 client.16920 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175698570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b175691530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 --2- 172.21.15.112:0/289178765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175690ff0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 >> 172.21.15.112:0/289178765 conn(0x55b175682420 msgr2=0x55b175681580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 shutdown_connections 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 1 -- 172.21.15.112:0/289178765 wait complete. 2024-05-10T10:46:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.332+0000 7f9952a55740 20 client.16920 trim_cache size 0 max 16384 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 Processor -- start 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 -- start start 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b175686db0 0x55b17567d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1756877a0 0x55b175690c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175691140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b19d0 con 0x55b1756877a0 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b3510 con 0x55b17568c910 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.348+0000 7f9952a55740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b20a0 con 0x55b175686db0 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1756877a0 0x55b175690c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1756877a0 0x55b175690c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43308/0 (socket says 172.21.15.112:43308) 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 -- 172.21.15.112:0/738162244 learned_addr learned my addr 172.21.15.112:0/738162244 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994dad4640 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b175686db0 0x55b17567d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994e2d5640 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175691140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 -- 172.21.15.112:0/738162244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b175686db0 msgr2=0x55b17567d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b175686db0 0x55b17567d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 -- 172.21.15.112:0/738162244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 msgr2=0x55b175691140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175691140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 -- 172.21.15.112:0/738162244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17568f160 con 0x55b1756877a0 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f994d2d3640 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1756877a0 0x55b175690c00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f993802a9c0 tx=0x7f9938005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657d07dbd12b484b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f99337fe640 1 -- 172.21.15.112:0/738162244 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993804a020 con 0x55b1756877a0 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f99337fe640 1 -- 172.21.15.112:0/738162244 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b1755f91e0 con 0x55b1756877a0 2024-05-10T10:46:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.349+0000 7f99337fe640 1 -- 172.21.15.112:0/738162244 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938035040 con 0x55b1756877a0 2024-05-10T10:46:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 -- 172.21.15.112:0/738162244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1756877a0 msgr2=0x55b175690c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1756877a0 0x55b175690c00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f993802a9c0 tx=0x7f9938005a20 comp rx=0 tx=0).stop 2024-05-10T10:46:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 -- 172.21.15.112:0/738162244 shutdown_connections 2024-05-10T10:46:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b175691140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1756877a0 0x55b175690c00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 --2- 172.21.15.112:0/738162244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b175686db0 0x55b17567d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 -- 172.21.15.112:0/738162244 >> 172.21.15.112:0/738162244 conn(0x55b175682420 msgr2=0x55b17568fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 -- 172.21.15.112:0/738162244 shutdown_connections 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 -- 172.21.15.112:0/738162244 wait complete. 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.350+0000 7f9952a55740 1 -- start start 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 1 Processor -- start 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175694d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b1756952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b1756962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b2e40 con 0x55b175686db0 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b2770 con 0x55b17568c910 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f99481af9b0 con 0x55b1756877a0 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175694d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994d2d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b1756952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175694d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43318/0 (socket says 172.21.15.112:43318) 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f9952a55740 10 client.0 _ll_register_callbacks cb 0x7f9938009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 -- 172.21.15.112:0/4076289453 learned_addr learned my addr 172.21.15.112:0/4076289453 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994e2d5640 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b1756962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 -- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 msgr2=0x55b1756952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b1756952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 -- 172.21.15.112:0/4076289453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 msgr2=0x55b1756962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b1756962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994dad4640 1 -- 172.21.15.112:0/4076289453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1756968b0 con 0x55b175686db0 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.351+0000 7f994d2d3640 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b1756952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f994dad4640 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175694d70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9940003b50 tx=0x7f9940009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7546f2d734fd904 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f99317fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f99317fa640 1 -- 172.21.15.112:0/4076289453 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940003e80 con 0x55b175686db0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f99317fa640 1 -- 172.21.15.112:0/4076289453 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9940004c50 con 0x55b175686db0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b1755f91e0 con 0x55b175686db0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f99317fa640 1 -- 172.21.15.112:0/4076289453 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f994001d920 con 0x55b175686db0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f9952a55740 10 client.16926 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b175697c40 con 0x55b175686db0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.352+0000 7f99317fa640 1 -- 172.21.15.112:0/4076289453 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b1755f91e0 con 0x55b175686db0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f99317fa640 1 -- 172.21.15.112:0/4076289453 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f9940027c00 con 0x55b175686db0 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f9952a55740 10 client.16926 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f9952a55740 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f9952a55740 20 client.16926 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f9952a55740 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b175709240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f9952a55740 4 client.16926 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:46:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1755f3050 con 0x55b17569c590 2024-05-10T10:46:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f994d2d3640 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b175709240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f99317fa640 10 client.16926 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.353+0000 7f994d2d3640 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b175709240 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fcdfa77f8b82dbd server_cookie=2f9099403e38c096 in_seq=0 out_seq=0 2024-05-10T10:46:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f99317fa640 1 -- 172.21.15.112:0/4076289453 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1755f3050 con 0x55b17569c590 2024-05-10T10:46:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f99317fa640 10 client.16926 handle_command_reply: tid=0 2024-05-10T10:46:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 client.16926 shutdown 2024-05-10T10:46:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 msgr2=0x55b175694d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175694d70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9940003b50 tx=0x7f9940009990 comp rx=0 tx=0).stop 2024-05-10T10:46:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 shutdown_connections 2024-05-10T10:46:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55b17569c590 0x55b175709240 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b17568c910 0x55b1756962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b1756877a0 0x55b1756952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 --2- 172.21.15.112:0/4076289453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b175686db0 0x55b175694d70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f99317fa640 0 client.16926 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 >> 172.21.15.112:0/4076289453 conn(0x55b175682420 msgr2=0x55b17568bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 shutdown_connections 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.378+0000 7f9952a55740 1 -- 172.21.15.112:0/4076289453 wait complete. 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f9952a55740 20 client.16926 trim_cache size 0 max 16384 2024-05-10T10:46:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:46:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:46:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:46:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:46:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99340b2920 msgr2=0x7f99340b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99340b2920 0x7f99340b4de0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9940005fd0 tx=0x7f9940009500 comp rx=0 tx=0).stop 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99481130f0 msgr2=0x7f99481a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99481130f0 0x7f99481a6670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f994400fd70 tx=0x7f9944015f90 comp rx=0 tx=0).stop 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 shutdown_connections 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99340b2920 0x7f99340b4de0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99481130f0 0x7f99481a6670 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f994810c320 0x7f994819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 --2- 172.21.15.112:0/2113741973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99481099b0 0x7f994819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 >> 172.21.15.112:0/2113741973 conn(0x7f994806fa20 msgr2=0x7f99481139a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 shutdown_connections 2024-05-10T10:46:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:42.379+0000 7f994fd38640 1 -- 172.21.15.112:0/2113741973 wait complete. 2024-05-10T10:46:42.818 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:42 smithi112 ceph-mon[30100]: pgmap v624: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 729 op/s 2024-05-10T10:46:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.851+0000 7f7f69248700 1 -- 172.21.15.112:0/297908540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f64107730 msgr2=0x7f7f6410f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.851+0000 7f7f69248700 1 --2- 172.21.15.112:0/297908540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f64107730 0x7f7f6410f2d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00b0d0 tx=0x7f7f5c00b3e0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.852+0000 7f7f69248700 1 -- 172.21.15.112:0/297908540 shutdown_connections 2024-05-10T10:46:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.852+0000 7f7f69248700 1 --2- 172.21.15.112:0/297908540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f64107730 0x7f7f6410f2d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.852+0000 7f7f69248700 1 --2- 172.21.15.112:0/297908540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f641029e0 0x7f7f64102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.852+0000 7f7f69248700 1 --2- 172.21.15.112:0/297908540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 0x7f7f64107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.852+0000 7f7f69248700 1 -- 172.21.15.112:0/297908540 >> 172.21.15.112:0/297908540 conn(0x7f7f64079350 msgr2=0x7f7f64079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.852+0000 7f7f69248700 1 -- 172.21.15.112:0/297908540 shutdown_connections 2024-05-10T10:46:42.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.852+0000 7f7f69248700 1 -- 172.21.15.112:0/297908540 wait complete. 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 Processor -- start 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 -- start start 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f641029e0 0x7f7f6419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 0x7f7f6419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f64107730 0x7f7f641998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f62d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f641029e0 0x7f7f6419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f64112470 con 0x7f7f641029e0 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f6259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 0x7f7f6419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f6259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 0x7f7f6419fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:45722/0 (socket says 172.21.15.112:45722) 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f6259c700 1 -- 172.21.15.112:0/3847771522 learned_addr learned my addr 172.21.15.112:0/3847771522 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641122f0 con 0x7f7f64103390 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f62d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f641029e0 0x7f7f6419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43334/0 (socket says 172.21.15.112:43334) 2024-05-10T10:46:42.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7f641125f0 con 0x7f7f64107730 2024-05-10T10:46:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.853+0000 7f7f6359e700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f64107730 0x7f7f641998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f6259c700 1 -- 172.21.15.112:0/3847771522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f64107730 msgr2=0x7f7f641998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f6259c700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f64107730 0x7f7f641998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f6259c700 1 -- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f641029e0 msgr2=0x7f7f6419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f6259c700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f641029e0 0x7f7f6419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f6259c700 1 -- 172.21.15.112:0/3847771522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6419a000 con 0x7f7f64103390 2024-05-10T10:46:42.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f6359e700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f64107730 0x7f7f641998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f6259c700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 0x7f7f6419fc20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800e940 tx=0x7f7f5800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f4bfff700 1 -- 172.21.15.112:0/3847771522 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800c9e0 con 0x7f7f64103390 2024-05-10T10:46:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f4bfff700 1 -- 172.21.15.112:0/3847771522 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f5800cb80 con 0x7f7f64103390 2024-05-10T10:46:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6419a2f0 con 0x7f7f64103390 2024-05-10T10:46:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f4bfff700 1 -- 172.21.15.112:0/3847771522 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58018420 con 0x7f7f64103390 2024-05-10T10:46:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.854+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f641a65b0 con 0x7f7f64103390 2024-05-10T10:46:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.855+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f64103ae0 con 0x7f7f64103390 2024-05-10T10:46:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.857+0000 7f7f4bfff700 1 -- 172.21.15.112:0/3847771522 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7f5804aab0 con 0x7f7f64103390 2024-05-10T10:46:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.857+0000 7f7f4bfff700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7f4c0b2880 0x7f7f4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.858+0000 7f7f62d9d700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7f4c0b2880 0x7f7f4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.858+0000 7f7f4bfff700 1 -- 172.21.15.112:0/3847771522 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f7f580e2680 con 0x7f7f64103390 2024-05-10T10:46:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.858+0000 7f7f62d9d700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7f4c0b2880 0x7f7f4c0b4d40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f6419a9e0 tx=0x7f7f5400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:42.861+0000 7f7f4bfff700 1 -- 172.21.15.112:0/3847771522 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7f580aa010 con 0x7f7f64103390 2024-05-10T10:46:43.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:42 smithi148 ceph-mon[33568]: pgmap v624: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 729 op/s 2024-05-10T10:46:43.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:42 smithi195 ceph-mon[33773]: pgmap v624: 129 pgs: 129 active+clean; 81 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 729 op/s 2024-05-10T10:46:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.218+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f6402e510 con 0x7f7f64103390 2024-05-10T10:46:43.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.219+0000 7f7f4bfff700 1 -- 172.21.15.112:0/3847771522 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11953 (secure 0 0 0) 0x7f7f580b2090 con 0x7f7f64103390 2024-05-10T10:46:43.221 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:46:43.221 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:46:09.581136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7f4c0b2880 msgr2=0x7f7f4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7f4c0b2880 0x7f7f4c0b4d40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f6419a9e0 tx=0x7f7f5400b3c0 comp rx=0 tx=0).stop 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 msgr2=0x7f7f6419fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 0x7f7f6419fc20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800e940 tx=0x7f7f5800ec50 comp rx=0 tx=0).stop 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 shutdown_connections 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f7f4c0b2880 0x7f7f4c0b4d40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f64107730 0x7f7f641998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f641029e0 0x7f7f6419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:43.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 --2- 172.21.15.112:0/3847771522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f64103390 0x7f7f6419fc20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:43.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.222+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 >> 172.21.15.112:0/3847771522 conn(0x7f7f64079350 msgr2=0x7f7f6406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:43.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.223+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 shutdown_connections 2024-05-10T10:46:43.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:43.223+0000 7f7f69248700 1 -- 172.21.15.112:0/3847771522 wait complete. 2024-05-10T10:46:43.225 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 42 2024-05-10T10:46:44.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:43 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3847771522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:46:44.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:43 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3847771522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:46:44.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:43 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3847771522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:46:44.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:46:45.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:44 smithi148 ceph-mon[33568]: pgmap v625: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.6 MiB/s wr, 530 op/s 2024-05-10T10:46:45.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:44 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:46:45.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:44 smithi195 ceph-mon[33773]: pgmap v625: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.6 MiB/s wr, 530 op/s 2024-05-10T10:46:45.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:44 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:46:45.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:44 smithi112 ceph-mon[30100]: pgmap v625: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.6 MiB/s wr, 530 op/s 2024-05-10T10:46:45.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:44 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:46:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:46:46.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:45 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:46:46.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:45 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:46:46.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:45 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:46:46.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:45 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:46:46.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:45 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:46:46.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:45 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:46:46.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:46:46.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:46:46.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:45 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.215+0000 7f0c62522700 1 -- 172.21.15.112:0/3318341366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06d770 msgr2=0x7f0c5c11c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.215+0000 7f0c62522700 1 --2- 172.21.15.112:0/3318341366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06d770 0x7f0c5c11c760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c500098f0 tx=0x7f0c50009c00 comp rx=0 tx=0).stop 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.215+0000 7f0c62522700 1 -- 172.21.15.112:0/3318341366 shutdown_connections 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.215+0000 7f0c62522700 1 --2- 172.21.15.112:0/3318341366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c5c11cca0 0x7f0c5c11f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.215+0000 7f0c62522700 1 --2- 172.21.15.112:0/3318341366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c5c06ce50 0x7f0c5c06d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.215+0000 7f0c62522700 1 --2- 172.21.15.112:0/3318341366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06d770 0x7f0c5c11c760 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.215+0000 7f0c62522700 1 -- 172.21.15.112:0/3318341366 >> 172.21.15.112:0/3318341366 conn(0x7f0c5c074af0 msgr2=0x7f0c5c074f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:47.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.216+0000 7f0c62522700 1 -- 172.21.15.112:0/3318341366 shutdown_connections 2024-05-10T10:46:47.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.217+0000 7f0c62522700 1 -- 172.21.15.112:0/3318341366 wait complete. 2024-05-10T10:46:47.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.218+0000 7f0c62522700 1 Processor -- start 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c62522700 1 -- start start 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c62522700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06ce50 0x7f0c5c1ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c62522700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c5c11cca0 0x7f0c5c1aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c62522700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c5c1a8a20 0x7f0c5c1a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c62522700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c1223d0 con 0x7f0c5c1a8a20 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c62522700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c122250 con 0x7f0c5c06ce50 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c62522700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c122550 con 0x7f0c5c11cca0 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.220+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06ce50 0x7f0c5c1ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06ce50 0x7f0c5c1ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55324/0 (socket says 172.21.15.112:55324) 2024-05-10T10:46:47.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 -- 172.21.15.112:0/3598016182 learned_addr learned my addr 172.21.15.112:0/3598016182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 -- 172.21.15.112:0/3598016182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c5c11cca0 msgr2=0x7f0c5c1aeda0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T10:46:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c5c11cca0 0x7f0c5c1aeda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 -- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c5c1a8a20 msgr2=0x7f0c5c1a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c5c1a8a20 0x7f0c5c1a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 -- 172.21.15.112:0/3598016182 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c1a95a0 con 0x7f0c5c06ce50 2024-05-10T10:46:47.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.221+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06ce50 0x7f0c5c1ae860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400ef10 tx=0x7f0c5400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:47.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.223+0000 7f0c597fa700 1 -- 172.21.15.112:0/3598016182 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400cad0 con 0x7f0c5c06ce50 2024-05-10T10:46:47.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.223+0000 7f0c597fa700 1 -- 172.21.15.112:0/3598016182 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c54019070 con 0x7f0c5c06ce50 2024-05-10T10:46:47.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.223+0000 7f0c597fa700 1 -- 172.21.15.112:0/3598016182 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c540146e0 con 0x7f0c5c06ce50 2024-05-10T10:46:47.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.223+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5c1a9830 con 0x7f0c5c06ce50 2024-05-10T10:46:47.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.224+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5c1b5a20 con 0x7f0c5c06ce50 2024-05-10T10:46:47.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.228+0000 7f0c597fa700 1 -- 172.21.15.112:0/3598016182 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0c54012050 con 0x7f0c5c06ce50 2024-05-10T10:46:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.230+0000 7f0c597fa700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c440b29c0 0x7f0c440b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.230+0000 7f0c597fa700 1 -- 172.21.15.112:0/3598016182 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c540e25d0 con 0x7f0c5c06ce50 2024-05-10T10:46:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.231+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c440b29c0 0x7f0c440b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 Processor -- start 2024-05-10T10:46:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 -- start start 2024-05-10T10:46:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c440b29c0 0x7f0c440b4e80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0c500098c0 tx=0x7f0c50005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cbb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cbf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbfc80 0x55d280cc0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d280c0d860 con 0x55d280cbbdf0 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d280bac7c0 con 0x55d280cbb4d0 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d280c09820 con 0x55d280cbfc80 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c5b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cbf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c5b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cbf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51878/0 (socket says 172.21.15.112:51878) 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/791349999 learned_addr learned my addr 172.21.15.112:0/791349999 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c60abf700 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbfc80 0x55d280cc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.232+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cbb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/791349999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbfc80 msgr2=0x55d280cc0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbfc80 0x55d280cc0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/791349999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 msgr2=0x55d280cbb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cbb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/791349999 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d280b5a8d0 con 0x55d280cbbdf0 2024-05-10T10:46:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cbf6f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0c500098f0 tx=0x7f0c5001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbc94bb24a16c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c427fc700 1 -- 172.21.15.112:0/791349999 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000f760 con 0x55d280cbbdf0 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.233+0000 7f0c427fc700 1 -- 172.21.15.112:0/791349999 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c5000f900 con 0x55d280cbbdf0 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c427fc700 1 -- 172.21.15.112:0/791349999 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50003d40 con 0x55d280cbbdf0 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/791349999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 msgr2=0x55d280cbf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cbf6f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0c500098f0 tx=0x7f0c5001b000 comp rx=0 tx=0).stop 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/791349999 shutdown_connections 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbfc80 0x55d280cc0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cbb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/791349999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cbf6f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.234+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/791349999 >> 172.21.15.112:0/791349999 conn(0x55d280be7100 msgr2=0x55d280cb5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.235+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/791349999 shutdown_connections 2024-05-10T10:46:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.235+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/791349999 wait complete. 2024-05-10T10:46:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.235+0000 7f0c6cdb5b80 1 -- start start 2024-05-10T10:46:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 1 Processor -- start 2024-05-10T10:46:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cc6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbbdf0 0x55d280cc43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280cc4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d280cc7e80 con 0x55d280cbfc80 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d280b73c30 con 0x55d280cbbdf0 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d280cc8000 con 0x55d280cbb4d0 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c6cdb5b80 10 client.0 _ll_register_callbacks cb 0x55d280ba5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c5b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbbdf0 0x55d280cc43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.236+0000 7f0c60abf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280cc4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbbdf0 0x55d280cc43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55344/0 (socket says 172.21.15.112:55344) 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c60abf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280cc4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51884/0 (socket says 172.21.15.112:51884) 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/922993392 learned_addr learned my addr 172.21.15.112:0/922993392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cc6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/922993392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 msgr2=0x55d280cc6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cc6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/922993392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 msgr2=0x55d280cc4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280cc4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/922993392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d280ba9690 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbbdf0 0x55d280cc43e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c50003c90 tx=0x7f0c50003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48134e20369045c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/922993392 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50005450 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d280c87650 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.237+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/922993392 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c500055f0 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/922993392 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0c5003f050 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 10 client.26309 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d280cc7e80 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/922993392 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d280c87650 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/922993392 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f0c50025420 con 0x55d280cbbdf0 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 10 client.26309 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 10 client.26309 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:46:47.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 20 client.26309 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280cd4980 0x55d280cd6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 4 client.26309 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d280c1b240 con 0x55d280cd4980 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.238+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280cd4980 0x55d280cd6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.239+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280cd4980 0x55d280cd6d70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbc5229747b2a627 server_cookie=fc5dccc8f2ebe21a in_seq=0 out_seq=0 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.239+0000 7f0c3b7fe700 10 client.26309 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.239+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/922993392 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d280c1b240 con 0x55d280cd4980 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.239+0000 7f0c3b7fe700 10 client.26309 handle_command_reply: tid=0 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.239+0000 7f0c6cdb5b80 1 client.26309 shutdown 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbbdf0 msgr2=0x55d280cc43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbbdf0 0x55d280cc43e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c50003c90 tx=0x7f0c50003d70 comp rx=0 tx=0).stop 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 shutdown_connections 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280cc4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280cd4980 0x55d280cd6d70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cc6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/922993392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbbdf0 0x55d280cc43e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 >> 172.21.15.112:0/922993392 conn(0x55d280be7100 msgr2=0x55d280cb0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:47.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c3b7fe700 0 client.26309 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:46:47.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 shutdown_connections 2024-05-10T10:46:47.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/922993392 wait complete. 2024-05-10T10:46:47.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.240+0000 7f0c6cdb5b80 20 client.26309 trim_cache size 0 max 16384 2024-05-10T10:46:47.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 Processor -- start 2024-05-10T10:46:47.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 -- start start 2024-05-10T10:46:47.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cb7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cd0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbfc80 0x55d280cbd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c1be6b0 con 0x55d280cbbdf0 2024-05-10T10:46:47.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c1c01f0 con 0x55d280cbfc80 2024-05-10T10:46:47.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.259+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c1bed80 con 0x55d280cbb4d0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cb7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cb7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44384/0 (socket says 172.21.15.112:44384) 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 -- 172.21.15.112:0/1219813656 learned_addr learned my addr 172.21.15.112:0/1219813656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cd0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c60abf700 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbfc80 0x55d280cbd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 -- 172.21.15.112:0/1219813656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbfc80 msgr2=0x55d280cbd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbfc80 0x55d280cbd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 -- 172.21.15.112:0/1219813656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 msgr2=0x55d280cd0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cd0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 -- 172.21.15.112:0/1219813656 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d280cc6ff0 con 0x55d280cbb4d0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c60abf700 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbfc80 0x55d280cbd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cb7740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400eee0 tx=0x7f0c5400c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc1eaf5554bf8d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c41ffb700 1 -- 172.21.15.112:0/1219813656 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c540c2020 con 0x55d280cbb4d0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c41ffb700 1 -- 172.21.15.112:0/1219813656 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c540a9800 con 0x55d280cbb4d0 2024-05-10T10:46:47.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.260+0000 7f0c41ffb700 1 -- 172.21.15.112:0/1219813656 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400cc70 con 0x55d280cbb4d0 2024-05-10T10:46:47.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.261+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/1219813656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 msgr2=0x55d280cb7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.261+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cb7740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400eee0 tx=0x7f0c5400c950 comp rx=0 tx=0).stop 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/1219813656 shutdown_connections 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbfc80 0x55d280cbd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbb4d0 0x55d280cb7740 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/1219813656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbbdf0 0x55d280cd0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/1219813656 >> 172.21.15.112:0/1219813656 conn(0x55d280be7100 msgr2=0x55d280cb0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/1219813656 shutdown_connections 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/1219813656 wait complete. 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 -- start start 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.262+0000 7f0c6cdb5b80 1 Processor -- start 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbbdf0 0x55d280cca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c6cdb5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280ccaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c1bfb20 con 0x55d280cbfc80 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c1bf450 con 0x55d280cbb4d0 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c6cdb5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c1bc650 con 0x55d280cbbdf0 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c6cdb5b80 10 client.0 _ll_register_callbacks cb 0x55d280ba5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbbdf0 0x55d280cca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbbdf0 0x55d280cca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44396/0 (socket says 172.21.15.112:44396) 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/2155489915 learned_addr learned my addr 172.21.15.112:0/2155489915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c60abf700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280ccaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 msgr2=0x55d280cca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/2155489915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 msgr2=0x55d280ccaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:47.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280ccaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 -- 172.21.15.112:0/2155489915 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d280d3d2a0 con 0x55d280cbbdf0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c5b7fe700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbbdf0 0x55d280cca560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c50005d90 tx=0x7f0c50025d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966ab1529396df2e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.263+0000 7f0c3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.264+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/2155489915 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50005250 con 0x55d280cbbdf0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.264+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/2155489915 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c500053f0 con 0x55d280cbbdf0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.264+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c540a9800 con 0x55d280cbbdf0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.264+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/2155489915 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0c50053050 con 0x55d280cbbdf0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.264+0000 7f0c6cdb5b80 10 client.26323 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.264+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d280d3e4f0 con 0x55d280cbbdf0 2024-05-10T10:46:47.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.264+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/2155489915 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c540a9800 con 0x55d280cbbdf0 2024-05-10T10:46:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/2155489915 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f0c5000ba80 con 0x55d280cbbdf0 2024-05-10T10:46:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c6cdb5b80 10 client.26323 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c6cdb5b80 10 client.26323 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:46:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c6cdb5b80 20 client.26323 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280d3eb30 0x55d280d40f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c6cdb5b80 4 client.26323 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d280c152e0 con 0x55d280d3eb30 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280d3eb30 0x55d280d40f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c3b7fe700 10 client.26323 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c5bfff700 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280d3eb30 0x55d280d40f50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12724991d0af4ec7 server_cookie=476b558911cd2475 in_seq=0 out_seq=0 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c3b7fe700 1 -- 172.21.15.112:0/2155489915 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d280c152e0 con 0x55d280d3eb30 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.265+0000 7f0c3b7fe700 10 client.26323 handle_command_reply: tid=0 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 client.26323 shutdown 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbbdf0 msgr2=0x55d280cca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbbdf0 0x55d280cca560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c50005d90 tx=0x7f0c50025d60 comp rx=0 tx=0).stop 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 shutdown_connections 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d280cbfc80 0x55d280ccaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d280cbb4d0 0x55d280cca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d280d3eb30 0x55d280d40f50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 --2- 172.21.15.112:0/2155489915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d280cbbdf0 0x55d280cca560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c3b7fe700 0 client.26323 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:46:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 >> 172.21.15.112:0/2155489915 conn(0x55d280be7100 msgr2=0x55d280cbe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 shutdown_connections 2024-05-10T10:46:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 1 -- 172.21.15.112:0/2155489915 wait complete. 2024-05-10T10:46:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.266+0000 7f0c6cdb5b80 20 client.26323 trim_cache size 0 max 16384 2024-05-10T10:46:47.272 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c440b29c0 msgr2=0x7f0c440b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c440b29c0 0x7f0c440b4e80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0c500098c0 tx=0x7f0c50005d20 comp rx=0 tx=0).stop 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06ce50 msgr2=0x7f0c5c1ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06ce50 0x7f0c5c1ae860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400ef10 tx=0x7f0c5400c3a0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 shutdown_connections 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c5c1a8a20 0x7f0c5c1a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c5c06ce50 0x7f0c5c1ae860 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c440b29c0 0x7f0c440b4e80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 --2- 172.21.15.112:0/3598016182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c5c11cca0 0x7f0c5c1aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:47.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 >> 172.21.15.112:0/3598016182 conn(0x7f0c5c074af0 msgr2=0x7f0c5c11e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:47.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 shutdown_connections 2024-05-10T10:46:47.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:47.267+0000 7f0c62522700 1 -- 172.21.15.112:0/3598016182 wait complete. 2024-05-10T10:46:47.283 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:47 smithi112 ceph-mon[30100]: pgmap v626: 129 pgs: 129 active+clean; 57 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 715 op/s 2024-05-10T10:46:47.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:47 smithi148 ceph-mon[33568]: pgmap v626: 129 pgs: 129 active+clean; 57 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 715 op/s 2024-05-10T10:46:47.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:47 smithi195 ceph-mon[33773]: pgmap v626: 129 pgs: 129 active+clean; 57 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 715 op/s 2024-05-10T10:46:48.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:46:49.070 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:46:49.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:49 smithi148 ceph-mon[33568]: pgmap v627: 129 pgs: 129 active+clean; 57 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 496 op/s 2024-05-10T10:46:49.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:49 smithi195 ceph-mon[33773]: pgmap v627: 129 pgs: 129 active+clean; 57 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 496 op/s 2024-05-10T10:46:49.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:49 smithi112 ceph-mon[30100]: pgmap v627: 129 pgs: 129 active+clean; 57 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 496 op/s 2024-05-10T10:46:51.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 -- 172.21.15.112:0/151776700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1023a0 msgr2=0x7f776c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 --2- 172.21.15.112:0/151776700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1023a0 0x7f776c102820 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f775c009990 tx=0x7f775c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 -- 172.21.15.112:0/151776700 shutdown_connections 2024-05-10T10:46:51.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 --2- 172.21.15.112:0/151776700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c102d60 0x7f776c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 --2- 172.21.15.112:0/151776700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f776c108410 0x7f776c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 --2- 172.21.15.112:0/151776700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1023a0 0x7f776c102820 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 -- 172.21.15.112:0/151776700 >> 172.21.15.112:0/151776700 conn(0x7f776c079350 msgr2=0x7f776c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:51.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 -- 172.21.15.112:0/151776700 shutdown_connections 2024-05-10T10:46:51.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.052+0000 7f77737c4700 1 -- 172.21.15.112:0/151776700 wait complete. 2024-05-10T10:46:51.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.053+0000 7f77737c4700 1 Processor -- start 2024-05-10T10:46:51.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.053+0000 7f77737c4700 1 -- start start 2024-05-10T10:46:51.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.053+0000 7f77737c4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c1023a0 0x7f776c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f77737c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f776c102d60 0x7f776c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f77737c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c108410 0x7f776c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f77737c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f776c112410 con 0x7f776c108410 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f77737c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f776c112290 con 0x7f776c102d60 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f77737c4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f776c112590 con 0x7f776c1023a0 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c1023a0 0x7f776c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c1023a0 0x7f776c19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44418/0 (socket says 172.21.15.112:44418) 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 -- 172.21.15.112:0/1608778107 learned_addr learned my addr 172.21.15.112:0/1608778107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771d61700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c108410 0x7f776c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 -- 172.21.15.112:0/1608778107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f776c102d60 msgr2=0x7f776c19ae50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f776c102d60 0x7f776c19ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 -- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c108410 msgr2=0x7f776c1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c108410 0x7f776c1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 -- 172.21.15.112:0/1608778107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c1a5ed0 con 0x7f776c1023a0 2024-05-10T10:46:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.054+0000 7f7771560700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c1023a0 0x7f776c19a910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9a0 tx=0x7f776800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.055+0000 7f77627fc700 1 -- 172.21.15.112:0/1608778107 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776800cbd0 con 0x7f776c1023a0 2024-05-10T10:46:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.055+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c1a60a0 con 0x7f776c1023a0 2024-05-10T10:46:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.055+0000 7f77627fc700 1 -- 172.21.15.112:0/1608778107 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f776800cd70 con 0x7f776c1023a0 2024-05-10T10:46:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.056+0000 7f77627fc700 1 -- 172.21.15.112:0/1608778107 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768018680 con 0x7f776c1023a0 2024-05-10T10:46:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.056+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c1a6570 con 0x7f776c1023a0 2024-05-10T10:46:51.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.058+0000 7f77627fc700 1 -- 172.21.15.112:0/1608778107 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7768010430 con 0x7f776c1023a0 2024-05-10T10:46:51.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.059+0000 7f77627fc700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f77580b2900 0x7f77580b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.059+0000 7f7770d5f700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f77580b2900 0x7f77580b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.059+0000 7f77627fc700 1 -- 172.21.15.112:0/1608778107 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f77680e2770 con 0x7f776c1023a0 2024-05-10T10:46:51.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f7770d5f700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f77580b2900 0x7f77580b4dc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098e0 tx=0x7f775c00b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 Processor -- start 2024-05-10T10:46:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 -- start start 2024-05-10T10:46:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23bad0 0x5580dc235810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc236190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5580dc187940 con 0x5580dc23bad0 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5580dc0ecda0 con 0x5580dc23b1b0 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f777e057b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5580dc183900 con 0x5580dc235d50 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f7771d61700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc236190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.060+0000 7f7770d5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23bad0 0x5580dc235810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc236190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44420/0 (socket says 172.21.15.112:44420) 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 -- 172.21.15.112:0/1919600937 learned_addr learned my addr 172.21.15.112:0/1919600937 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771560700 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 -- 172.21.15.112:0/1919600937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 msgr2=0x5580dc23b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 -- 172.21.15.112:0/1919600937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23bad0 msgr2=0x5580dc235810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23bad0 0x5580dc235810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 -- 172.21.15.112:0/1919600937 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580dc0d48d0 con 0x5580dc235d50 2024-05-10T10:46:51.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f7771d61700 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc236190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77640098f0 tx=0x7f7764009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d67744fa06c94abe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:51.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:51 smithi112 ceph-mon[30100]: pgmap v628: 129 pgs: 129 active+clean; 55 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 537 op/s 2024-05-10T10:46:51.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f77577fe700 1 -- 172.21.15.112:0/1919600937 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764004bb0 con 0x5580dc235d50 2024-05-10T10:46:51.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f77577fe700 1 -- 172.21.15.112:0/1919600937 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7764004d50 con 0x5580dc235d50 2024-05-10T10:46:51.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.061+0000 7f77577fe700 1 -- 172.21.15.112:0/1919600937 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764007cc0 con 0x5580dc235d50 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.062+0000 7f777e057b80 1 -- 172.21.15.112:0/1919600937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 msgr2=0x5580dc236190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.062+0000 7f777e057b80 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc236190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77640098f0 tx=0x7f7764009c00 comp rx=0 tx=0).stop 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 -- 172.21.15.112:0/1919600937 shutdown_connections 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc236190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 --2- 172.21.15.112:0/1919600937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23bad0 0x5580dc235810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 -- 172.21.15.112:0/1919600937 >> 172.21.15.112:0/1919600937 conn(0x5580dc161b70 msgr2=0x5580dc2302a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 -- 172.21.15.112:0/1919600937 shutdown_connections 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 -- 172.21.15.112:0/1919600937 wait complete. 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.063+0000 7f777e057b80 1 -- start start 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 1 Processor -- start 2024-05-10T10:46:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc2416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23bad0 0x5580dc23efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5580dc246860 con 0x5580dc235d50 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5580dc140ed0 con 0x5580dc23b1b0 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5580dc2469e0 con 0x5580dc23bad0 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f777e057b80 10 client.0 _ll_register_callbacks cb 0x5580dc11b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f7771d61700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23bad0 0x5580dc23efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f7771d61700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23bad0 0x5580dc23efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44434/0 (socket says 172.21.15.112:44434) 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f7771d61700 1 -- 172.21.15.112:0/378202235 learned_addr learned my addr 172.21.15.112:0/378202235 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f7771560700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc2416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.064+0000 7f7770d5f700 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7771d61700 1 -- 172.21.15.112:0/378202235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 msgr2=0x5580dc23ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7771d61700 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7771d61700 1 -- 172.21.15.112:0/378202235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 msgr2=0x5580dc2416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7771d61700 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc2416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7771d61700 1 -- 172.21.15.112:0/378202235 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580dc13e760 con 0x5580dc23bad0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7771d61700 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23bad0 0x5580dc23efd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7764003db0 tx=0x7f77640094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4edea07738789ee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7754ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7754ff9700 1 -- 172.21.15.112:0/378202235 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764022070 con 0x5580dc23bad0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7754ff9700 1 -- 172.21.15.112:0/378202235 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f77640047e0 con 0x5580dc23bad0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7764004d50 con 0x5580dc23bad0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f7754ff9700 1 -- 172.21.15.112:0/378202235 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f776402e050 con 0x5580dc23bad0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f777e057b80 10 client.26335 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.065+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5580dc242270 con 0x5580dc23bad0 2024-05-10T10:46:51.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f7754ff9700 1 -- 172.21.15.112:0/378202235 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7764004d50 con 0x5580dc23bad0 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f7754ff9700 1 -- 172.21.15.112:0/378202235 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f7764015af0 con 0x5580dc23bad0 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f777e057b80 10 client.26335 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f777e057b80 10 client.26335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f777e057b80 20 client.26335 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f777e057b80 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2447a0 0x5580dc244bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f777e057b80 4 client.26335 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580dc195320 con 0x5580dc2447a0 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f7771560700 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2447a0 0x5580dc244bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f7754ff9700 10 client.26335 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:46:51.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.066+0000 7f7771560700 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2447a0 0x5580dc244bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5521d694ec97a3fe server_cookie=8c75f590f6c48a89 in_seq=0 out_seq=0 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.067+0000 7f7754ff9700 1 -- 172.21.15.112:0/378202235 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580dc195320 con 0x5580dc2447a0 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.067+0000 7f7754ff9700 10 client.26335 handle_command_reply: tid=0 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.067+0000 7f777e057b80 1 client.26335 shutdown 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.067+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23bad0 msgr2=0x5580dc23efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.067+0000 7f777e057b80 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23bad0 0x5580dc23efd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7764003db0 tx=0x7f77640094d0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 shutdown_connections 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2447a0 0x5580dc244bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23bad0 0x5580dc23efd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc2416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f7754ff9700 0 client.26335 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:46:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 --2- 172.21.15.112:0/378202235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23b1b0 0x5580dc23ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 >> 172.21.15.112:0/378202235 conn(0x5580dc161b70 msgr2=0x5580dc22ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 shutdown_connections 2024-05-10T10:46:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 1 -- 172.21.15.112:0/378202235 wait complete. 2024-05-10T10:46:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.068+0000 7f777e057b80 20 client.26335 trim_cache size 0 max 16384 2024-05-10T10:46:51.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 Processor -- start 2024-05-10T10:46:51.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 -- start start 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc235810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23b1b0 0x5580dc2390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc22ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1ad0d0 con 0x5580dc235d50 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1aec10 con 0x5580dc23bad0 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f777e057b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1ad7a0 con 0x5580dc23b1b0 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f7771d61700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc22ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.087+0000 7f7771d61700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc22ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55406/0 (socket says 172.21.15.112:55406) 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771d61700 1 -- 172.21.15.112:0/3933349898 learned_addr learned my addr 172.21.15.112:0/3933349898 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7770d5f700 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23b1b0 0x5580dc2390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771560700 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc235810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771560700 1 -- 172.21.15.112:0/3933349898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23b1b0 msgr2=0x5580dc2390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771560700 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23b1b0 0x5580dc2390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771560700 1 -- 172.21.15.112:0/3933349898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 msgr2=0x5580dc22ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771560700 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc22ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771560700 1 -- 172.21.15.112:0/3933349898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580dc23e860 con 0x5580dc235d50 2024-05-10T10:46:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.088+0000 7f7771560700 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc235810 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f77680a96a0 tx=0x7f77680cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a32006ba96b91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.089+0000 7f7756ffd700 1 -- 172.21.15.112:0/3933349898 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680aa0a0 con 0x5580dc235d50 2024-05-10T10:46:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.089+0000 7f7756ffd700 1 -- 172.21.15.112:0/3933349898 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77680a99a0 con 0x5580dc235d50 2024-05-10T10:46:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.089+0000 7f7756ffd700 1 -- 172.21.15.112:0/3933349898 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680a9c00 con 0x5580dc235d50 2024-05-10T10:46:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 -- 172.21.15.112:0/3933349898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 msgr2=0x5580dc235810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc235810 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f77680a96a0 tx=0x7f77680cc040 comp rx=0 tx=0).stop 2024-05-10T10:46:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 -- 172.21.15.112:0/3933349898 shutdown_connections 2024-05-10T10:46:51.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc22ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc235d50 0x5580dc235810 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 --2- 172.21.15.112:0/3933349898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc23b1b0 0x5580dc2390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 -- 172.21.15.112:0/3933349898 >> 172.21.15.112:0/3933349898 conn(0x5580dc161b70 msgr2=0x5580dc23d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:51.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 -- 172.21.15.112:0/3933349898 shutdown_connections 2024-05-10T10:46:51.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.090+0000 7f777e057b80 1 -- 172.21.15.112:0/3933349898 wait complete. 2024-05-10T10:46:51.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.091+0000 7f777e057b80 1 -- start start 2024-05-10T10:46:51.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.091+0000 7f777e057b80 1 Processor -- start 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.091+0000 7f777e057b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc2607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.091+0000 7f777e057b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23b1b0 0x5580dc260d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.091+0000 7f777e057b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc25b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.091+0000 7f7770d5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23b1b0 0x5580dc260d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.091+0000 7f7771560700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc2607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc2607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44452/0 (socket says 172.21.15.112:44452) 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7770d5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23b1b0 0x5580dc260d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51940/0 (socket says 172.21.15.112:51940) 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 -- 172.21.15.112:0/155602324 learned_addr learned my addr 172.21.15.112:0/155602324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1ae540 con 0x5580dc23b1b0 2024-05-10T10:46:51.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1ade70 con 0x5580dc23bad0 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1ab070 con 0x5580dc235d50 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771d61700 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc25b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f777e057b80 10 client.0 _ll_register_callbacks cb 0x5580dc11b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 -- 172.21.15.112:0/155602324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 msgr2=0x5580dc25b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc25b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 -- 172.21.15.112:0/155602324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23b1b0 msgr2=0x5580dc260d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23b1b0 0x5580dc260d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 -- 172.21.15.112:0/155602324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580dc25bd90 con 0x5580dc235d50 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7771560700 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc2607f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77680a96a0 tx=0x7f77680b2230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde3b7abead2ebb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7754ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7754ff9700 1 -- 172.21.15.112:0/155602324 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680c4070 con 0x5580dc235d50 2024-05-10T10:46:51.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f7754ff9700 1 -- 172.21.15.112:0/155602324 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f776808f040 con 0x5580dc235d50 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.092+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77680a99a0 con 0x5580dc235d50 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f7754ff9700 1 -- 172.21.15.112:0/155602324 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f77680aa0a0 con 0x5580dc235d50 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 10 client.26341 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5580dc2c1bf0 con 0x5580dc235d50 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f7754ff9700 1 -- 172.21.15.112:0/155602324 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77680a99a0 con 0x5580dc235d50 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f7754ff9700 1 -- 172.21.15.112:0/155602324 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f77680b2370 con 0x5580dc235d50 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 10 client.26341 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 20 client.26341 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2c2240 0x5580dc2c4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 4 client.26341 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580dc18f3c0 con 0x5580dc2c2240 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.093+0000 7f7770d5f700 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2c2240 0x5580dc2c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:51.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f7754ff9700 10 client.26341 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f7770d5f700 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2c2240 0x5580dc2c4660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65f1eb2743449f16 server_cookie=7846775ed375e214 in_seq=0 out_seq=0 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f7754ff9700 1 -- 172.21.15.112:0/155602324 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580dc18f3c0 con 0x5580dc2c2240 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f7754ff9700 10 client.26341 handle_command_reply: tid=0 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 client.26341 shutdown 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 msgr2=0x5580dc2607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc2607f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77680a96a0 tx=0x7f77680b2230 comp rx=0 tx=0).stop 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 shutdown_connections 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x5580dc2c2240 0x5580dc2c4660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580dc23bad0 0x5580dc25b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5580dc235d50 0x5580dc2607f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 --2- 172.21.15.112:0/155602324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580dc23b1b0 0x5580dc260d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.094+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 >> 172.21.15.112:0/155602324 conn(0x5580dc161b70 msgr2=0x5580dc23db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.095+0000 7f7754ff9700 0 client.26341 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:46:51.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.095+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 shutdown_connections 2024-05-10T10:46:51.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.095+0000 7f777e057b80 1 -- 172.21.15.112:0/155602324 wait complete. 2024-05-10T10:46:51.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.095+0000 7f777e057b80 20 client.26341 trim_cache size 0 max 16384 2024-05-10T10:46:51.101 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:46:51.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f77580b2900 msgr2=0x7f77580b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f77580b2900 0x7f77580b4dc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098e0 tx=0x7f775c00b610 comp rx=0 tx=0).stop 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c1023a0 msgr2=0x7f776c19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c1023a0 0x7f776c19a910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9a0 tx=0x7f776800ecb0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 shutdown_connections 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c108410 0x7f776c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f77580b2900 0x7f77580b4dc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f776c1023a0 0x7f776c19a910 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9a0 tx=0x7f776800ecb0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 --2- 172.21.15.112:0/1608778107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f776c102d60 0x7f776c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 >> 172.21.15.112:0/1608778107 conn(0x7f776c079350 msgr2=0x7f776c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 shutdown_connections 2024-05-10T10:46:51.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:51.096+0000 7f77737c4700 1 -- 172.21.15.112:0/1608778107 wait complete. 2024-05-10T10:46:51.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:51 smithi148 ceph-mon[33568]: pgmap v628: 129 pgs: 129 active+clean; 55 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 537 op/s 2024-05-10T10:46:51.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:51 smithi195 ceph-mon[33773]: pgmap v628: 129 pgs: 129 active+clean; 55 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 537 op/s 2024-05-10T10:46:51.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:46:52.267 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:46:53.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:53 smithi195 ceph-mon[33773]: pgmap v629: 129 pgs: 129 active+clean; 61 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 694 op/s 2024-05-10T10:46:53.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:53 smithi112 ceph-mon[30100]: pgmap v629: 129 pgs: 129 active+clean; 61 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 694 op/s 2024-05-10T10:46:53.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:53 smithi148 ceph-mon[33568]: pgmap v629: 129 pgs: 129 active+clean; 61 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 694 op/s 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 -- 172.21.15.112:0/2904674822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044108440 msgr2=0x7f9044108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 --2- 172.21.15.112:0/2904674822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044108440 0x7f9044108820 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9034009990 tx=0x7f9034009ca0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 -- 172.21.15.112:0/2904674822 shutdown_connections 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 --2- 172.21.15.112:0/2904674822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9044102d90 0x7f904410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 --2- 172.21.15.112:0/2904674822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044108440 0x7f9044108820 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 --2- 172.21.15.112:0/2904674822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 0x7f9044102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 -- 172.21.15.112:0/2904674822 >> 172.21.15.112:0/2904674822 conn(0x7f90440793e0 msgr2=0x7f90440797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:54.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 -- 172.21.15.112:0/2904674822 shutdown_connections 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.656+0000 7f9049ea8700 1 -- 172.21.15.112:0/2904674822 wait complete. 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.657+0000 7f9049ea8700 1 Processor -- start 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9049ea8700 1 -- start start 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9049ea8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 0x7f904419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9049ea8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044102d90 0x7f904419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9049ea8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9044108440 0x7f9044199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9049ea8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9044112410 con 0x7f9044102d90 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9049ea8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9044112290 con 0x7f90441023d0 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9049ea8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9044112590 con 0x7f9044108440 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9042ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044102d90 0x7f904419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 0x7f904419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 0x7f904419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55422/0 (socket says 172.21.15.112:55422) 2024-05-10T10:46:54.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 -- 172.21.15.112:0/1035720252 learned_addr learned my addr 172.21.15.112:0/1035720252 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f9043fff700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9044108440 0x7f9044199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 -- 172.21.15.112:0/1035720252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9044108440 msgr2=0x7f9044199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9044108440 0x7f9044199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 -- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044102d90 msgr2=0x7f904419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044102d90 0x7f904419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.658+0000 7f90437fe700 1 -- 172.21.15.112:0/1035720252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9044199fe0 con 0x7f90441023d0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.659+0000 7f90437fe700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 0x7f904419f6c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9034006010 tx=0x7f90340047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.659+0000 7f9040ff9700 1 -- 172.21.15.112:0/1035720252 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034027070 con 0x7f90441023d0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.659+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904419a270 con 0x7f90441023d0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.659+0000 7f9040ff9700 1 -- 172.21.15.112:0/1035720252 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9034022470 con 0x7f90441023d0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.659+0000 7f9040ff9700 1 -- 172.21.15.112:0/1035720252 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034017650 con 0x7f90441023d0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.659+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90441a6590 con 0x7f90441023d0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.662+0000 7f9040ff9700 1 -- 172.21.15.112:0/1035720252 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9034009d10 con 0x7f90441023d0 2024-05-10T10:46:54.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.665+0000 7f9040ff9700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f902c0b2900 0x7f902c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.665+0000 7f9040ff9700 1 -- 172.21.15.112:0/1035720252 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f90340e3070 con 0x7f90441023d0 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.666+0000 7f9042ffd700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f902c0b2900 0x7f902c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 Processor -- start 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 -- start start 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09831f0 0x560cc09835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc097f810 0x560cc097fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09801d0 0x560cc0994480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560cc08d1940 con 0x560cc09831f0 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560cc0836da0 con 0x560cc097f810 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.667+0000 7f905473bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560cc08cd900 con 0x560cc09801d0 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.668+0000 7f9042ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc097f810 0x560cc097fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.668+0000 7f9042ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc097f810 0x560cc097fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55428/0 (socket says 172.21.15.112:55428) 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.668+0000 7f9042ffd700 1 -- 172.21.15.112:0/3260889172 learned_addr learned my addr 172.21.15.112:0/3260889172 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:54.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.669+0000 7f90437fe700 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09831f0 0x560cc09835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.669+0000 7f90437fe700 1 -- 172.21.15.112:0/3260889172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09801d0 msgr2=0x560cc0994480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.669+0000 7f90437fe700 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09801d0 0x560cc0994480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.669+0000 7f90437fe700 1 -- 172.21.15.112:0/3260889172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc097f810 msgr2=0x560cc097fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.669+0000 7f90437fe700 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc097f810 0x560cc097fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.669+0000 7f90437fe700 1 -- 172.21.15.112:0/3260889172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc081e8d0 con 0x560cc09831f0 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.669+0000 7f90437fe700 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09831f0 0x560cc09835d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f90340a75a0 tx=0x7f90340a9aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac080769d62126f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.670+0000 7f9042ffd700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f902c0b2900 0x7f902c0b4dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9038009680 tx=0x7f9038011720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.671+0000 7f9029ffb700 1 -- 172.21.15.112:0/3260889172 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340aa330 con 0x560cc09831f0 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.671+0000 7f9029ffb700 1 -- 172.21.15.112:0/3260889172 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90340ae6e0 con 0x560cc09831f0 2024-05-10T10:46:54.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.672+0000 7f9029ffb700 1 -- 172.21.15.112:0/3260889172 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340aa7f0 con 0x560cc09831f0 2024-05-10T10:46:54.894 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:46:54.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.672+0000 7f905473bb80 1 -- 172.21.15.112:0/3260889172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09831f0 msgr2=0x560cc09835d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.672+0000 7f905473bb80 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09831f0 0x560cc09835d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f90340a75a0 tx=0x7f90340a9aa0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 -- 172.21.15.112:0/3260889172 shutdown_connections 2024-05-10T10:46:54.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09801d0 0x560cc0994480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09831f0 0x560cc09835d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 --2- 172.21.15.112:0/3260889172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc097f810 0x560cc097fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 -- 172.21.15.112:0/3260889172 >> 172.21.15.112:0/3260889172 conn(0x560cc08abb70 msgr2=0x560cc097a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 -- 172.21.15.112:0/3260889172 shutdown_connections 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 -- 172.21.15.112:0/3260889172 wait complete. 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.673+0000 7f905473bb80 1 -- start start 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 1 Processor -- start 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc097f810 0x560cc098dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc098e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc0993a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560cc0983c60 con 0x560cc09801d0 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560cc088aed0 con 0x560cc09831f0 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560cc0983de0 con 0x560cc097f810 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.674+0000 7f905473bb80 10 client.0 _ll_register_callbacks cb 0x560cc08655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc097f810 0x560cc098dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc097f810 0x560cc098dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44486/0 (socket says 172.21.15.112:44486) 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 -- 172.21.15.112:0/299996133 learned_addr learned my addr 172.21.15.112:0/299996133 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f9042ffd700 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc098e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 -- 172.21.15.112:0/299996133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 msgr2=0x560cc0993a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc0993a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 -- 172.21.15.112:0/299996133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 msgr2=0x560cc098e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc098e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 -- 172.21.15.112:0/299996133 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc0888760 con 0x560cc097f810 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f90437fe700 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc097f810 0x560cc098dcc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f903400b2e0 tx=0x7f90340aa870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cee6277d94c3e04 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f9022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.675+0000 7f9022ffd700 1 -- 172.21.15.112:0/299996133 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340c4060 con 0x560cc097f810 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f9022ffd700 1 -- 172.21.15.112:0/299996133 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f90340b2e80 con 0x560cc097f810 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90340ae6e0 con 0x560cc097f810 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f9022ffd700 1 -- 172.21.15.112:0/299996133 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f903406d030 con 0x560cc097f810 2024-05-10T10:46:54.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 10 client.26350 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560cc0983c60 con 0x560cc097f810 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f9022ffd700 1 -- 172.21.15.112:0/299996133 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90340ae6e0 con 0x560cc097f810 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f9022ffd700 1 -- 172.21.15.112:0/299996133 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f90340a5ca0 con 0x560cc097f810 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 10 client.26350 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 10 client.26350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 20 client.26350 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc09a9df0 0x560cc09ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 4 client.26350 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.676+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cc08df320 con 0x560cc09a9df0 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.677+0000 7f9042ffd700 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc09a9df0 0x560cc09ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.677+0000 7f9022ffd700 10 client.26350 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.677+0000 7f9042ffd700 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc09a9df0 0x560cc09ac1e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e54935e4bdcb80d server_cookie=818eee711757101a in_seq=0 out_seq=0 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.678+0000 7f9022ffd700 1 -- 172.21.15.112:0/299996133 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560cc08df320 con 0x560cc09a9df0 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.678+0000 7f9022ffd700 10 client.26350 handle_command_reply: tid=0 2024-05-10T10:46:54.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.678+0000 7f905473bb80 1 client.26350 shutdown 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc097f810 msgr2=0x560cc098dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc097f810 0x560cc098dcc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f903400b2e0 tx=0x7f90340aa870 comp rx=0 tx=0).stop 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 shutdown_connections 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc0993a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc097f810 0x560cc098dcc0 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f903400b2e0 tx=0x7f90340aa870 comp rx=0 tx=0).stop 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc09a9df0 0x560cc09ac1e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 --2- 172.21.15.112:0/299996133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc098e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 >> 172.21.15.112:0/299996133 conn(0x560cc08abb70 msgr2=0x560cc0974b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f9022ffd700 0 client.26350 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 shutdown_connections 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 1 -- 172.21.15.112:0/299996133 wait complete. 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.679+0000 7f905473bb80 20 client.26350 trim_cache size 0 max 16384 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 Processor -- start 2024-05-10T10:46:54.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 -- start start 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc097f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09831f0 0x560cc0974c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09879c0 0x560cc0987e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90441ace80 con 0x560cc09801d0 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90441ae9c0 con 0x560cc09879c0 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f905473bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90441ad550 con 0x560cc09831f0 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc097f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f9042ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09831f0 0x560cc0974c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc097f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51986/0 (socket says 172.21.15.112:51986) 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 -- 172.21.15.112:0/856092656 learned_addr learned my addr 172.21.15.112:0/856092656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f9043fff700 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09879c0 0x560cc0987e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 -- 172.21.15.112:0/856092656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09831f0 msgr2=0x560cc0974c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09831f0 0x560cc0974c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 -- 172.21.15.112:0/856092656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09879c0 msgr2=0x560cc0987e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09879c0 0x560cc0987e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.702+0000 7f90437fe700 1 -- 172.21.15.112:0/856092656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc0993d80 con 0x560cc09801d0 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.703+0000 7f90437fe700 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc097f210 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f90340aa2c0 tx=0x7f90340a9aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d403d4d53f0c08f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.703+0000 7f90297fa700 1 -- 172.21.15.112:0/856092656 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340c4040 con 0x560cc09801d0 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.703+0000 7f90297fa700 1 -- 172.21.15.112:0/856092656 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90340aa330 con 0x560cc09801d0 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.703+0000 7f90297fa700 1 -- 172.21.15.112:0/856092656 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340a5ca0 con 0x560cc09801d0 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 -- 172.21.15.112:0/856092656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 msgr2=0x560cc097f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc097f210 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f90340aa2c0 tx=0x7f90340a9aa0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 -- 172.21.15.112:0/856092656 shutdown_connections 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09879c0 0x560cc0987e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc097f210 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 --2- 172.21.15.112:0/856092656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09831f0 0x560cc0974c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 -- 172.21.15.112:0/856092656 >> 172.21.15.112:0/856092656 conn(0x560cc08abb70 msgr2=0x560cc097a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 -- 172.21.15.112:0/856092656 shutdown_connections 2024-05-10T10:46:54.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 -- 172.21.15.112:0/856092656 wait complete. 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 -- start start 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.704+0000 7f905473bb80 1 Processor -- start 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f905473bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc09bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f905473bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc09b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f905473bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09879c0 0x560cc09b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f905473bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90441ae2f0 con 0x560cc09801d0 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f905473bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90441adc20 con 0x560cc09831f0 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f905473bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90441aae20 con 0x560cc09879c0 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f905473bb80 10 client.0 _ll_register_callbacks cb 0x560cc0994810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f9043fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09879c0 0x560cc09b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f9042ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc09b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f9043fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09879c0 0x560cc09b9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:44496/0 (socket says 172.21.15.112:44496) 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f9043fff700 1 -- 172.21.15.112:0/1519698593 learned_addr learned my addr 172.21.15.112:0/1519698593 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.705+0000 7f90437fe700 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc09bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9043fff700 1 -- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 msgr2=0x560cc09b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9043fff700 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc09b8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9043fff700 1 -- 172.21.15.112:0/1519698593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 msgr2=0x560cc09bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9043fff700 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc09bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9043fff700 1 -- 172.21.15.112:0/1519698593 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc09b96d0 con 0x560cc09879c0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9042ffd700 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc09b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9043fff700 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09879c0 0x560cc09b9090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f903c011f10 tx=0x7f903c014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fda3fc063b85ea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9022ffd700 1 -- 172.21.15.112:0/1519698593 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c014ba0 con 0x560cc09879c0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90340aa330 con 0x560cc09879c0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9022ffd700 1 -- 172.21.15.112:0/1519698593 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f903c014d40 con 0x560cc09879c0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9022ffd700 1 -- 172.21.15.112:0/1519698593 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f903c027050 con 0x560cc09879c0 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f905473bb80 10 client.26356 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:54.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560cc0a14230 con 0x560cc09879c0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.706+0000 7f9022ffd700 1 -- 172.21.15.112:0/1519698593 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90340aa330 con 0x560cc09879c0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f9022ffd700 1 -- 172.21.15.112:0/1519698593 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f903c004550 con 0x560cc09879c0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f905473bb80 10 client.26356 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f905473bb80 10 client.26356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f905473bb80 20 client.26356 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f905473bb80 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc0a148f0 0x560cc0a16d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f905473bb80 4 client.26356 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cc08d93c0 con 0x560cc0a148f0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f90437fe700 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc0a148f0 0x560cc0a16d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f9022ffd700 10 client.26356 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.707+0000 7f90437fe700 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc0a148f0 0x560cc0a16d10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=668611c7bddbeb93 server_cookie=218b02e454aa279c in_seq=0 out_seq=0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f9022ffd700 1 -- 172.21.15.112:0/1519698593 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cc08d93c0 con 0x560cc0a148f0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f9022ffd700 10 client.26356 handle_command_reply: tid=0 2024-05-10T10:46:54.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 client.26356 shutdown 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09879c0 msgr2=0x560cc09b9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09879c0 0x560cc09b9090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f903c011f10 tx=0x7f903c014600 comp rx=0 tx=0).stop 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 shutdown_connections 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560cc09879c0 0x560cc09b9090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x560cc0a148f0 0x560cc0a16d10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cc09801d0 0x560cc09bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f9022ffd700 0 client.26356 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 --2- 172.21.15.112:0/1519698593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560cc09831f0 0x560cc09b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 >> 172.21.15.112:0/1519698593 conn(0x560cc08abb70 msgr2=0x560cc0987e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 shutdown_connections 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 1 -- 172.21.15.112:0/1519698593 wait complete. 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.708+0000 7f905473bb80 20 client.26356 trim_cache size 0 max 16384 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f902c0b2900 msgr2=0x7f902c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f902c0b2900 0x7f902c0b4dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9038009680 tx=0x7f9038011720 comp rx=0 tx=0).stop 2024-05-10T10:46:54.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 msgr2=0x7f904419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 0x7f904419f6c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9034006010 tx=0x7f90340047b0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 shutdown_connections 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f902c0b2900 0x7f902c0b4dc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9044108440 0x7f9044199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90441023d0 0x7f904419f6c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 --2- 172.21.15.112:0/1035720252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9044102d90 0x7f904419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 >> 172.21.15.112:0/1035720252 conn(0x7f90440793e0 msgr2=0x7f904406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 shutdown_connections 2024-05-10T10:46:54.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:54.709+0000 7f9049ea8700 1 -- 172.21.15.112:0/1035720252 wait complete. 2024-05-10T10:46:55.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:55 smithi195 ceph-mon[33773]: pgmap v630: 129 pgs: 129 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 524 op/s 2024-05-10T10:46:55.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:55 smithi112 ceph-mon[30100]: pgmap v630: 129 pgs: 129 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 524 op/s 2024-05-10T10:46:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3928065602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 msgr2=0x7f2e0c10ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3928065602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 0x7f2e0c10ab40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009ca0 tx=0x7f2e0402a8c0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3928065602 shutdown_connections 2024-05-10T10:46:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3928065602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 0x7f2e0c10ab40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3928065602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e0c105e20 0x7f2e0c108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3928065602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 0x7f2e0c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3928065602 >> 172.21.15.112:0/3928065602 conn(0x7f2e0c0fce80 msgr2=0x7f2e0c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3928065602 shutdown_connections 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.487+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3928065602 wait complete. 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 Processor -- start 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 -- start start 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 0x7f2e0c102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e0c105e20 0x7f2e0c1014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 0x7f2e0c1019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10e440 con 0x7f2e0c108750 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10e2c0 con 0x7f2e0c1034f0 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e13a6b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10e5c0 con 0x7f2e0c105e20 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e11807640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 0x7f2e0c102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e11006640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e0c105e20 0x7f2e0c1014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e11807640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 0x7f2e0c102e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:55460/0 (socket says 172.21.15.112:55460) 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e11807640 1 -- 172.21.15.112:0/3201422696 learned_addr learned my addr 172.21.15.112:0/3201422696 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.488+0000 7f2e12008640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 0x7f2e0c1019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e11807640 1 -- 172.21.15.112:0/3201422696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e0c105e20 msgr2=0x7f2e0c1014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e11807640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e0c105e20 0x7f2e0c1014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e11807640 1 -- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 msgr2=0x7f2e0c1019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e11807640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 0x7f2e0c1019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e11807640 1 -- 172.21.15.112:0/3201422696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c102270 con 0x7f2e0c1034f0 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e12008640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 0x7f2e0c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e11006640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e0c105e20 0x7f2e0c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e11807640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 0x7f2e0c102e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800d950 tx=0x7f2e0800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e02ffd640 1 -- 172.21.15.112:0/3201422696 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08017030 con 0x7f2e0c1034f0 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e02ffd640 1 -- 172.21.15.112:0/3201422696 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e08004020 con 0x7f2e0c1034f0 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c1a2b80 con 0x7f2e0c1034f0 2024-05-10T10:46:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e02ffd640 1 -- 172.21.15.112:0/3201422696 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080055d0 con 0x7f2e0c1034f0 2024-05-10T10:46:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.489+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c1a30d0 con 0x7f2e0c1034f0 2024-05-10T10:46:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.492+0000 7f2e02ffd640 1 -- 172.21.15.112:0/3201422696 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2e08009a30 con 0x7f2e0c1034f0 2024-05-10T10:46:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.494+0000 7f2e16788740 1 Processor -- start 2024-05-10T10:46:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e16788740 1 -- start start 2024-05-10T10:46:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e16788740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e16788740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 0x563cb04782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e16788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0478910 0x563cb047ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e16788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cb03d75d0 con 0x563cb0478910 2024-05-10T10:46:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e16788740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563cb0390890 con 0x563cb0472db0 2024-05-10T10:46:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e16788740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563cb03d3590 con 0x563cb04737a0 2024-05-10T10:46:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e12008640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e12008640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56360/0 (socket says 172.21.15.112:56360) 2024-05-10T10:46:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e12008640 1 -- 172.21.15.112:0/23242615 learned_addr learned my addr 172.21.15.112:0/23242615 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e12008640 1 -- 172.21.15.112:0/23242615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 msgr2=0x563cb04782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e12008640 1 --2- 172.21.15.112:0/23242615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 0x563cb04782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e12008640 1 -- 172.21.15.112:0/23242615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0478910 msgr2=0x563cb047ada0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.495+0000 7f2e12008640 1 --2- 172.21.15.112:0/23242615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0478910 0x563cb047ada0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.496+0000 7f2e12008640 1 -- 172.21.15.112:0/23242615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb00e9210 con 0x563cb0472db0 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.496+0000 7f2e12008640 1 --2- 172.21.15.112:0/23242615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04731d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e0402ade0 tx=0x7f2e04009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106eb834e844b0d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.496+0000 7f2e02ffd640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2df80b28a0 0x7f2df80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.496+0000 7f2e11807640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2df80b28a0 0x7f2df80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.497+0000 7f2df7fff640 1 -- 172.21.15.112:0/23242615 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04003970 con 0x563cb0472db0 2024-05-10T10:46:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.497+0000 7f2df7fff640 1 -- 172.21.15.112:0/23242615 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e04036030 con 0x563cb0472db0 2024-05-10T10:46:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.497+0000 7f2e02ffd640 1 -- 172.21.15.112:0/3201422696 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2e080e1fc0 con 0x7f2e0c1034f0 2024-05-10T10:46:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.497+0000 7f2e11807640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2df80b28a0 0x7f2df80b4d60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2e080df780 tx=0x7f2e080a3840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.497+0000 7f2df7fff640 1 -- 172.21.15.112:0/23242615 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e040305c0 con 0x563cb0472db0 2024-05-10T10:46:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.498+0000 7f2e16788740 1 -- 172.21.15.112:0/23242615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 msgr2=0x563cb04731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.498+0000 7f2e16788740 1 --2- 172.21.15.112:0/23242615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04731d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e0402ade0 tx=0x7f2e04009a20 comp rx=0 tx=0).stop 2024-05-10T10:46:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.498+0000 7f2e16788740 1 -- 172.21.15.112:0/23242615 shutdown_connections 2024-05-10T10:46:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.498+0000 7f2e16788740 1 --2- 172.21.15.112:0/23242615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0478910 0x563cb047ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.498+0000 7f2e16788740 1 --2- 172.21.15.112:0/23242615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 0x563cb04782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.498+0000 7f2e16788740 1 --2- 172.21.15.112:0/23242615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04731d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.498+0000 7f2e16788740 1 -- 172.21.15.112:0/23242615 >> 172.21.15.112:0/23242615 conn(0x563cb046e420 msgr2=0x563cb046e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.499+0000 7f2e16788740 1 -- 172.21.15.112:0/23242615 shutdown_connections 2024-05-10T10:46:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.499+0000 7f2e16788740 1 -- 172.21.15.112:0/23242615 wait complete. 2024-05-10T10:46:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.499+0000 7f2e16788740 1 -- start start 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.499+0000 7f2e16788740 1 Processor -- start 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e16788740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0472db0 0x563cb047cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e16788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb047d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e16788740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb0484420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e16788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cb047b6e0 con 0x563cb04737a0 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e16788740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563cb0394370 con 0x563cb0478910 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e16788740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563cb047b860 con 0x563cb0472db0 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e16788740 10 client.0 _ll_register_callbacks cb 0x563cb00e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e12008640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0472db0 0x563cb047cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e11006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb047d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e12008640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0472db0 0x563cb047cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60578/0 (socket says 172.21.15.112:60578) 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e12008640 1 -- 172.21.15.112:0/3903488560 learned_addr learned my addr 172.21.15.112:0/3903488560 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.500+0000 7f2e12008640 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb0484420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e11006640 1 -- 172.21.15.112:0/3903488560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0472db0 msgr2=0x563cb047cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e11006640 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0472db0 0x563cb047cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e11006640 1 -- 172.21.15.112:0/3903488560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 msgr2=0x563cb0484420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e11006640 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb0484420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e11006640 1 -- 172.21.15.112:0/3903488560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb01b5620 con 0x563cb04737a0 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e12008640 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb0484420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e11006640 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb047d3e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x563cb046f2f0 tx=0x7f2dfc0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae97499c62691d04 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2df57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2df57fa640 1 -- 172.21.15.112:0/3903488560 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc005220 con 0x563cb04737a0 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e04036030 con 0x563cb04737a0 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2df57fa640 1 -- 172.21.15.112:0/3903488560 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2dfc0053c0 con 0x563cb04737a0 2024-05-10T10:46:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2df57fa640 1 -- 172.21.15.112:0/3903488560 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc024050 con 0x563cb04737a0 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e16788740 10 client.16992 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.501+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563cb0487520 con 0x563cb04737a0 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2df57fa640 1 -- 172.21.15.112:0/3903488560 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e04036030 con 0x563cb04737a0 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2df57fa640 1 -- 172.21.15.112:0/3903488560 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f2dfc011910 con 0x563cb04737a0 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e16788740 10 client.16992 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e16788740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e16788740 20 client.16992 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e16788740 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb048b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e16788740 4 client.16992 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cb03e51e0 con 0x563cb0489080 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e12008640 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb048b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2e12008640 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb048b4a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be57f1b1861c823e server_cookie=1d8534c0b28062b9 in_seq=0 out_seq=0 2024-05-10T10:46:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.502+0000 7f2df57fa640 10 client.16992 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.503+0000 7f2df57fa640 1 -- 172.21.15.112:0/3903488560 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563cb03e51e0 con 0x563cb0489080 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.503+0000 7f2df57fa640 10 client.16992 handle_command_reply: tid=0 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.503+0000 7f2e16788740 1 client.16992 shutdown 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.503+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 msgr2=0x563cb047d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.503+0000 7f2e16788740 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb047d3e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x563cb046f2f0 tx=0x7f2dfc0108b0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.504+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 shutdown_connections 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.504+0000 7f2e16788740 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb048b4a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.504+0000 7f2e16788740 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb0484420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.504+0000 7f2df57fa640 0 client.16992 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.504+0000 7f2e16788740 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb047d3e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.504+0000 7f2e16788740 1 --2- 172.21.15.112:0/3903488560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0472db0 0x563cb047cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.504+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 >> 172.21.15.112:0/3903488560 conn(0x563cb046e420 msgr2=0x563cb046d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.505+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 shutdown_connections 2024-05-10T10:46:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.505+0000 7f2e16788740 1 -- 172.21.15.112:0/3903488560 wait complete. 2024-05-10T10:46:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.505+0000 7f2e16788740 20 client.16992 trim_cache size 0 max 16384 2024-05-10T10:46:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 Processor -- start 2024-05-10T10:46:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 -- start start 2024-05-10T10:46:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0472db0 0x563cb04693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 0x563cb047cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb047d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c1ad5a0 con 0x563cb0472db0 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c1af0e0 con 0x563cb0478910 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e16788740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c1adc70 con 0x563cb04737a0 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0472db0 0x563cb04693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0472db0 0x563cb04693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40444/0 (socket says 172.21.15.112:40444) 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 -- 172.21.15.112:0/2091093176 learned_addr learned my addr 172.21.15.112:0/2091093176 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e11006640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 0x563cb047cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 -- 172.21.15.112:0/2091093176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 msgr2=0x563cb047cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 --2- 172.21.15.112:0/2091093176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 0x563cb047cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 -- 172.21.15.112:0/2091093176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 msgr2=0x563cb047d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 --2- 172.21.15.112:0/2091093176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb047d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.524+0000 7f2e12008640 1 -- 172.21.15.112:0/2091093176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb046e250 con 0x563cb0472db0 2024-05-10T10:46:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.525+0000 7f2e12008640 1 --2- 172.21.15.112:0/2091093176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0472db0 0x563cb04693e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2e04005fe0 tx=0x7f2e04009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8677d31e2fadb625 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.525+0000 7f2df77fe640 1 -- 172.21.15.112:0/2091093176 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04009540 con 0x563cb0472db0 2024-05-10T10:46:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.525+0000 7f2df77fe640 1 -- 172.21.15.112:0/2091093176 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e04036070 con 0x563cb0472db0 2024-05-10T10:46:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.525+0000 7f2df77fe640 1 -- 172.21.15.112:0/2091093176 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04003e40 con 0x563cb0472db0 2024-05-10T10:46:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 -- 172.21.15.112:0/2091093176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0472db0 msgr2=0x563cb04693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 --2- 172.21.15.112:0/2091093176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0472db0 0x563cb04693e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2e04005fe0 tx=0x7f2e04009920 comp rx=0 tx=0).stop 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 -- 172.21.15.112:0/2091093176 shutdown_connections 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 --2- 172.21.15.112:0/2091093176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0478910 0x563cb047d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 --2- 172.21.15.112:0/2091093176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb04737a0 0x563cb047cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 --2- 172.21.15.112:0/2091093176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb0472db0 0x563cb04693e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 -- 172.21.15.112:0/2091093176 >> 172.21.15.112:0/2091093176 conn(0x563cb046e420 msgr2=0x563cb046e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 -- 172.21.15.112:0/2091093176 shutdown_connections 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 -- 172.21.15.112:0/2091093176 wait complete. 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 -- start start 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.526+0000 7f2e16788740 1 Processor -- start 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb0480f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0478910 0x563cb04f1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c1aea10 con 0x563cb04737a0 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c1ae340 con 0x563cb0472db0 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c1ab580 con 0x563cb0478910 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 10 client.0 _ll_register_callbacks cb 0x7f2dfc005c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb0480f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb0480f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40452/0 (socket says 172.21.15.112:40452) 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 -- 172.21.15.112:0/3082201905 learned_addr learned my addr 172.21.15.112:0/3082201905 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e12008640 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e12008640 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0478910 0x563cb04f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 -- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0478910 msgr2=0x563cb04f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0478910 0x563cb04f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 -- 172.21.15.112:0/3082201905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 msgr2=0x563cb04809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04809f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 -- 172.21.15.112:0/3082201905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb04729f0 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e11006640 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb0480f30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc011d50 tx=0x7f2dfc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f95a52015b446 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2df57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2df57fa640 1 -- 172.21.15.112:0/3082201905 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc021040 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2df57fa640 1 -- 172.21.15.112:0/3082201905 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2dfc005040 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.527+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e04036070 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2df57fa640 1 -- 172.21.15.112:0/3082201905 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc005240 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 10 client.16998 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563cb04f2f80 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2df57fa640 1 -- 172.21.15.112:0/3082201905 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e04036070 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2df57fa640 1 -- 172.21.15.112:0/3082201905 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f2dfc030070 con 0x563cb04737a0 2024-05-10T10:46:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 10 client.16998 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 20 client.16998 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb04f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 4 client.16998 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563cb03df050 con 0x563cb0489080 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.528+0000 7f2e12008640 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb04f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.529+0000 7f2df57fa640 10 client.16998 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.529+0000 7f2e12008640 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb04f5990 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11a1e2ceefdbd093 server_cookie=e1fc7dde0768b61d in_seq=0 out_seq=0 2024-05-10T10:46:55.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:55 smithi148 ceph-mon[33568]: pgmap v630: 129 pgs: 129 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 524 op/s 2024-05-10T10:46:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2df57fa640 1 -- 172.21.15.112:0/3082201905 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563cb03df050 con 0x563cb0489080 2024-05-10T10:46:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2df57fa640 10 client.16998 handle_command_reply: tid=0 2024-05-10T10:46:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 client.16998 shutdown 2024-05-10T10:46:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 msgr2=0x563cb0480f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb0480f30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc011d50 tx=0x7f2dfc00bd60 comp rx=0 tx=0).stop 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 shutdown_connections 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb0478910 0x563cb04f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb04737a0 0x563cb0480f30 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563cb0489080 0x563cb04f5990 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 --2- 172.21.15.112:0/3082201905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb0472db0 0x563cb04809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2df57fa640 0 client.16998 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.547+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 >> 172.21.15.112:0/3082201905 conn(0x563cb046e420 msgr2=0x563cb04693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 shutdown_connections 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e16788740 1 -- 172.21.15.112:0/3082201905 wait complete. 2024-05-10T10:46:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e16788740 20 client.16998 trim_cache size 0 max 16384 2024-05-10T10:46:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:46:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:46:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:46:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2df80b28a0 msgr2=0x7f2df80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2df80b28a0 0x7f2df80b4d60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2e080df780 tx=0x7f2e080a3840 comp rx=0 tx=0).stop 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 msgr2=0x7f2e0c102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 0x7f2e0c102e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800d950 tx=0x7f2e0800de10 comp rx=0 tx=0).stop 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 shutdown_connections 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f2df80b28a0 0x7f2df80b4d60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0c108750 0x7f2e0c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e0c105e20 0x7f2e0c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 --2- 172.21.15.112:0/3201422696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0c1034f0 0x7f2e0c102e60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.548+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 >> 172.21.15.112:0/3201422696 conn(0x7f2e0c0fce80 msgr2=0x7f2e0c103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.549+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 shutdown_connections 2024-05-10T10:46:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:46:55.549+0000 7f2e13a6b640 1 -- 172.21.15.112:0/3201422696 wait complete. 2024-05-10T10:46:55.945 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:46:56.269 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:46:57.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:57 smithi195 ceph-mon[33773]: pgmap v631: 129 pgs: 129 active+clean; 56 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.0 MiB/s wr, 692 op/s 2024-05-10T10:46:57.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:57 smithi112 ceph-mon[30100]: pgmap v631: 129 pgs: 129 active+clean; 56 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.0 MiB/s wr, 692 op/s 2024-05-10T10:46:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:57 smithi148 ceph-mon[33568]: pgmap v631: 129 pgs: 129 active+clean; 56 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.0 MiB/s wr, 692 op/s 2024-05-10T10:46:59.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:46:59 smithi195 ceph-mon[33773]: pgmap v632: 129 pgs: 129 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 525 op/s 2024-05-10T10:46:59.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:46:59 smithi112 ceph-mon[30100]: pgmap v632: 129 pgs: 129 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 525 op/s 2024-05-10T10:46:59.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:46:59 smithi148 ceph-mon[33568]: pgmap v632: 129 pgs: 129 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 525 op/s 2024-05-10T10:46:59.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 -- 172.21.15.112:0/3703936673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 msgr2=0x7f91bc109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/3703936673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc109900 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009990 tx=0x7f91b8009ca0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 -- 172.21.15.112:0/3703936673 shutdown_connections 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/3703936673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91bc102450 0x7f91bc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/3703936673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc109900 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/3703936673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91bc101a90 0x7f91bc101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 -- 172.21.15.112:0/3703936673 >> 172.21.15.112:0/3703936673 conn(0x7f91bc078a10 msgr2=0x7f91bc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 -- 172.21.15.112:0/3703936673 shutdown_connections 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.337+0000 7f91c3b4e700 1 -- 172.21.15.112:0/3703936673 wait complete. 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 Processor -- start 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 -- start start 2024-05-10T10:46:59.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91bc101a90 0x7f91bc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91bc102450 0x7f91bc1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc113500 con 0x7f91bc109520 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc113380 con 0x7f91bc101a90 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c3b4e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc113680 con 0x7f91bc102450 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40476/0 (socket says 172.21.15.112:40476) 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 -- 172.21.15.112:0/4216351317 learned_addr learned my addr 172.21.15.112:0/4216351317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c18ea700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91bc101a90 0x7f91bc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c10e9700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91bc102450 0x7f91bc1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 -- 172.21.15.112:0/4216351317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91bc102450 msgr2=0x7f91bc1a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91bc102450 0x7f91bc1a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 -- 172.21.15.112:0/4216351317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91bc101a90 msgr2=0x7f91bc1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91bc101a90 0x7f91bc1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 -- 172.21.15.112:0/4216351317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc19c1b0 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.338+0000 7f91c20eb700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc19ba50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f91b4007ee0 tx=0x7f91b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.339+0000 7f91b2ffd700 1 -- 172.21.15.112:0/4216351317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4004bb0 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.339+0000 7f91b2ffd700 1 -- 172.21.15.112:0/4216351317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b4004d50 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.339+0000 7f91b2ffd700 1 -- 172.21.15.112:0/4216351317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40058b0 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.339+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc19c4a0 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.339+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc1a8760 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.341+0000 7f91b2ffd700 1 -- 172.21.15.112:0/4216351317 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f91b401a020 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.342+0000 7f91b2ffd700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91a80b2880 0x7f91a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.342+0000 7f91c18ea700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91a80b2880 0x7f91a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.342+0000 7f91b2ffd700 1 -- 172.21.15.112:0/4216351317 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91b40e1c00 con 0x7f91bc109520 2024-05-10T10:46:59.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.342+0000 7f91c18ea700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91a80b2880 0x7f91a80b4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91b800b5c0 tx=0x7f91b801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.342+0000 7f91ce3e1b80 1 Processor -- start 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.342+0000 7f91ce3e1b80 1 -- start start 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d285590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d285ad0 0x55f93d27f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d27fd50 0x55f93d280190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f93d1d1940 con 0x55f93d27fd50 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f93d136da0 con 0x55f93d285ad0 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f93d1cd900 con 0x55f93d2851b0 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d27fd50 0x55f93d280190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c18ea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d285590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d27fd50 0x55f93d280190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40480/0 (socket says 172.21.15.112:40480) 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 -- 172.21.15.112:0/1828876327 learned_addr learned my addr 172.21.15.112:0/1828876327 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c10e9700 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d285ad0 0x55f93d27f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 -- 172.21.15.112:0/1828876327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 msgr2=0x55f93d285590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d285590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 -- 172.21.15.112:0/1828876327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d285ad0 msgr2=0x55f93d27f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d285ad0 0x55f93d27f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 -- 172.21.15.112:0/1828876327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93d11e8d0 con 0x55f93d27fd50 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91c20eb700 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d27fd50 0x55f93d280190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f91b40a6100 tx=0x7f91b40a8d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e2f1722b2a8bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91a7fff700 1 -- 172.21.15.112:0/1828876327 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40a9590 con 0x55f93d27fd50 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.343+0000 7f91a7fff700 1 -- 172.21.15.112:0/1828876327 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b40a8e90 con 0x55f93d27fd50 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91a7fff700 1 -- 172.21.15.112:0/1828876327 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40a90f0 con 0x55f93d27fd50 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1828876327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d27fd50 msgr2=0x55f93d280190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d27fd50 0x55f93d280190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f91b40a6100 tx=0x7f91b40a8d80 comp rx=0 tx=0).stop 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1828876327 shutdown_connections 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d27fd50 0x55f93d280190 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d285590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1828876327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d285ad0 0x55f93d27f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1828876327 >> 172.21.15.112:0/1828876327 conn(0x55f93d1abb70 msgr2=0x55f93d27a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1828876327 shutdown_connections 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.344+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1828876327 wait complete. 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 -- start start 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 Processor -- start 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d27fd50 0x55f93d28b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d2851b0 0x55f93d288b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d289050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f93d28c4a0 con 0x55f93d285ad0 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f93d18aed0 con 0x55f93d2851b0 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f93d28c620 con 0x55f93d27fd50 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91ce3e1b80 10 client.0 _ll_register_callbacks cb 0x55f93d1655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91c20eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d289050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91c20eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d289050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40496/0 (socket says 172.21.15.112:40496) 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91c20eb700 1 -- 172.21.15.112:0/1683781836 learned_addr learned my addr 172.21.15.112:0/1683781836 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.345+0000 7f91c10e9700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d2851b0 0x55f93d288b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c18ea700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d27fd50 0x55f93d28b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c20eb700 1 -- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d27fd50 msgr2=0x55f93d28b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c20eb700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d27fd50 0x55f93d28b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c20eb700 1 -- 172.21.15.112:0/1683781836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d2851b0 msgr2=0x55f93d288b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c20eb700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d2851b0 0x55f93d288b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c20eb700 1 -- 172.21.15.112:0/1683781836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f93d188760 con 0x55f93d285ad0 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c10e9700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d2851b0 0x55f93d288b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91c20eb700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d289050 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f91b40df3c0 tx=0x7f91b40b1e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da12befca1345e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91a57fa700 1 -- 172.21.15.112:0/1683781836 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40a90f0 con 0x55f93d285ad0 2024-05-10T10:46:59.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91a57fa700 1 -- 172.21.15.112:0/1683781836 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91b40a9290 con 0x55f93d285ad0 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b40a8e90 con 0x55f93d285ad0 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91a57fa700 1 -- 172.21.15.112:0/1683781836 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f91b40b1860 con 0x55f93d285ad0 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91ce3e1b80 10 client.17010 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f93d296d80 con 0x55f93d285ad0 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91a57fa700 1 -- 172.21.15.112:0/1683781836 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b40a8e90 con 0x55f93d285ad0 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.346+0000 7f91a57fa700 1 -- 172.21.15.112:0/1683781836 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f91b40a7b60 con 0x55f93d285ad0 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91ce3e1b80 10 client.17010 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91ce3e1b80 10 client.17010 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91ce3e1b80 20 client.17010 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:59.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d299250 0x55f93d29b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91ce3e1b80 4 client.17010 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f93d1df320 con 0x55f93d299250 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91c18ea700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d299250 0x55f93d29b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91a57fa700 10 client.17010 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.347+0000 7f91c18ea700 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d299250 0x55f93d29b640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6459bc3c6d44cc85 server_cookie=808eb40eaf65c71a in_seq=0 out_seq=0 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91a57fa700 1 -- 172.21.15.112:0/1683781836 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f93d1df320 con 0x55f93d299250 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91a57fa700 10 client.17010 handle_command_reply: tid=0 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 client.17010 shutdown 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 msgr2=0x55f93d289050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d289050 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f91b40df3c0 tx=0x7f91b40b1e30 comp rx=0 tx=0).stop 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 shutdown_connections 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d299250 0x55f93d29b640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d289050 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d27fd50 0x55f93d28b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1683781836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d2851b0 0x55f93d288b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91a57fa700 0 client.17010 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 >> 172.21.15.112:0/1683781836 conn(0x55f93d1abb70 msgr2=0x55f93d274b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 shutdown_connections 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1683781836 wait complete. 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.348+0000 7f91ce3e1b80 20 client.17010 trim_cache size 0 max 16384 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.364+0000 7f91ce3e1b80 1 Processor -- start 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.364+0000 7f91ce3e1b80 1 -- start start 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d286770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d2851b0 0x55f93d289080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d285ad0 0x55f93d295530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1b1330 con 0x55f93d2851b0 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1b2e70 con 0x55f93d27fd50 2024-05-10T10:46:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1b1a00 con 0x55f93d285ad0 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c10e9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d2851b0 0x55f93d289080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c10e9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d2851b0 0x55f93d289080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40504/0 (socket says 172.21.15.112:40504) 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c10e9700 1 -- 172.21.15.112:0/302518986 learned_addr learned my addr 172.21.15.112:0/302518986 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c18ea700 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d286770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c20eb700 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d285ad0 0x55f93d295530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c18ea700 1 -- 172.21.15.112:0/302518986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d285ad0 msgr2=0x55f93d295530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c18ea700 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d285ad0 0x55f93d295530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c18ea700 1 -- 172.21.15.112:0/302518986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d2851b0 msgr2=0x55f93d289080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c18ea700 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d2851b0 0x55f93d289080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c18ea700 1 -- 172.21.15.112:0/302518986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93d28c610 con 0x55f93d27fd50 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c20eb700 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d285ad0 0x55f93d295530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.365+0000 7f91c18ea700 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d286770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91b80059d0 tx=0x7f91b801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d84ab51ec3f538 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.366+0000 7f91a77fe700 1 -- 172.21.15.112:0/302518986 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8003da0 con 0x55f93d27fd50 2024-05-10T10:46:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.366+0000 7f91a77fe700 1 -- 172.21.15.112:0/302518986 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b8004690 con 0x55f93d27fd50 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.366+0000 7f91a77fe700 1 -- 172.21.15.112:0/302518986 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b800fa60 con 0x55f93d27fd50 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/302518986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 msgr2=0x55f93d286770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d286770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91b80059d0 tx=0x7f91b801f000 comp rx=0 tx=0).stop 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/302518986 shutdown_connections 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d285ad0 0x55f93d295530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d286770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/302518986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d2851b0 0x55f93d289080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/302518986 >> 172.21.15.112:0/302518986 conn(0x55f93d1abb70 msgr2=0x55f93d283c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/302518986 shutdown_connections 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/302518986 wait complete. 2024-05-10T10:46:59.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 -- start start 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.367+0000 7f91ce3e1b80 1 Processor -- start 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d291d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d2a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91ce3e1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d2a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1b27a0 con 0x55f93d285ad0 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1b20d0 con 0x55f93d27fd50 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91ce3e1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1af2d0 con 0x55f93d2851b0 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91ce3e1b80 10 client.0 _ll_register_callbacks cb 0x55f93d1655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d291d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d291d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56434/0 (socket says 172.21.15.112:56434) 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 -- 172.21.15.112:0/1234773479 learned_addr learned my addr 172.21.15.112:0/1234773479 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c20eb700 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d2a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c10e9700 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d2a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 -- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 msgr2=0x55f93d2a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d2a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 -- 172.21.15.112:0/1234773479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 msgr2=0x55f93d2a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d2a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.368+0000 7f91c18ea700 1 -- 172.21.15.112:0/1234773479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f93d2857e0 con 0x55f93d27fd50 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91c18ea700 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d291d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f91b80043d0 tx=0x7f91b800f4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94fd205298ef57d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91a57fa700 1 -- 172.21.15.112:0/1234773479 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b800b7d0 con 0x55f93d27fd50 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f93d1d93c0 con 0x55f93d27fd50 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91a57fa700 1 -- 172.21.15.112:0/1234773479 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91b800b970 con 0x55f93d27fd50 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91a57fa700 1 -- 172.21.15.112:0/1234773479 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f91b800bb70 con 0x55f93d27fd50 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 10 client.26372 fetch_fsmap learned FSMap version 42 2024-05-10T10:46:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f93d303510 con 0x55f93d27fd50 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91a57fa700 1 -- 172.21.15.112:0/1234773479 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f93d1d93c0 con 0x55f93d27fd50 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91a57fa700 1 -- 172.21.15.112:0/1234773479 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f91b8032440 con 0x55f93d27fd50 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 10 client.26372 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 10 client.26372 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 20 client.26372 populate_metadata read hostname 'smithi112' 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d303ac0 0x55f93d305ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 4 client.26372 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.369+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f93d051250 con 0x55f93d303ac0 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.370+0000 7f91c10e9700 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d303ac0 0x55f93d305ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.370+0000 7f91c10e9700 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d303ac0 0x55f93d305ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=800f4e2cd96fad5 server_cookie=73f43bebff44d567 in_seq=0 out_seq=0 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.370+0000 7f91a57fa700 10 client.26372 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91a57fa700 1 -- 172.21.15.112:0/1234773479 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f93d051250 con 0x55f93d303ac0 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91a57fa700 10 client.26372 handle_command_reply: tid=0 2024-05-10T10:46:59.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 client.26372 shutdown 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 msgr2=0x55f93d291d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d291d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f91b80043d0 tx=0x7f91b800f4f0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 shutdown_connections 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x55f93d303ac0 0x55f93d305ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f93d285ad0 0x55f93d2a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f93d27fd50 0x55f93d291d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 --2- 172.21.15.112:0/1234773479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f93d2851b0 0x55f93d2a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91a57fa700 0 client.26372 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:46:59.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 >> 172.21.15.112:0/1234773479 conn(0x55f93d1abb70 msgr2=0x55f93d284500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 shutdown_connections 2024-05-10T10:46:59.963 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:46:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 1 -- 172.21.15.112:0/1234773479 wait complete. 2024-05-10T10:46:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.371+0000 7f91ce3e1b80 20 client.26372 trim_cache size 0 max 16384 2024-05-10T10:46:59.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91a80b2880 msgr2=0x7f91a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91a80b2880 0x7f91a80b4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91b800b5c0 tx=0x7f91b801a040 comp rx=0 tx=0).stop 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 msgr2=0x7f91bc19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc19ba50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f91b4007ee0 tx=0x7f91b400d3b0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 shutdown_connections 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91bc109520 0x7f91bc19ba50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91bc101a90 0x7f91bc1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91a80b2880 0x7f91a80b4d40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 --2- 172.21.15.112:0/4216351317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91bc102450 0x7f91bc1a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:46:59.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 >> 172.21.15.112:0/4216351317 conn(0x7f91bc078a10 msgr2=0x7f91bc0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:46:59.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 shutdown_connections 2024-05-10T10:46:59.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:46:59.372+0000 7f91c3b4e700 1 -- 172.21.15.112:0/4216351317 wait complete. 2024-05-10T10:47:00.996 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:47:01.315 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:47:01.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:01 smithi195 ceph-mon[33773]: pgmap v633: 129 pgs: 129 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 567 op/s 2024-05-10T10:47:01.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:01 smithi112 ceph-mon[30100]: pgmap v633: 129 pgs: 129 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 567 op/s 2024-05-10T10:47:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:01 smithi148 ceph-mon[33568]: pgmap v633: 129 pgs: 129 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 567 op/s 2024-05-10T10:47:03.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:03 smithi195 ceph-mon[33773]: pgmap v634: 129 pgs: 129 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.8 MiB/s wr, 693 op/s 2024-05-10T10:47:03.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:03 smithi112 ceph-mon[30100]: pgmap v634: 129 pgs: 129 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.8 MiB/s wr, 693 op/s 2024-05-10T10:47:03.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:03 smithi148 ceph-mon[33568]: pgmap v634: 129 pgs: 129 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.8 MiB/s wr, 693 op/s 2024-05-10T10:47:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.733+0000 7f3568616700 1 -- 172.21.15.112:0/3760028041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3560108410 msgr2=0x7f35601087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.733+0000 7f3568616700 1 --2- 172.21.15.112:0/3760028041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3560108410 0x7f35601087f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f355c009990 tx=0x7f355c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.733+0000 7f3568616700 1 -- 172.21.15.112:0/3760028041 shutdown_connections 2024-05-10T10:47:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.733+0000 7f3568616700 1 --2- 172.21.15.112:0/3760028041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3560102d60 0x7f356010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.733+0000 7f3568616700 1 --2- 172.21.15.112:0/3760028041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3560108410 0x7f35601087f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.733+0000 7f3568616700 1 --2- 172.21.15.112:0/3760028041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35601023a0 0x7f3560102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.733+0000 7f3568616700 1 -- 172.21.15.112:0/3760028041 >> 172.21.15.112:0/3760028041 conn(0x7f3560079350 msgr2=0x7f3560079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.734+0000 7f3568616700 1 -- 172.21.15.112:0/3760028041 shutdown_connections 2024-05-10T10:47:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.734+0000 7f3568616700 1 -- 172.21.15.112:0/3760028041 wait complete. 2024-05-10T10:47:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.735+0000 7f3568616700 1 Processor -- start 2024-05-10T10:47:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3568616700 1 -- start start 2024-05-10T10:47:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3568616700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35601023a0 0x7f356019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3568616700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3560102d60 0x7f3560197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3568616700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3560108410 0x7f3560197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3568616700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35601113c0 con 0x7f35601023a0 2024-05-10T10:47:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3568616700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3560111240 con 0x7f3560108410 2024-05-10T10:47:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3568616700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3560111540 con 0x7f3560102d60 2024-05-10T10:47:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3565bb1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3560102d60 0x7f3560197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3565bb1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3560102d60 0x7f3560197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:60644/0 (socket says 172.21.15.112:60644) 2024-05-10T10:47:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3565bb1700 1 -- 172.21.15.112:0/199559969 learned_addr learned my addr 172.21.15.112:0/199559969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.736+0000 7f3566bb3700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3560108410 0x7f3560197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.737+0000 7f3566bb3700 1 -- 172.21.15.112:0/199559969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3560102d60 msgr2=0x7f3560197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.737+0000 7f3566bb3700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3560102d60 0x7f3560197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.737+0000 7f3566bb3700 1 -- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35601023a0 msgr2=0x7f356019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.737+0000 7f3566bb3700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35601023a0 0x7f356019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.737+0000 7f3566bb3700 1 -- 172.21.15.112:0/199559969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35601982d0 con 0x7f3560108410 2024-05-10T10:47:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.737+0000 7f3566bb3700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3560108410 0x7f3560197c60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f355800efc0 tx=0x7f355800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.738+0000 7f35577fe700 1 -- 172.21.15.112:0/199559969 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580092e0 con 0x7f3560108410 2024-05-10T10:47:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.738+0000 7f35577fe700 1 -- 172.21.15.112:0/199559969 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3558004bb0 con 0x7f3560108410 2024-05-10T10:47:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.738+0000 7f35577fe700 1 -- 172.21.15.112:0/199559969 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558004020 con 0x7f3560108410 2024-05-10T10:47:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.739+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3560198560 con 0x7f3560108410 2024-05-10T10:47:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.740+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356019c4b0 con 0x7f3560108410 2024-05-10T10:47:04.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.745+0000 7f3572ea9b80 1 Processor -- start 2024-05-10T10:47:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.745+0000 7f3572ea9b80 1 -- start start 2024-05-10T10:47:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.745+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 0x5632ce44e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.745+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce4526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce453100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632ce3a0860 con 0x5632ce44edf0 2024-05-10T10:47:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5632ce33f7c0 con 0x5632ce452c80 2024-05-10T10:47:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5632ce39c820 con 0x5632ce44e4d0 2024-05-10T10:47:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce4526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce4526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40544/0 (socket says 172.21.15.112:40544) 2024-05-10T10:47:04.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 -- 172.21.15.112:0/300887448 learned_addr learned my addr 172.21.15.112:0/300887448 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 -- 172.21.15.112:0/300887448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 msgr2=0x5632ce44e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 --2- 172.21.15.112:0/300887448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 0x5632ce44e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 -- 172.21.15.112:0/300887448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 msgr2=0x5632ce453100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 --2- 172.21.15.112:0/300887448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce453100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 -- 172.21.15.112:0/300887448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ce2ed8d0 con 0x5632ce44edf0 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.746+0000 7f35663b2700 1 --2- 172.21.15.112:0/300887448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce4526f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f355c005910 tx=0x7f355c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb528490dfb35240 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.747+0000 7f3554ff9700 1 -- 172.21.15.112:0/300887448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c004410 con 0x5632ce44edf0 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.747+0000 7f35577fe700 1 -- 172.21.15.112:0/199559969 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3558012050 con 0x7f3560108410 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.747+0000 7f3554ff9700 1 -- 172.21.15.112:0/300887448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f355c0045b0 con 0x5632ce44edf0 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.747+0000 7f3554ff9700 1 -- 172.21.15.112:0/300887448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c00bb40 con 0x5632ce44edf0 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.748+0000 7f35577fe700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f354c0b2930 0x7f354c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.748+0000 7f35577fe700 1 -- 172.21.15.112:0/199559969 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f35580e17b0 con 0x7f3560108410 2024-05-10T10:47:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.748+0000 7f3572ea9b80 1 -- 172.21.15.112:0/300887448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 msgr2=0x5632ce4526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.748+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/300887448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce4526f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f355c005910 tx=0x7f355c0096a0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3565bb1700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f354c0b2930 0x7f354c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3572ea9b80 1 -- 172.21.15.112:0/300887448 shutdown_connections 2024-05-10T10:47:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/300887448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce453100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/300887448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 0x5632ce44e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/300887448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce4526f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3572ea9b80 1 -- 172.21.15.112:0/300887448 >> 172.21.15.112:0/300887448 conn(0x5632ce37a100 msgr2=0x5632ce448c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3565bb1700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f354c0b2930 0x7f354c0b4df0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5632ce44a710 tx=0x7f3550009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3572ea9b80 1 -- 172.21.15.112:0/300887448 shutdown_connections 2024-05-10T10:47:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.749+0000 7f3572ea9b80 1 -- 172.21.15.112:0/300887448 wait complete. 2024-05-10T10:47:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.750+0000 7f3572ea9b80 1 -- start start 2024-05-10T10:47:04.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.750+0000 7f3572ea9b80 1 Processor -- start 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.751+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44e4d0 0x5632ce45a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.751+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44edf0 0x5632ce457470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.751+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce4579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.751+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632ce45ae30 con 0x5632ce44e4d0 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.751+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5632ce306c30 con 0x5632ce452c80 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.751+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5632ce45afb0 con 0x5632ce44edf0 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.751+0000 7f3572ea9b80 10 client.0 _ll_register_callbacks cb 0x5632ce338920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:04.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce4579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce4579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:56462/0 (socket says 172.21.15.112:56462) 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 -- 172.21.15.112:0/1243974287 learned_addr learned my addr 172.21.15.112:0/1243974287 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f35663b2700 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44e4d0 0x5632ce45a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 -- 172.21.15.112:0/1243974287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44edf0 msgr2=0x5632ce457470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44edf0 0x5632ce457470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 -- 172.21.15.112:0/1243974287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44e4d0 msgr2=0x5632ce45a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44e4d0 0x5632ce45a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 -- 172.21.15.112:0/1243974287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ce33c690 con 0x5632ce452c80 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3566bb3700 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce4579b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35580a5cb0 tx=0x7f35580a8f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e814c675bb03dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.752+0000 7f3549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:47:04.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3549ffb700 1 -- 172.21.15.112:0/1243974287 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580a9140 con 0x5632ce452c80 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3549ffb700 1 -- 172.21.15.112:0/1243974287 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f35580cb0d0 con 0x5632ce452c80 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f355c0045b0 con 0x5632ce452c80 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3549ffb700 1 -- 172.21.15.112:0/1243974287 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f35580a8a40 con 0x5632ce452c80 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3572ea9b80 10 client.26381 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5632ce4654a0 con 0x5632ce452c80 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3549ffb700 1 -- 172.21.15.112:0/1243974287 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35580acdf0 con 0x5632ce452c80 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.753+0000 7f3549ffb700 1 -- 172.21.15.112:0/1243974287 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f35580a45b0 con 0x5632ce452c80 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f3572ea9b80 10 client.26381 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f3572ea9b80 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f3572ea9b80 20 client.26381 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce467a30 0x5632ce469e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f3572ea9b80 4 client.26381 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:04.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f355c0045b0 con 0x5632ce467a30 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f35663b2700 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce467a30 0x5632ce469e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f35663b2700 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce467a30 0x5632ce469e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8fc6a24834b1d2c server_cookie=2bfdb6688baee421 in_seq=0 out_seq=0 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.754+0000 7f3549ffb700 10 client.26381 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.755+0000 7f3549ffb700 1 -- 172.21.15.112:0/1243974287 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f355c0045b0 con 0x5632ce467a30 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.755+0000 7f3549ffb700 10 client.26381 handle_command_reply: tid=0 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.755+0000 7f3572ea9b80 1 client.26381 shutdown 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.755+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 msgr2=0x5632ce4579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.755+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce4579b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35580a5cb0 tx=0x7f35580a8f80 comp rx=0 tx=0).stop 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 shutdown_connections 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce467a30 0x5632ce469e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3549ffb700 0 client.26381 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:04.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce452c80 0x5632ce4579b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44e4d0 0x5632ce45a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1243974287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44edf0 0x5632ce457470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 >> 172.21.15.112:0/1243974287 conn(0x5632ce37a100 msgr2=0x5632ce443720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 shutdown_connections 2024-05-10T10:47:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1243974287 wait complete. 2024-05-10T10:47:04.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.756+0000 7f3572ea9b80 20 client.26381 trim_cache size 0 max 16384 2024-05-10T10:47:04.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.778+0000 7f3572ea9b80 1 Processor -- start 2024-05-10T10:47:04.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.779+0000 7f3572ea9b80 1 -- start start 2024-05-10T10:47:04.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.779+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 0x5632ce4579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.779+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44edf0 0x5632ce4563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.779+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce452c80 0x5632ce463ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.779+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35601be020 con 0x5632ce452c80 2024-05-10T10:47:04.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.779+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35601bfb60 con 0x5632ce44edf0 2024-05-10T10:47:04.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.779+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35601be6f0 con 0x5632ce44e4d0 2024-05-10T10:47:04.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce452c80 0x5632ce463ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce452c80 0x5632ce463ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40558/0 (socket says 172.21.15.112:40558) 2024-05-10T10:47:04.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 -- 172.21.15.112:0/2904465675 learned_addr learned my addr 172.21.15.112:0/2904465675 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:04.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3565bb1700 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44edf0 0x5632ce4563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 -- 172.21.15.112:0/2904465675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 msgr2=0x5632ce4579e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 0x5632ce4579e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 -- 172.21.15.112:0/2904465675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44edf0 msgr2=0x5632ce4563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44edf0 0x5632ce4563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 -- 172.21.15.112:0/2904465675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ce46a450 con 0x5632ce452c80 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.780+0000 7f3566bb3700 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce452c80 0x5632ce463ef0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f35580df290 tx=0x7f35580a41d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bd232dfdae11ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.781+0000 7f354bfff700 1 -- 172.21.15.112:0/2904465675 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580a9140 con 0x5632ce452c80 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.781+0000 7f354bfff700 1 -- 172.21.15.112:0/2904465675 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35580acdf0 con 0x5632ce452c80 2024-05-10T10:47:04.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.781+0000 7f354bfff700 1 -- 172.21.15.112:0/2904465675 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35580cb0c0 con 0x5632ce452c80 2024-05-10T10:47:04.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.782+0000 7f3572ea9b80 1 -- 172.21.15.112:0/2904465675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce452c80 msgr2=0x5632ce463ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.782+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce452c80 0x5632ce463ef0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f35580df290 tx=0x7f35580a41d0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.782+0000 7f3572ea9b80 1 -- 172.21.15.112:0/2904465675 shutdown_connections 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.782+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce452c80 0x5632ce463ef0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.782+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce44e4d0 0x5632ce4579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.782+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/2904465675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44edf0 0x5632ce4563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.782+0000 7f3572ea9b80 1 -- 172.21.15.112:0/2904465675 >> 172.21.15.112:0/2904465675 conn(0x5632ce37a100 msgr2=0x5632ce443720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.783+0000 7f3572ea9b80 1 -- 172.21.15.112:0/2904465675 shutdown_connections 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.783+0000 7f3572ea9b80 1 -- 172.21.15.112:0/2904465675 wait complete. 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.783+0000 7f3572ea9b80 1 -- start start 2024-05-10T10:47:04.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.783+0000 7f3572ea9b80 1 Processor -- start 2024-05-10T10:47:04.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.783+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44e4d0 0x5632ce45c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce45c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3572ea9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce452c80 0x5632ce45cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35601bf490 con 0x5632ce44edf0 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35601bedc0 con 0x5632ce44e4d0 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3572ea9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35601bbfc0 con 0x5632ce452c80 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3572ea9b80 10 client.0 _ll_register_callbacks cb 0x5632ce338920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce45c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce45c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40570/0 (socket says 172.21.15.112:40570) 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 -- 172.21.15.112:0/1351105949 learned_addr learned my addr 172.21.15.112:0/1351105949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f35663b2700 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44e4d0 0x5632ce45c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 -- 172.21.15.112:0/1351105949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce452c80 msgr2=0x5632ce45cad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce452c80 0x5632ce45cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 -- 172.21.15.112:0/1351105949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44e4d0 msgr2=0x5632ce45c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44e4d0 0x5632ce45c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f3565bb1700 1 -- 172.21.15.112:0/1351105949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ce4d1460 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.784+0000 7f35663b2700 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44e4d0 0x5632ce45c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3565bb1700 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce45c590 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3560198ee0 tx=0x7f3550004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a54b03b32ab296d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3549ffb700 1 -- 172.21.15.112:0/1351105949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550009ac0 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35580acdf0 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3549ffb700 1 -- 172.21.15.112:0/1351105949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3550004830 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3549ffb700 1 -- 172.21.15.112:0/1351105949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3550031050 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3572ea9b80 10 client.17031 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.785+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5632ce4d27c0 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3549ffb700 1 -- 172.21.15.112:0/1351105949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35580acdf0 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3549ffb700 1 -- 172.21.15.112:0/1351105949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f355001c610 con 0x5632ce44edf0 2024-05-10T10:47:04.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3572ea9b80 10 client.17031 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3572ea9b80 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3572ea9b80 20 client.17031 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce4d2f30 0x5632ce4d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3572ea9b80 4 client.17031 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632ce41a650 con 0x5632ce4d2f30 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.786+0000 7f35663b2700 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce4d2f30 0x5632ce4d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.787+0000 7f3549ffb700 10 client.17031 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.787+0000 7f35663b2700 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce4d2f30 0x5632ce4d5350 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295bd8b774733944 server_cookie=7bdd6b289bd8bc27 in_seq=0 out_seq=0 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3549ffb700 1 -- 172.21.15.112:0/1351105949 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632ce41a650 con 0x5632ce4d2f30 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3549ffb700 10 client.17031 handle_command_reply: tid=0 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 client.17031 shutdown 2024-05-10T10:47:04.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 msgr2=0x5632ce45c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce45c590 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3560198ee0 tx=0x7f3550004750 comp rx=0 tx=0).stop 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 shutdown_connections 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5632ce4d2f30 0x5632ce4d5350 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5632ce452c80 0x5632ce45cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632ce44e4d0 0x5632ce45c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3549ffb700 0 client.17031 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 --2- 172.21.15.112:0/1351105949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632ce44edf0 0x5632ce45c590 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.788+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 >> 172.21.15.112:0/1351105949 conn(0x5632ce37a100 msgr2=0x5632ce451520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.789+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 shutdown_connections 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.789+0000 7f3572ea9b80 1 -- 172.21.15.112:0/1351105949 wait complete. 2024-05-10T10:47:04.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.789+0000 7f3572ea9b80 20 client.17031 trim_cache size 0 max 16384 2024-05-10T10:47:04.796 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:47:04.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f354c0b2930 msgr2=0x7f354c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f354c0b2930 0x7f354c0b4df0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5632ce44a710 tx=0x7f3550009260 comp rx=0 tx=0).stop 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3560108410 msgr2=0x7f3560197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3560108410 0x7f3560197c60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f355800efc0 tx=0x7f355800c3a0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 shutdown_connections 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3560108410 0x7f3560197c60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35601023a0 0x7f356019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f354c0b2930 0x7f354c0b4df0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 --2- 172.21.15.112:0/199559969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3560102d60 0x7f3560197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 >> 172.21.15.112:0/199559969 conn(0x7f3560079350 msgr2=0x7f3560100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 shutdown_connections 2024-05-10T10:47:04.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:04.790+0000 7f3568616700 1 -- 172.21.15.112:0/199559969 wait complete. 2024-05-10T10:47:05.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:05 smithi195 ceph-mon[33773]: pgmap v635: 129 pgs: 129 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 502 op/s 2024-05-10T10:47:05.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:05 smithi112 ceph-mon[30100]: pgmap v635: 129 pgs: 129 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 502 op/s 2024-05-10T10:47:05.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:05 smithi148 ceph-mon[33568]: pgmap v635: 129 pgs: 129 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 502 op/s 2024-05-10T10:47:06.669 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T10:47:06.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T10:47:07.002 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:47:07.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:07 smithi195 ceph-mon[33773]: pgmap v636: 129 pgs: 129 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 659 op/s 2024-05-10T10:47:07.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:07 smithi112 ceph-mon[30100]: pgmap v636: 129 pgs: 129 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 659 op/s 2024-05-10T10:47:07.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:07 smithi148 ceph-mon[33568]: pgmap v636: 129 pgs: 129 active+clean; 73 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 659 op/s 2024-05-10T10:47:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.673+0000 7f3321732640 1 -- 172.21.15.112:0/3909165173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 msgr2=0x7f331c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.673+0000 7f3321732640 1 --2- 172.21.15.112:0/3909165173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c109d90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3314009ca0 tx=0x7f331402a8e0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 -- 172.21.15.112:0/3909165173 shutdown_connections 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 --2- 172.21.15.112:0/3909165173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c109d90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 --2- 172.21.15.112:0/3909165173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c102b40 0x7f331c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 --2- 172.21.15.112:0/3909165173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f331c101950 0x7f331c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 -- 172.21.15.112:0/3909165173 >> 172.21.15.112:0/3909165173 conn(0x7f331c0fcec0 msgr2=0x7f331c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 -- 172.21.15.112:0/3909165173 shutdown_connections 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 -- 172.21.15.112:0/3909165173 wait complete. 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 Processor -- start 2024-05-10T10:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.674+0000 7f3321732640 1 -- start start 2024-05-10T10:47:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f3321732640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c101950 0x7f331c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f3321732640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f331c102b40 0x7f331c19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f3321732640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c1a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f3321732640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f331c10e4b0 con 0x7f331c103500 2024-05-10T10:47:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f3321732640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f331c10e330 con 0x7f331c101950 2024-05-10T10:47:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f3321732640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f331c10e630 con 0x7f331c102b40 2024-05-10T10:47:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c1a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c101950 0x7f331c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c1a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55528/0 (socket says 172.21.15.112:55528) 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331b7fe640 1 -- 172.21.15.112:0/2176314106 learned_addr learned my addr 172.21.15.112:0/2176314106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c101950 0x7f331c19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:53344/0 (socket says 172.21.15.112:53344) 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f3313fff640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f331c102b40 0x7f331c19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 -- 172.21.15.112:0/2176314106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f331c102b40 msgr2=0x7f331c19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f331c102b40 0x7f331c19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 -- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 msgr2=0x7f331c1a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c1a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 -- 172.21.15.112:0/2176314106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c1a2960 con 0x7f331c101950 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331b7fe640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c1a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.675+0000 7f331affd640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c101950 0x7f331c19ad90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f330c009fd0 tx=0x7f330c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.676+0000 7f3318ff9640 1 -- 172.21.15.112:0/2176314106 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c015020 con 0x7f331c101950 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.676+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c1a2c50 con 0x7f331c101950 2024-05-10T10:47:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.676+0000 7f3318ff9640 1 -- 172.21.15.112:0/2176314106 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f330c011cc0 con 0x7f331c101950 2024-05-10T10:47:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.676+0000 7f3318ff9640 1 -- 172.21.15.112:0/2176314106 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c010d20 con 0x7f331c101950 2024-05-10T10:47:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.676+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c1a31a0 con 0x7f331c101950 2024-05-10T10:47:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f3318ff9640 1 -- 172.21.15.112:0/2176314106 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f330c018020 con 0x7f331c101950 2024-05-10T10:47:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 Processor -- start 2024-05-10T10:47:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 -- start start 2024-05-10T10:47:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a253a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a253cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614a24995d0 con 0x5614a25357a0 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5614a2452890 con 0x5614a253a910 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f332444f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5614a2495590 con 0x5614a2534db0 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a253a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a253a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55544/0 (socket says 172.21.15.112:55544) 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 -- 172.21.15.112:0/1427945558 learned_addr learned my addr 172.21.15.112:0/1427945558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331b7fe640 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f3318ff9640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33080b28a0 0x7f33080b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 -- 172.21.15.112:0/1427945558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 msgr2=0x5614a25351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 -- 172.21.15.112:0/1427945558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 msgr2=0x5614a253cda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a253cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 -- 172.21.15.112:0/1427945558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a21ab210 con 0x5614a25357a0 2024-05-10T10:47:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331b7fe640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33080b28a0 0x7f33080b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f3318ff9640 1 -- 172.21.15.112:0/2176314106 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f330c0e1d80 con 0x7f331c101950 2024-05-10T10:47:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f331affd640 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a253a2a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f330c0a57d0 tx=0x7f330c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f1d75de31ebaf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f33117fa640 1 -- 172.21.15.112:0/1427945558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c00c150 con 0x5614a25357a0 2024-05-10T10:47:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f33117fa640 1 -- 172.21.15.112:0/1427945558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f330c010ec0 con 0x5614a25357a0 2024-05-10T10:47:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.677+0000 7f33117fa640 1 -- 172.21.15.112:0/1427945558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c057a00 con 0x5614a25357a0 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/1427945558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 msgr2=0x5614a253a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a253a2a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f330c0a57d0 tx=0x7f330c021000 comp rx=0 tx=0).stop 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33080b28a0 0x7f33080b4d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3314031e70 tx=0x7f33140095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/1427945558 shutdown_connections 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a253cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a253a2a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/1427945558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/1427945558 >> 172.21.15.112:0/1427945558 conn(0x5614a2530420 msgr2=0x5614a2530820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/1427945558 shutdown_connections 2024-05-10T10:47:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/1427945558 wait complete. 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- start start 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 Processor -- start 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a253efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a25357a0 0x5614a253f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a253a910 0x5614a2546520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614a253d8c0 con 0x5614a253a910 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5614a2456370 con 0x5614a25357a0 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5614a253da40 con 0x5614a2534db0 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a253a910 0x5614a2546520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a253a910 0x5614a2546520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55554/0 (socket says 172.21.15.112:55554) 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 -- 172.21.15.112:0/396563724 learned_addr learned my addr 172.21.15.112:0/396563724 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331affd640 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a253efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 10 client.0 _ll_register_callbacks cb 0x5614a21aaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 -- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 msgr2=0x5614a253efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a253efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 -- 172.21.15.112:0/396563724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a25357a0 msgr2=0x5614a253f4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a25357a0 0x5614a253f4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 -- 172.21.15.112:0/396563724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a2277620 con 0x5614a253a910 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331affd640 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a253efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331b7fe640 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a253a910 0x5614a2546520 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3314003eb0 tx=0x7f331402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd175b263c7b27e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 1 -- 172.21.15.112:0/396563724 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33140309f0 con 0x5614a253a910 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f330c010ec0 con 0x5614a253a910 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 1 -- 172.21.15.112:0/396563724 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3314030b90 con 0x5614a253a910 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 1 -- 172.21.15.112:0/396563724 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3314030d90 con 0x5614a253a910 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 10 client.17043 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5614a25495a0 con 0x5614a253a910 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 1 -- 172.21.15.112:0/396563724 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f330c010ec0 con 0x5614a253a910 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 1 -- 172.21.15.112:0/396563724 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f3314051020 con 0x5614a253a910 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 10 client.17043 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 20 client.17043 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a254ace0 0x5614a254d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 4 client.17043 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614a24a71e0 con 0x5614a254ace0 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331affd640 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a254ace0 0x5614a254d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f331affd640 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a254ace0 0x5614a254d170 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58df86e112a381cc server_cookie=8c61600ee84fcce0 in_seq=0 out_seq=0 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 10 client.17043 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 1 -- 172.21.15.112:0/396563724 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614a24a71e0 con 0x5614a254ace0 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 10 client.17043 handle_command_reply: tid=0 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 client.17043 shutdown 2024-05-10T10:47:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a253a910 msgr2=0x5614a2546520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a253a910 0x5614a2546520 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3314003eb0 tx=0x7f331402a8e0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 shutdown_connections 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a253a910 0x5614a2546520 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a254ace0 0x5614a254d170 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a25357a0 0x5614a253f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 --2- 172.21.15.112:0/396563724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a253efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f33027fc640 0 client.17043 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.682+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 >> 172.21.15.112:0/396563724 conn(0x5614a2530420 msgr2=0x5614a252f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.696+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 shutdown_connections 2024-05-10T10:47:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.696+0000 7f332444f740 1 -- 172.21.15.112:0/396563724 wait complete. 2024-05-10T10:47:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.696+0000 7f332444f740 20 client.17043 trim_cache size 0 max 16384 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.707+0000 7f332444f740 1 Processor -- start 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.707+0000 7f332444f740 1 -- start start 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a2533a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a25464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a2551a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1ad6c0 con 0x5614a25357a0 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1af200 con 0x5614a253a910 2024-05-10T10:47:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1add90 con 0x5614a2534db0 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a25464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a25464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55560/0 (socket says 172.21.15.112:55560) 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/1507218680 learned_addr learned my addr 172.21.15.112:0/1507218680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f331affd640 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a2533a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/1507218680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 msgr2=0x5614a2533a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a2533a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/1507218680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 msgr2=0x5614a2551a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a2551a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/1507218680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a252bda0 con 0x5614a25357a0 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a25464e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f330400b9d0 tx=0x7f330400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523fcf236d7265ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3310ff9640 1 -- 172.21.15.112:0/1507218680 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304010d10 con 0x5614a25357a0 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3310ff9640 1 -- 172.21.15.112:0/1507218680 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33040035f0 con 0x5614a25357a0 2024-05-10T10:47:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3310ff9640 1 -- 172.21.15.112:0/1507218680 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33040044c0 con 0x5614a25357a0 2024-05-10T10:47:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- 172.21.15.112:0/1507218680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 msgr2=0x5614a25464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a25464e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f330400b9d0 tx=0x7f330400be90 comp rx=0 tx=0).stop 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- 172.21.15.112:0/1507218680 shutdown_connections 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a2551a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a25464e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- 172.21.15.112:0/1507218680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a2533a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- 172.21.15.112:0/1507218680 >> 172.21.15.112:0/1507218680 conn(0x5614a2530420 msgr2=0x5614a2533330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- 172.21.15.112:0/1507218680 shutdown_connections 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- 172.21.15.112:0/1507218680 wait complete. 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- start start 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 Processor -- start 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a2544720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a25456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1aeb30 con 0x5614a25357a0 2024-05-10T10:47:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1ae460 con 0x5614a253a910 2024-05-10T10:47:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1ab6a0 con 0x5614a2534db0 2024-05-10T10:47:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 10 client.0 _ll_register_callbacks cb 0x7f3314005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a2544720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a2544720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55562/0 (socket says 172.21.15.112:55562) 2024-05-10T10:47:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/2213057982 learned_addr learned my addr 172.21.15.112:0/2213057982 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f331affd640 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 msgr2=0x5614a25441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/2213057982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 msgr2=0x5614a25456e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a25456e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 -- 172.21.15.112:0/2213057982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a2530250 con 0x5614a25357a0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f3313fff640 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a2544720 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3304003e00 tx=0x7f330400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2321287a22c494 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f33027fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f33027fc640 1 -- 172.21.15.112:0/2213057982 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330402a060 con 0x5614a25357a0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f33027fc640 1 -- 172.21.15.112:0/2213057982 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3304010d20 con 0x5614a25357a0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33040035f0 con 0x5614a25357a0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f33027fc640 1 -- 172.21.15.112:0/2213057982 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3304031050 con 0x5614a25357a0 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 10 client.17049 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5614a25b5a50 con 0x5614a25357a0 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f33027fc640 1 -- 172.21.15.112:0/2213057982 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33040035f0 con 0x5614a25357a0 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.712+0000 7f33027fc640 1 -- 172.21.15.112:0/2213057982 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f33040046c0 con 0x5614a25357a0 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f332444f740 10 client.17049 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f332444f740 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f332444f740 20 client.17049 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f332444f740 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a25b6300 0x5614a25b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f332444f740 4 client.17049 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614a24a1050 con 0x5614a25b6300 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f331affd640 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a25b6300 0x5614a25b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f331affd640 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a25b6300 0x5614a25b8720 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ea05aae48fedaf server_cookie=3262d68424584523 in_seq=0 out_seq=0 2024-05-10T10:47:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.715+0000 7f33027fc640 10 client.17049 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f33027fc640 1 -- 172.21.15.112:0/2213057982 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614a24a1050 con 0x5614a25b6300 2024-05-10T10:47:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f33027fc640 10 client.17049 handle_command_reply: tid=0 2024-05-10T10:47:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 client.17049 shutdown 2024-05-10T10:47:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 msgr2=0x5614a2544720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a2544720 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3304003e00 tx=0x7f330400b7d0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 shutdown_connections 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5614a25b6300 0x5614a25b8720 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614a253a910 0x5614a25456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f33027fc640 0 client.17049 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614a25357a0 0x5614a2544720 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 --2- 172.21.15.112:0/2213057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614a2534db0 0x5614a25441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 >> 172.21.15.112:0/2213057982 conn(0x5614a2530420 msgr2=0x5614a252ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 shutdown_connections 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 1 -- 172.21.15.112:0/2213057982 wait complete. 2024-05-10T10:47:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.756+0000 7f332444f740 20 client.17049 trim_cache size 0 max 16384 2024-05-10T10:47:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:47:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:47:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:47:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33080b28a0 msgr2=0x7f33080b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33080b28a0 0x7f33080b4d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3314031e70 tx=0x7f33140095c0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c101950 msgr2=0x7f331c19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c101950 0x7f331c19ad90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f330c009fd0 tx=0x7f330c00b870 comp rx=0 tx=0).stop 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 shutdown_connections 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f33080b28a0 0x7f33080b4d60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331c103500 0x7f331c1a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f331c102b40 0x7f331c19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 --2- 172.21.15.112:0/2176314106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331c101950 0x7f331c19ad90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 >> 172.21.15.112:0/2176314106 conn(0x7f331c0fcec0 msgr2=0x7f331c10a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 shutdown_connections 2024-05-10T10:47:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:08.757+0000 7f3321732640 1 -- 172.21.15.112:0/2176314106 wait complete. 2024-05-10T10:47:08.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.936+0000 7f917fa79700 1 -- 172.21.15.112:0/3432691979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 msgr2=0x7f91781103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.936+0000 7f917fa79700 1 --2- 172.21.15.112:0/3432691979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f91781103d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f917000b0a0 tx=0x7f917000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.936+0000 7f917fa79700 1 -- 172.21.15.112:0/3432691979 shutdown_connections 2024-05-10T10:47:08.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.936+0000 7f917fa79700 1 --2- 172.21.15.112:0/3432691979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f91781103d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.936+0000 7f917fa79700 1 --2- 172.21.15.112:0/3432691979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9178108460 0x7f9178108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.937+0000 7f917fa79700 1 --2- 172.21.15.112:0/3432691979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9178103550 0x7f91781039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.937+0000 7f917fa79700 1 -- 172.21.15.112:0/3432691979 >> 172.21.15.112:0/3432691979 conn(0x7f917807a600 msgr2=0x7f917807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.940+0000 7f917fa79700 1 -- 172.21.15.112:0/3432691979 shutdown_connections 2024-05-10T10:47:08.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.940+0000 7f917fa79700 1 -- 172.21.15.112:0/3432691979 wait complete. 2024-05-10T10:47:08.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.941+0000 7f917fa79700 1 Processor -- start 2024-05-10T10:47:08.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.942+0000 7f917fa79700 1 -- start start 2024-05-10T10:47:08.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.942+0000 7f917fa79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9178103550 0x7f917819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917fa79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f917819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917d815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9178103550 0x7f917819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917d815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9178103550 0x7f917819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:53398/0 (socket says 172.21.15.112:53398) 2024-05-10T10:47:08.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917d014700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f917819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917fa79700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9178108460 0x7f9178199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917d014700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f917819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55584/0 (socket says 172.21.15.112:55584) 2024-05-10T10:47:08.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917d815700 1 -- 172.21.15.112:0/595691753 learned_addr learned my addr 172.21.15.112:0/595691753 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9178113600 con 0x7f9178103ef0 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9178113480 con 0x7f9178103550 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9178113780 con 0x7f9178108460 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.943+0000 7f917e016700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9178108460 0x7f9178199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917d014700 1 -- 172.21.15.112:0/595691753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9178108460 msgr2=0x7f9178199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917d014700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9178108460 0x7f9178199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917d014700 1 -- 172.21.15.112:0/595691753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9178103550 msgr2=0x7f917819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917d014700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9178103550 0x7f917819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917d014700 1 -- 172.21.15.112:0/595691753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917819a080 con 0x7f9178103ef0 2024-05-10T10:47:08.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917e016700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9178108460 0x7f9178199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:08.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917d014700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f917819fca0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f916800b450 tx=0x7f916800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f916effd700 1 -- 172.21.15.112:0/595691753 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168015020 con 0x7f9178103ef0 2024-05-10T10:47:08.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f916effd700 1 -- 172.21.15.112:0/595691753 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9168011cc0 con 0x7f9178103ef0 2024-05-10T10:47:08.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917819a370 con 0x7f9178103ef0 2024-05-10T10:47:08.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.944+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91781a6630 con 0x7f9178103ef0 2024-05-10T10:47:08.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.945+0000 7f916effd700 1 -- 172.21.15.112:0/595691753 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168010be0 con 0x7f9178103ef0 2024-05-10T10:47:08.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.949+0000 7f916effd700 1 -- 172.21.15.112:0/595691753 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9168018020 con 0x7f9178103ef0 2024-05-10T10:47:08.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.949+0000 7f918a30cb80 1 Processor -- start 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.949+0000 7f918a30cb80 1 -- start start 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59861610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5985b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f5985bdd0 0x563f5985c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f918a30cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f597ad9c0 con 0x563f5985bdd0 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f918a30cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563f59712e20 con 0x563f59861230 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f918a30cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563f597a9980 con 0x563f59861b50 2024-05-10T10:47:08.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f917d014700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f5985bdd0 0x563f5985c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f917d014700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f5985bdd0 0x563f5985c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55598/0 (socket says 172.21.15.112:55598) 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f917d014700 1 -- 172.21.15.112:0/392219188 learned_addr learned my addr 172.21.15.112:0/392219188 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f917d815700 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5985b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.950+0000 7f917e016700 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59861610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f916effd700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91640b28c0 0x7f91640b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f917d014700 1 -- 172.21.15.112:0/392219188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 msgr2=0x563f5985b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f917d014700 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5985b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f917d014700 1 -- 172.21.15.112:0/392219188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 msgr2=0x563f59861610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f917d014700 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59861610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f916effd700 1 -- 172.21.15.112:0/595691753 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91680e1d40 con 0x7f9178103ef0 2024-05-10T10:47:08.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f917d014700 1 -- 172.21.15.112:0/392219188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f596fa950 con 0x563f5985bdd0 2024-05-10T10:47:08.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f917e016700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91640b28c0 0x7f91640b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f917d014700 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f5985bdd0 0x563f5985c210 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f91680a8cd0 tx=0x7f9168004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c07e4f2e471481e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.951+0000 7f9163fff700 1 -- 172.21.15.112:0/392219188 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680a8920 con 0x563f5985bdd0 2024-05-10T10:47:08.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f9163fff700 1 -- 172.21.15.112:0/392219188 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91680a8ae0 con 0x563f5985bdd0 2024-05-10T10:47:08.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f9163fff700 1 -- 172.21.15.112:0/392219188 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680a8740 con 0x563f5985bdd0 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 -- 172.21.15.112:0/392219188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f5985bdd0 msgr2=0x563f5985c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f917e016700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91640b28c0 0x7f91640b4d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9170009720 tx=0x7f917000b3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f5985bdd0 0x563f5985c210 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f91680a8cd0 tx=0x7f9168004860 comp rx=0 tx=0).stop 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 -- 172.21.15.112:0/392219188 shutdown_connections 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f5985bdd0 0x563f5985c210 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59861610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 --2- 172.21.15.112:0/392219188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5985b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 -- 172.21.15.112:0/392219188 >> 172.21.15.112:0/392219188 conn(0x563f59787bf0 msgr2=0x563f59856320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.952+0000 7f918a30cb80 1 -- 172.21.15.112:0/392219188 shutdown_connections 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 -- 172.21.15.112:0/392219188 wait complete. 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 -- start start 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 Processor -- start 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f5985bdd0 0x563f598677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59864b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f598650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f598684d0 con 0x563f59861b50 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563f59766f50 con 0x563f59861230 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563f59868650 con 0x563f5985bdd0 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.953+0000 7f918a30cb80 10 client.0 _ll_register_callbacks cb 0x563f59741630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917e016700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f598650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917e016700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f598650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55608/0 (socket says 172.21.15.112:55608) 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917e016700 1 -- 172.21.15.112:0/2263879013 learned_addr learned my addr 172.21.15.112:0/2263879013 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917d014700 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59864b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917d014700 1 -- 172.21.15.112:0/2263879013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f5985bdd0 msgr2=0x563f598677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917d014700 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f5985bdd0 0x563f598677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917d014700 1 -- 172.21.15.112:0/2263879013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 msgr2=0x563f598650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917d014700 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f598650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917d014700 1 -- 172.21.15.112:0/2263879013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f597647e0 con 0x563f59861230 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917e016700 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f598650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f917d014700 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59864b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f91680a5790 tx=0x7f91680e53e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a68b01bb46eee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f91617fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f91617fa700 1 -- 172.21.15.112:0/2263879013 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680a8920 con 0x563f59861230 2024-05-10T10:47:08.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f91617fa700 1 -- 172.21.15.112:0/2263879013 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f91680b19a0 con 0x563f59861230 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.954+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91680a8ae0 con 0x563f59861230 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f91617fa700 1 -- 172.21.15.112:0/2263879013 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f91680a96d0 con 0x563f59861230 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 10 client.26405 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563f598684d0 con 0x563f59861230 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f91617fa700 1 -- 172.21.15.112:0/2263879013 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91680a8ae0 con 0x563f59861230 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f91617fa700 1 -- 172.21.15.112:0/2263879013 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f91680a8230 con 0x563f59861230 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 10 client.26405 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 20 client.26405 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598750e0 0x563f598774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 4 client.26405 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.955+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f597bb3a0 con 0x563f598750e0 2024-05-10T10:47:08.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.956+0000 7f917d815700 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598750e0 0x563f598774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.957+0000 7f91617fa700 10 client.26405 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.957+0000 7f917d815700 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598750e0 0x563f598774d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2088e6a8f59b8ae0 server_cookie=3bd71691dcb9fce2 in_seq=0 out_seq=0 2024-05-10T10:47:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f91617fa700 1 -- 172.21.15.112:0/2263879013 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f597bb3a0 con 0x563f598750e0 2024-05-10T10:47:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f91617fa700 10 client.26405 handle_command_reply: tid=0 2024-05-10T10:47:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 client.26405 shutdown 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 msgr2=0x563f59864b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59864b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f91680a5790 tx=0x7f91680e53e0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 shutdown_connections 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598750e0 0x563f598774d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f598650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f91617fa700 0 client.26405 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f5985bdd0 0x563f598677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2263879013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f59861230 0x563f59864b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.958+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 >> 172.21.15.112:0/2263879013 conn(0x563f59787bf0 msgr2=0x563f59850bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.959+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 shutdown_connections 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.959+0000 7f918a30cb80 1 -- 172.21.15.112:0/2263879013 wait complete. 2024-05-10T10:47:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.959+0000 7f918a30cb80 20 client.26405 trim_cache size 0 max 16384 2024-05-10T10:47:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.983+0000 7f918a30cb80 1 Processor -- start 2024-05-10T10:47:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.983+0000 7f918a30cb80 1 -- start start 2024-05-10T10:47:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5985b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861230 0x563f59871590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f59863d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f918a30cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91781af340 con 0x563f59861b50 2024-05-10T10:47:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f918a30cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b0e80 con 0x563f5985bdd0 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f918a30cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91781afa10 con 0x563f59861230 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f59863d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f59863d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55620/0 (socket says 172.21.15.112:55620) 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 -- 172.21.15.112:0/2291628199 learned_addr learned my addr 172.21.15.112:0/2291628199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917d815700 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5985b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917d014700 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861230 0x563f59871590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 -- 172.21.15.112:0/2291628199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861230 msgr2=0x563f59871590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861230 0x563f59871590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 -- 172.21.15.112:0/2291628199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 msgr2=0x563f5985b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5985b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 -- 172.21.15.112:0/2291628199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f59861860 con 0x563f59861b50 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917d815700 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5985b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f917e016700 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f59863d30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f917000bf70 tx=0x7f91700093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944508afaf68f682 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.984+0000 7f91637fe700 1 -- 172.21.15.112:0/2291628199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170031070 con 0x563f59861b50 2024-05-10T10:47:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f91637fe700 1 -- 172.21.15.112:0/2291628199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9170005a00 con 0x563f59861b50 2024-05-10T10:47:08.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f91637fe700 1 -- 172.21.15.112:0/2291628199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170007d90 con 0x563f59861b50 2024-05-10T10:47:08.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f918a30cb80 1 -- 172.21.15.112:0/2291628199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 msgr2=0x563f59863d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f59863d30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f917000bf70 tx=0x7f91700093a0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f918a30cb80 1 -- 172.21.15.112:0/2291628199 shutdown_connections 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861b50 0x563f59863d30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5985b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2291628199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861230 0x563f59871590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.985+0000 7f918a30cb80 1 -- 172.21.15.112:0/2291628199 >> 172.21.15.112:0/2291628199 conn(0x563f59787bf0 msgr2=0x563f59860580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.986+0000 7f918a30cb80 1 -- 172.21.15.112:0/2291628199 shutdown_connections 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.986+0000 7f918a30cb80 1 -- 172.21.15.112:0/2291628199 wait complete. 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.986+0000 7f918a30cb80 1 -- start start 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.986+0000 7f918a30cb80 1 Processor -- start 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5986d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861230 0x563f5986de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f918a30cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5986a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f918a30cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b07b0 con 0x563f59861230 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f918a30cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b00e0 con 0x563f5985bdd0 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f918a30cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91781ad2e0 con 0x563f59861b50 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f918a30cb80 10 client.0 _ll_register_callbacks cb 0x563f59741630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f917d815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5986d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f917d815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5986d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:53430/0 (socket says 172.21.15.112:53430) 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f917d815700 1 -- 172.21.15.112:0/2840130649 learned_addr learned my addr 172.21.15.112:0/2840130649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f917d014700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861230 0x563f5986de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f917e016700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5986a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f917d815700 1 -- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 msgr2=0x563f5986a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.987+0000 7f917d815700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5986a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f917d815700 1 -- 172.21.15.112:0/2840130649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861230 msgr2=0x563f5986de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f917d815700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861230 0x563f5986de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f917d815700 1 -- 172.21.15.112:0/2840130649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5986af90 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f917d014700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861230 0x563f5986de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f917d815700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5986d910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9174000c00 tx=0x7f917401d6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac8bf3c97e5f95c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f91617fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f91617fa700 1 -- 172.21.15.112:0/2840130649 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174010900 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9170005a00 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f91617fa700 1 -- 172.21.15.112:0/2840130649 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9174010aa0 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f91617fa700 1 -- 172.21.15.112:0/2840130649 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f917402f050 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f918a30cb80 10 client.26411 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.988+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563f598e4620 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f91617fa700 1 -- 172.21.15.112:0/2840130649 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9170005a00 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f91617fa700 1 -- 172.21.15.112:0/2840130649 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f917400f3c0 con 0x563f5985bdd0 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f918a30cb80 10 client.26411 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f918a30cb80 10 client.26411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:08.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f918a30cb80 20 client.26411 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598e4cb0 0x563f598e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f918a30cb80 4 client.26411 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f597b5440 con 0x563f598e4cb0 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.989+0000 7f917d014700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598e4cb0 0x563f598e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.990+0000 7f91617fa700 10 client.26411 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.990+0000 7f917d014700 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598e4cb0 0x563f598e70d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7866c226e30fd805 server_cookie=ecc915d2d84f0d03 in_seq=0 out_seq=0 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f91617fa700 1 -- 172.21.15.112:0/2840130649 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f597b5440 con 0x563f598e4cb0 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f91617fa700 10 client.26411 handle_command_reply: tid=0 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 client.26411 shutdown 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 msgr2=0x563f5986d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5986d910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9174000c00 tx=0x7f917401d6f0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 shutdown_connections 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x563f598e4cb0 0x563f598e70d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f91617fa700 0 client.26411 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f59861b50 0x563f5986a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5985bdd0 0x563f5986d910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 --2- 172.21.15.112:0/2840130649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f59861230 0x563f5986de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 >> 172.21.15.112:0/2840130649 conn(0x563f59787bf0 msgr2=0x563f59866190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 shutdown_connections 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.991+0000 7f918a30cb80 1 -- 172.21.15.112:0/2840130649 wait complete. 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.992+0000 7f918a30cb80 20 client.26411 trim_cache size 0 max 16384 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:47:08.998 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-10T10:47:08.999 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "ee0d0c99-eda8-47b3-b8a3-105151059d89", 2024-05-10T10:47:08.999 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-10T10:47:08.999 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:47:08.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.992+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91640b28c0 msgr2=0x7f91640b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.992+0000 7f917fa79700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91640b28c0 0x7f91640b4d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9170009720 tx=0x7f917000b3b0 comp rx=0 tx=0).stop 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.992+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 msgr2=0x7f917819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.992+0000 7f917fa79700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f917819fca0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f916800b450 tx=0x7f916800b760 comp rx=0 tx=0).stop 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 shutdown_connections 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f91640b28c0 0x7f91640b4d80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9178108460 0x7f9178199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9178103550 0x7f917819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 --2- 172.21.15.112:0/595691753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178103ef0 0x7f917819fca0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 >> 172.21.15.112:0/595691753 conn(0x7f917807a600 msgr2=0x7f9178101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 shutdown_connections 2024-05-10T10:47:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:08.993+0000 7f917fa79700 1 -- 172.21.15.112:0/595691753 wait complete. 2024-05-10T10:47:09.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:09 smithi195 ceph-mon[33773]: pgmap v637: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 509 op/s 2024-05-10T10:47:09.316 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:47:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: 2024-05-10T10:47:09.014+0000 7ff3584cc700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002bb [...2,head] /volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp/ auth{0=1,3=1,4=1} v5735 ap=2 f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1) n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2) (inest sync->mix g=0,3,4) caps={15294=pAsLsXsFs/-@1} | lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x5572a8cae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000002bb:[<0x10000000007/ltp v436>,<0x10000000006/xfstests-dev v72>,<0x10000000005/tmp v10>,<0x10000000003/client.0 v12>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v28>,<0x10000000001/sv_0 v52>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x100000002bb:[<0x10000000007/ltp v5735>,<0x10000000006/xfstests-dev v12369>,<0x10000000005/tmp v6802>,<0x10000000003/client.0 v703>,<0x10000000002/18bfb460-85d5-4334-ae0a-f99cd839dfc2 v712>,<0x10000000001/sv_0 v718>,<0x10000000000/qa v700>,<0x1/volumes v684>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T10:42:55.038905+0000 15=14+1)","ondisk_value.rstat":"n(v0 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","memory_value.dirstat":"f(v3 m2024-05-10T10:42:55.898522+0000 17=16+1)","memory_value.rstat":"n(v1 rc2024-05-10T10:42:55.943281+0000 b1382672 18=16+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T10:47:09.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:09 smithi112 ceph-mon[30100]: pgmap v637: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 509 op/s 2024-05-10T10:47:09.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:09 smithi148 ceph-mon[33568]: pgmap v637: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 509 op/s 2024-05-10T10:47:09.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:47:10.211 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:47:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub queued for path: / 2024-05-10T10:47:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: active paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:10 smithi112 ceph-mon[30100]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub queued for path: / 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: active paths [/] 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:10 smithi148 ceph-mon[33568]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub queued for path: / 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: active paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:10.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:10 smithi195 ceph-mon[33773]: scrub summary: idle+waiting paths [/] 2024-05-10T10:47:11.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:11 smithi112 ceph-mon[30100]: pgmap v638: 129 pgs: 129 active+clean; 74 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 511 op/s 2024-05-10T10:47:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:11 smithi148 ceph-mon[33568]: pgmap v638: 129 pgs: 129 active+clean; 74 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 511 op/s 2024-05-10T10:47:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:11 smithi195 ceph-mon[33773]: pgmap v638: 129 pgs: 129 active+clean; 74 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 511 op/s 2024-05-10T10:47:12.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/781505695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 msgr2=0x7f9ee41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/781505695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 0x7f9ee41099c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80099c0 tx=0x7f9ed8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/781505695 shutdown_connections 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/781505695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4103f00 0x7f9ee41103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/781505695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 0x7f9ee41099c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/781505695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4103540 0x7f9ee41039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/781505695 >> 172.21.15.112:0/781505695 conn(0x7f9ee407a650 msgr2=0x7f9ee407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/781505695 shutdown_connections 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.056+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/781505695 wait complete. 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 Processor -- start 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 -- start start 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4103540 0x7f9ee40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4103f00 0x7f9ee4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 0x7f9ee4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4113590 con 0x7f9ee41095e0 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4113410 con 0x7f9ee4103f00 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9eeaa9c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4113710 con 0x7f9ee4103540 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4103540 0x7f9ee40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4103f00 0x7f9ee4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4103540 0x7f9ee40757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:40504/0 (socket says 172.21.15.112:40504) 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee8838700 1 -- 172.21.15.112:0/4201589832 learned_addr learned my addr 172.21.15.112:0/4201589832 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee9039700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 0x7f9ee4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee3fff700 1 -- 172.21.15.112:0/4201589832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4103540 msgr2=0x7f9ee40757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4103540 0x7f9ee40757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.057+0000 7f9ee3fff700 1 -- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 msgr2=0x7f9ee4076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 0x7f9ee4076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9ee3fff700 1 -- 172.21.15.112:0/4201589832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee41a7f70 con 0x7f9ee4103f00 2024-05-10T10:47:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9ee9039700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 0x7f9ee4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4103f00 0x7f9ee4075ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000d750 tx=0x7f9ed000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9ee1ffb700 1 -- 172.21.15.112:0/4201589832 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0014030 con 0x7f9ee4103f00 2024-05-10T10:47:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9ee1ffb700 1 -- 172.21.15.112:0/4201589832 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ed00044b0 con 0x7f9ee4103f00 2024-05-10T10:47:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9ee1ffb700 1 -- 172.21.15.112:0/4201589832 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00052f0 con 0x7f9ee4103f00 2024-05-10T10:47:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee41a8260 con 0x7f9ee4103f00 2024-05-10T10:47:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.058+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a87f0 con 0x7f9ee4103f00 2024-05-10T10:47:12.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.061+0000 7f9ee1ffb700 1 -- 172.21.15.112:0/4201589832 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9ed0004750 con 0x7f9ee4103f00 2024-05-10T10:47:12.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.061+0000 7f9ee1ffb700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ed40b2900 0x7f9ed40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ee8838700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ed40b2900 0x7f9ed40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ee1ffb700 1 -- 172.21.15.112:0/4201589832 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9ed00e1e30 con 0x7f9ee4103f00 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 Processor -- start 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 -- start start 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ee8838700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ed40b2900 0x7f9ed40b4dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009990 tx=0x7f9ed8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b7d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b78190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560bf3ac9940 con 0x560bf3b7d1b0 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bf3a2eda0 con 0x560bf3b7dad0 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560bf3ac5900 con 0x560bf3b77d50 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b7d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b7d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55656/0 (socket says 172.21.15.112:55656) 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ee3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.062+0000 7f9ee8838700 1 -- 172.21.15.112:0/4018804829 learned_addr learned my addr 172.21.15.112:0/4018804829 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee9039700 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b78190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee3fff700 1 -- 172.21.15.112:0/4018804829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 msgr2=0x560bf3b78190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b78190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee3fff700 1 -- 172.21.15.112:0/4018804829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 msgr2=0x560bf3b7d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b7d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee3fff700 1 -- 172.21.15.112:0/4018804829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf3a168d0 con 0x560bf3b7dad0 2024-05-10T10:47:12.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee8838700 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:12.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b77810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00a62d0 tx=0x7f9ed00a9060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5033834a0b142a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:12.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9eceffd700 1 -- 172.21.15.112:0/4018804829 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed001d040 con 0x560bf3b7dad0 2024-05-10T10:47:12.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9eceffd700 1 -- 172.21.15.112:0/4018804829 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ed00b1ec0 con 0x560bf3b7dad0 2024-05-10T10:47:12.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.063+0000 7f9eceffd700 1 -- 172.21.15.112:0/4018804829 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0005490 con 0x560bf3b7dad0 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.064+0000 7f9ef532fb80 1 -- 172.21.15.112:0/4018804829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 msgr2=0x560bf3b77810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b77810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00a62d0 tx=0x7f9ed00a9060 comp rx=0 tx=0).stop 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 -- 172.21.15.112:0/4018804829 shutdown_connections 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b78190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/4018804829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b77810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 -- 172.21.15.112:0/4018804829 >> 172.21.15.112:0/4018804829 conn(0x560bf3aa3b70 msgr2=0x560bf3b722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 -- 172.21.15.112:0/4018804829 shutdown_connections 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 -- 172.21.15.112:0/4018804829 wait complete. 2024-05-10T10:47:12.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.065+0000 7f9ef532fb80 1 -- start start 2024-05-10T10:47:12.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 1 Processor -- start 2024-05-10T10:47:12.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b77d50 0x560bf3b836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b80a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7dad0 0x560bf3b80fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560bf3b84450 con 0x560bf3b7d1b0 2024-05-10T10:47:12.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bf3a82ed0 con 0x560bf3b77d50 2024-05-10T10:47:12.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560bf3b845d0 con 0x560bf3b7dad0 2024-05-10T10:47:12.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ee9039700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7dad0 0x560bf3b80fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ee9039700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7dad0 0x560bf3b80fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:40526/0 (socket says 172.21.15.112:40526) 2024-05-10T10:47:12.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ee9039700 1 -- 172.21.15.112:0/1536834799 learned_addr learned my addr 172.21.15.112:0/1536834799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.066+0000 7f9ef532fb80 10 client.0 _ll_register_callbacks cb 0x560bf3a5d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee8838700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b77d50 0x560bf3b836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b80a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee9039700 1 -- 172.21.15.112:0/1536834799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b77d50 msgr2=0x560bf3b836e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee9039700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b77d50 0x560bf3b836e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee9039700 1 -- 172.21.15.112:0/1536834799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 msgr2=0x560bf3b80a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee9039700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b80a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee9039700 1 -- 172.21.15.112:0/1536834799 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf3a80760 con 0x560bf3b7dad0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b80a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ee9039700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7dad0 0x560bf3b80fc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0046e0 tx=0x7f9edc00e7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f990e3cfc8791c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.067+0000 7f9ec3fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.068+0000 7f9ec3fff700 1 -- 172.21.15.112:0/1536834799 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0090d0 con 0x560bf3b7dad0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.069+0000 7f9ec3fff700 1 -- 172.21.15.112:0/1536834799 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9edc009270 con 0x560bf3b7dad0 2024-05-10T10:47:12.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.069+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed00b1ec0 con 0x560bf3b7dad0 2024-05-10T10:47:12.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.069+0000 7f9ec3fff700 1 -- 172.21.15.112:0/1536834799 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9edc027050 con 0x560bf3b7dad0 2024-05-10T10:47:12.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.070+0000 7f9ef532fb80 10 client.26431 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:12.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.070+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560bf3b8ecf0 con 0x560bf3b7dad0 2024-05-10T10:47:12.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.071+0000 7f9ec3fff700 1 -- 172.21.15.112:0/1536834799 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9edc007990 con 0x560bf3b7dad0 2024-05-10T10:47:12.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.072+0000 7f9ec3fff700 1 -- 172.21.15.112:0/1536834799 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f9edc016600 con 0x560bf3b7dad0 2024-05-10T10:47:12.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.072+0000 7f9ef532fb80 10 client.26431 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:12.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.072+0000 7f9ef532fb80 10 client.26431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.072+0000 7f9ef532fb80 20 client.26431 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.072+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3b91330 0x560bf3b93720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.072+0000 7f9ef532fb80 4 client.26431 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.072+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9ed00b1ec0 con 0x560bf3b91330 2024-05-10T10:47:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.073+0000 7f9ee8838700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3b91330 0x560bf3b93720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.076+0000 7f9ec3fff700 10 client.26431 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.076+0000 7f9ee8838700 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3b91330 0x560bf3b93720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6d2a21c81f4d58 server_cookie=91cf12209c244000 in_seq=0 out_seq=0 2024-05-10T10:47:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.078+0000 7f9ec3fff700 1 -- 172.21.15.112:0/1536834799 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9ed00b1ec0 con 0x560bf3b91330 2024-05-10T10:47:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.078+0000 7f9ec3fff700 10 client.26431 handle_command_reply: tid=0 2024-05-10T10:47:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.078+0000 7f9ef532fb80 1 client.26431 shutdown 2024-05-10T10:47:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.078+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7dad0 msgr2=0x560bf3b80fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.078+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7dad0 0x560bf3b80fc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0046e0 tx=0x7f9edc00e7d0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 shutdown_connections 2024-05-10T10:47:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3b91330 0x560bf3b93720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7dad0 0x560bf3b80fc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b77d50 0x560bf3b836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/1536834799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7d1b0 0x560bf3b80a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 >> 172.21.15.112:0/1536834799 conn(0x560bf3aa3b70 msgr2=0x560bf3b6cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ec3fff700 0 client.26431 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 shutdown_connections 2024-05-10T10:47:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.079+0000 7f9ef532fb80 1 -- 172.21.15.112:0/1536834799 wait complete. 2024-05-10T10:47:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.080+0000 7f9ef532fb80 20 client.26431 trim_cache size 0 max 16384 2024-05-10T10:47:12.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.107+0000 7f9ef532fb80 1 Processor -- start 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ef532fb80 1 -- start start 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7d1b0 0x560bf3b8d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7dad0 0x560bf3b7e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ee3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7d1b0 0x560bf3b8d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ee3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7d1b0 0x560bf3b8d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:53496/0 (socket says 172.21.15.112:53496) 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b82f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:40538/0 (socket says 172.21.15.112:40538) 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41b13d0 con 0x560bf3b7dad0 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ee3fff700 1 -- 172.21.15.112:0/3513422715 learned_addr learned my addr 172.21.15.112:0/3513422715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41b2f10 con 0x560bf3b7d1b0 2024-05-10T10:47:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.108+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41b1aa0 con 0x560bf3b77d50 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee9039700 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7dad0 0x560bf3b7e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee8838700 1 -- 172.21.15.112:0/3513422715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7d1b0 msgr2=0x560bf3b8d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee8838700 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7d1b0 0x560bf3b8d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee8838700 1 -- 172.21.15.112:0/3513422715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7dad0 msgr2=0x560bf3b7e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee8838700 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7dad0 0x560bf3b7e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee8838700 1 -- 172.21.15.112:0/3513422715 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf3b6ec80 con 0x560bf3b77d50 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7d1b0 0x560bf3b8d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ee8838700 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b82f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006010 tx=0x7f9ed801a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e34cc941ff53c31 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.109+0000 7f9ece7fc700 1 -- 172.21.15.112:0/3513422715 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed803a070 con 0x560bf3b77d50 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ece7fc700 1 -- 172.21.15.112:0/3513422715 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ed800bc20 con 0x560bf3b77d50 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ece7fc700 1 -- 172.21.15.112:0/3513422715 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8035740 con 0x560bf3b77d50 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 msgr2=0x560bf3b82f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b82f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006010 tx=0x7f9ed801a070 comp rx=0 tx=0).stop 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 shutdown_connections 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b7dad0 0x560bf3b7e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b77d50 0x560bf3b82f20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/3513422715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7d1b0 0x560bf3b8d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.110+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 >> 172.21.15.112:0/3513422715 conn(0x560bf3aa3b70 msgr2=0x560bf3b7dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 shutdown_connections 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 -- 172.21.15.112:0/3513422715 wait complete. 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 -- start start 2024-05-10T10:47:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 Processor -- start 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b77d50 0x560bf3b98a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7d1b0 0x560bf3b98f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b99490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41b2840 con 0x560bf3b77d50 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41b2170 con 0x560bf3b7dad0 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41af370 con 0x560bf3b7d1b0 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.111+0000 7f9ef532fb80 10 client.0 _ll_register_callbacks cb 0x560bf3a5d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b77d50 0x560bf3b98a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b77d50 0x560bf3b98a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55688/0 (socket says 172.21.15.112:55688) 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 -- 172.21.15.112:0/2712840637 learned_addr learned my addr 172.21.15.112:0/2712840637 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7d1b0 0x560bf3b98f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee9039700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b99490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 -- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7d1b0 msgr2=0x560bf3b98f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7d1b0 0x560bf3b98f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 -- 172.21.15.112:0/2712840637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 msgr2=0x560bf3b99490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b99490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 -- 172.21.15.112:0/2712840637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf3bfada0 con 0x560bf3b77d50 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7d1b0 0x560bf3b98f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee9039700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b99490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ee8838700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b77d50 0x560bf3b98a10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8004f40 tx=0x7f9ed8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2903a080b98fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ec3fff700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ec3fff700 1 -- 172.21.15.112:0/2712840637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed803a030 con 0x560bf3b77d50 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ec3fff700 1 -- 172.21.15.112:0/2712840637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9ed800b730 con 0x560bf3b77d50 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.112+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed800bc20 con 0x560bf3b77d50 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ec3fff700 1 -- 172.21.15.112:0/2712840637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9ed800b970 con 0x560bf3b77d50 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 10 client.17082 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560bf3bfc070 con 0x560bf3b77d50 2024-05-10T10:47:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ec3fff700 1 -- 172.21.15.112:0/2712840637 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ed800bc20 con 0x560bf3b77d50 2024-05-10T10:47:12.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ec3fff700 1 -- 172.21.15.112:0/2712840637 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f9ed8023460 con 0x560bf3b77d50 2024-05-10T10:47:12.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 10 client.17082 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:12.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:12.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 20 client.17082 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:12.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3bfc660 0x560bf3bfea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:12.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 4 client.17082 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:47:12.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bf3ad7320 con 0x560bf3bfc660 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.113+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3bfc660 0x560bf3bfea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ec3fff700 10 client.17082 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ee3fff700 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3bfc660 0x560bf3bfea80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d4f5d2fd5dc7ee server_cookie=de9f190a43d11161 in_seq=0 out_seq=0 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ec3fff700 1 -- 172.21.15.112:0/2712840637 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560bf3ad7320 con 0x560bf3bfc660 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ec3fff700 10 client.17082 handle_command_reply: tid=0 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 client.17082 shutdown 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b77d50 msgr2=0x560bf3b98a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b77d50 0x560bf3b98a10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8004f40 tx=0x7f9ed8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 shutdown_connections 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x560bf3bfc660 0x560bf3bfea80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bf3b7dad0 0x560bf3b99490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bf3b77d50 0x560bf3b98a10 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 --2- 172.21.15.112:0/2712840637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560bf3b7d1b0 0x560bf3b98f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 >> 172.21.15.112:0/2712840637 conn(0x560bf3aa3b70 msgr2=0x560bf3b7bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ec3fff700 0 client.17082 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 shutdown_connections 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.114+0000 7f9ef532fb80 1 -- 172.21.15.112:0/2712840637 wait complete. 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9ef532fb80 20 client.17082 trim_cache size 0 max 16384 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stdout: "ee0d0c99-eda8-47b3-b8a3-105151059d89": { 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "ee0d0c99-eda8-47b3-b8a3-105151059d89", 2024-05-10T10:47:12.121 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-10T10:47:12.122 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:47:12.122 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:47:12.122 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ed40b2900 msgr2=0x7f9ed40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ed40b2900 0x7f9ed40b4dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009990 tx=0x7f9ed8005960 comp rx=0 tx=0).stop 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4103f00 msgr2=0x7f9ee4075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4103f00 0x7f9ee4075ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000d750 tx=0x7f9ed000da60 comp rx=0 tx=0).stop 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 shutdown_connections 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee41095e0 0x7f9ee4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9ed40b2900 0x7f9ed40b4dc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee4103540 0x7f9ee40757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 --2- 172.21.15.112:0/4201589832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4103f00 0x7f9ee4075ce0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 >> 172.21.15.112:0/4201589832 conn(0x7f9ee407a650 msgr2=0x7f9ee4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.115+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 shutdown_connections 2024-05-10T10:47:12.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:12.116+0000 7f9eeaa9c700 1 -- 172.21.15.112:0/4201589832 wait complete. 2024-05-10T10:47:12.994 INFO:tasks.cephfs.filesystem:scrub status for tag:ee0d0c99-eda8-47b3-b8a3-105151059d89 - {'path': '/', 'tag': 'ee0d0c99-eda8-47b3-b8a3-105151059d89', 'options': 'recursive,force'} 2024-05-10T10:47:13.142 INFO:tasks.workunit.client.0.smithi112.stdout:fallocating to largest ever: 0x6400000 2024-05-10T10:47:13.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:13 smithi112 ceph-mon[30100]: pgmap v639: 129 pgs: 129 active+clean; 70 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 885 op/s 2024-05-10T10:47:13.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:13 smithi148 ceph-mon[33568]: pgmap v639: 129 pgs: 129 active+clean; 70 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 885 op/s 2024-05-10T10:47:13.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:13 smithi195 ceph-mon[33773]: pgmap v639: 129 pgs: 129 active+clean; 70 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 885 op/s 2024-05-10T10:47:15.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:15 smithi195 ceph-mon[33773]: pgmap v640: 129 pgs: 129 active+clean; 70 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 818 op/s 2024-05-10T10:47:15.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:15 smithi148 ceph-mon[33568]: pgmap v640: 129 pgs: 129 active+clean; 70 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 818 op/s 2024-05-10T10:47:15.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:15 smithi112 ceph-mon[30100]: pgmap v640: 129 pgs: 129 active+clean; 70 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 818 op/s 2024-05-10T10:47:17.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:17 smithi148 ceph-mon[33568]: pgmap v641: 129 pgs: 129 active+clean; 61 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.7 MiB/s wr, 1.00k op/s 2024-05-10T10:47:17.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:17 smithi195 ceph-mon[33773]: pgmap v641: 129 pgs: 129 active+clean; 61 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.7 MiB/s wr, 1.00k op/s 2024-05-10T10:47:17.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:17 smithi112 ceph-mon[30100]: pgmap v641: 129 pgs: 129 active+clean; 61 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.7 MiB/s wr, 1.00k op/s 2024-05-10T10:47:19.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:19 smithi148 ceph-mon[33568]: pgmap v642: 129 pgs: 129 active+clean; 62 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 857 op/s 2024-05-10T10:47:19.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:19 smithi195 ceph-mon[33773]: pgmap v642: 129 pgs: 129 active+clean; 62 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 857 op/s 2024-05-10T10:47:19.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:19 smithi112 ceph-mon[30100]: pgmap v642: 129 pgs: 129 active+clean; 62 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 857 op/s 2024-05-10T10:47:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:20 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:47:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:20 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:47:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:20 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:47:20.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:20 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:47:20.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:20 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:47:20.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:20 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:47:20.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:20 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:47:20.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:20 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:47:20.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:20 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:47:20.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:20 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:47:20.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:20 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:47:20.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:20 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:47:21.445 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:21 smithi148 ceph-mon[33568]: pgmap v643: 129 pgs: 129 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 868 op/s 2024-05-10T10:47:21.445 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:21 smithi148 ceph-mon[33568]: scrub summary: idle 2024-05-10T10:47:21.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:21 smithi195 ceph-mon[33773]: pgmap v643: 129 pgs: 129 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 868 op/s 2024-05-10T10:47:21.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:21 smithi195 ceph-mon[33773]: scrub summary: idle 2024-05-10T10:47:21.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:21 smithi112 ceph-mon[30100]: pgmap v643: 129 pgs: 129 active+clean; 67 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 868 op/s 2024-05-10T10:47:21.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:21 smithi112 ceph-mon[30100]: scrub summary: idle 2024-05-10T10:47:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.865+0000 7f608d72b640 1 -- 172.21.15.112:0/1273063336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 msgr2=0x7f6088101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.865+0000 7f608d72b640 1 --2- 172.21.15.112:0/1273063336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f6088101fc0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6070009900 tx=0x7f607002a400 comp rx=0 tx=0).stop 2024-05-10T10:47:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.865+0000 7f608d72b640 1 -- 172.21.15.112:0/1273063336 shutdown_connections 2024-05-10T10:47:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.865+0000 7f608d72b640 1 --2- 172.21.15.112:0/1273063336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6088102500 0x7f6088109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.865+0000 7f608d72b640 1 --2- 172.21.15.112:0/1273063336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f6088101fc0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.865+0000 7f608d72b640 1 --2- 172.21.15.112:0/1273063336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6088102f80 0x7f60881033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.865+0000 7f608d72b640 1 -- 172.21.15.112:0/1273063336 >> 172.21.15.112:0/1273063336 conn(0x7f60880fc610 msgr2=0x7f60880fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.866+0000 7f608d72b640 1 -- 172.21.15.112:0/1273063336 shutdown_connections 2024-05-10T10:47:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.866+0000 7f608d72b640 1 -- 172.21.15.112:0/1273063336 wait complete. 2024-05-10T10:47:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.867+0000 7f608d72b640 1 Processor -- start 2024-05-10T10:47:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.870+0000 7f608d72b640 1 -- start start 2024-05-10T10:47:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.870+0000 7f608d72b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f60881015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.870+0000 7f608d72b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6088102500 0x7f6088073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.870+0000 7f608d72b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6088102f80 0x7f60880736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.870+0000 7f608d72b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f608810d2d0 con 0x7f6088101b40 2024-05-10T10:47:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.870+0000 7f608d72b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f608810d150 con 0x7f6088102500 2024-05-10T10:47:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.870+0000 7f608d72b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f608810d450 con 0x7f6088102f80 2024-05-10T10:47:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f60877fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6088102f80 0x7f60880736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f60881015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f60881015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35548/0 (socket says 172.21.15.112:35548) 2024-05-10T10:47:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 -- 172.21.15.112:0/4045272058 learned_addr learned my addr 172.21.15.112:0/4045272058 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f607e7fc640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6088102500 0x7f6088073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 -- 172.21.15.112:0/4045272058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6088102f80 msgr2=0x7f60880736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6088102f80 0x7f60880736d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 -- 172.21.15.112:0/4045272058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6088102500 msgr2=0x7f6088073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6088102500 0x7f6088073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f6086ffd640 1 -- 172.21.15.112:0/4045272058 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60881a7d60 con 0x7f6088101b40 2024-05-10T10:47:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.873+0000 7f607e7fc640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6088102500 0x7f6088073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.874+0000 7f6086ffd640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f60881015b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f607800d950 tx=0x7f607800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.874+0000 7f6084ff9640 1 -- 172.21.15.112:0/4045272058 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078017070 con 0x7f6088101b40 2024-05-10T10:47:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.874+0000 7f6084ff9640 1 -- 172.21.15.112:0/4045272058 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6078009c30 con 0x7f6088101b40 2024-05-10T10:47:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.875+0000 7f6084ff9640 1 -- 172.21.15.112:0/4045272058 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607800f620 con 0x7f6088101b40 2024-05-10T10:47:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.875+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60881a8050 con 0x7f6088101b40 2024-05-10T10:47:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.875+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60881a8520 con 0x7f6088101b40 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.879+0000 7f609044a740 1 Processor -- start 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.879+0000 7f609044a740 1 -- start start 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.879+0000 7f609044a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.879+0000 7f609044a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0af7a0 0x55c82f0b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f609044a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 0x55c82f0b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f609044a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c82f0135d0 con 0x55c82f0b4910 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f609044a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c82efcc890 con 0x55c82f0af7a0 2024-05-10T10:47:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f609044a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c82f00f590 con 0x55c82f0aedb0 2024-05-10T10:47:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:57122/0 (socket says 172.21.15.112:57122) 2024-05-10T10:47:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 -- 172.21.15.112:0/2502456087 learned_addr learned my addr 172.21.15.112:0/2502456087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 -- 172.21.15.112:0/2502456087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0af7a0 msgr2=0x55c82f0b42a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T10:47:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 --2- 172.21.15.112:0/2502456087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0af7a0 0x55c82f0b42a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 -- 172.21.15.112:0/2502456087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 msgr2=0x55c82f0b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 --2- 172.21.15.112:0/2502456087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 0x55c82f0b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.880+0000 7f60877fe640 1 -- 172.21.15.112:0/2502456087 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82ed25210 con 0x55c82f0aedb0 2024-05-10T10:47:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.881+0000 7f60877fe640 1 --2- 172.21.15.112:0/2502456087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0af1d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6080009eb0 tx=0x7f608000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d5dee828e42755 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.882+0000 7f607d7fa640 1 -- 172.21.15.112:0/2502456087 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080011640 con 0x55c82f0aedb0 2024-05-10T10:47:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.882+0000 7f6084ff9640 1 -- 172.21.15.112:0/4045272058 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f607800f7c0 con 0x7f6088101b40 2024-05-10T10:47:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.882+0000 7f607d7fa640 1 -- 172.21.15.112:0/2502456087 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6080011c70 con 0x55c82f0aedb0 2024-05-10T10:47:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.882+0000 7f607d7fa640 1 -- 172.21.15.112:0/2502456087 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080010bf0 con 0x55c82f0aedb0 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f6084ff9640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60740b2a70 0x7f60740b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 -- 172.21.15.112:0/2502456087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 msgr2=0x55c82f0af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 --2- 172.21.15.112:0/2502456087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0af1d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6080009eb0 tx=0x7f608000f730 comp rx=0 tx=0).stop 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 -- 172.21.15.112:0/2502456087 shutdown_connections 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 --2- 172.21.15.112:0/2502456087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 0x55c82f0b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 --2- 172.21.15.112:0/2502456087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0af7a0 0x55c82f0b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 --2- 172.21.15.112:0/2502456087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0af1d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 -- 172.21.15.112:0/2502456087 >> 172.21.15.112:0/2502456087 conn(0x55c82f0aa420 msgr2=0x55c82f0aa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f6084ff9640 1 -- 172.21.15.112:0/4045272058 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f60780e2690 con 0x7f6088101b40 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f6086ffd640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60740b2a70 0x7f60740b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.883+0000 7f609044a740 1 -- 172.21.15.112:0/2502456087 shutdown_connections 2024-05-10T10:47:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 -- 172.21.15.112:0/2502456087 wait complete. 2024-05-10T10:47:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 -- start start 2024-05-10T10:47:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f6086ffd640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60740b2a70 0x7f60740b4f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6078000c00 tx=0x7f60780a5040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 Processor -- start 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0aedb0 0x55c82f0b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 0x55c82f0c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c82f0b7960 con 0x55c82f0b4910 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c82efd0370 con 0x55c82f0aedb0 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.884+0000 7f609044a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c82f0b7ae0 con 0x55c82f0af7a0 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f609044a740 10 client.0 _ll_register_callbacks cb 0x55c82ed24a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:57134/0 (socket says 172.21.15.112:57134) 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 -- 172.21.15.112:0/2926859523 learned_addr learned my addr 172.21.15.112:0/2926859523 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f60877fe640 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0aedb0 0x55c82f0b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f60877fe640 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 0x55c82f0c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 -- 172.21.15.112:0/2926859523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0aedb0 msgr2=0x55c82f0b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0aedb0 0x55c82f0b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 -- 172.21.15.112:0/2926859523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 msgr2=0x55c82f0c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 0x55c82f0c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.885+0000 7f607e7fc640 1 -- 172.21.15.112:0/2926859523 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82edf1620 con 0x55c82f0af7a0 2024-05-10T10:47:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.886+0000 7f607e7fc640 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b95a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6088102c10 tx=0x7f6070031730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad51585739bacd1e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.886+0000 7f606e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:47:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.886+0000 7f606e7fc640 1 -- 172.21.15.112:0/2926859523 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070035030 con 0x55c82f0af7a0 2024-05-10T10:47:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.886+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6080011c70 con 0x55c82f0af7a0 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.886+0000 7f606e7fc640 1 -- 172.21.15.112:0/2926859523 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6070004040 con 0x55c82f0af7a0 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.886+0000 7f606e7fc640 1 -- 172.21.15.112:0/2926859523 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f607004d050 con 0x55c82f0af7a0 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.887+0000 7f609044a740 10 client.26446 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.887+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c82f0c3660 con 0x55c82f0af7a0 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.887+0000 7f606e7fc640 1 -- 172.21.15.112:0/2926859523 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6080011c70 con 0x55c82f0af7a0 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.888+0000 7f606e7fc640 1 -- 172.21.15.112:0/2926859523 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f6070003c10 con 0x55c82f0af7a0 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.888+0000 7f609044a740 10 client.26446 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.888+0000 7f609044a740 10 client.26446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.888+0000 7f609044a740 20 client.26446 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.888+0000 7f609044a740 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f0c5210 0x55c82f0c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.888+0000 7f609044a740 4 client.26446 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.888+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c82f0211e0 con 0x55c82f0c5210 2024-05-10T10:47:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.889+0000 7f60877fe640 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f0c5210 0x55c82f0c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.889+0000 7f60877fe640 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f0c5210 0x55c82f0c7630 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0bdf0d0c93ca7f server_cookie=fbfdeefdc8b6e40a in_seq=0 out_seq=0 2024-05-10T10:47:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.889+0000 7f606e7fc640 10 client.26446 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.890+0000 7f606e7fc640 1 -- 172.21.15.112:0/2926859523 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c82f0211e0 con 0x55c82f0c5210 2024-05-10T10:47:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.890+0000 7f606e7fc640 10 client.26446 handle_command_reply: tid=0 2024-05-10T10:47:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.890+0000 7f609044a740 1 client.26446 shutdown 2024-05-10T10:47:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.890+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 msgr2=0x55c82f0b95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.890+0000 7f609044a740 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b95a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6088102c10 tx=0x7f6070031730 comp rx=0 tx=0).stop 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 shutdown_connections 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0b4910 0x55c82f0c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b95a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f0c5210 0x55c82f0c7630 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 --2- 172.21.15.112:0/2926859523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0aedb0 0x55c82f0b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 >> 172.21.15.112:0/2926859523 conn(0x55c82f0aa420 msgr2=0x55c82f0a9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f606e7fc640 0 client.26446 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 shutdown_connections 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 1 -- 172.21.15.112:0/2926859523 wait complete. 2024-05-10T10:47:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.891+0000 7f609044a740 20 client.26446 trim_cache size 0 max 16384 2024-05-10T10:47:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.907+0000 7f609044a740 1 Processor -- start 2024-05-10T10:47:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.907+0000 7f609044a740 1 -- start start 2024-05-10T10:47:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.907+0000 7f609044a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0aedb0 0x55c82f0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.907+0000 7f609044a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.907+0000 7f607e7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.907+0000 7f607e7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:57150/0 (socket says 172.21.15.112:57150) 2024-05-10T10:47:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.907+0000 7f609044a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f607e7fc640 1 -- 172.21.15.112:0/2823134204 learned_addr learned my addr 172.21.15.112:0/2823134204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f60877fe640 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0aedb0 0x55c82f0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f606800fc90 con 0x55c82f0aedb0 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60680117d0 con 0x55c82f0b4910 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6068010360 con 0x55c82f0af7a0 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f607e7fc640 1 -- 172.21.15.112:0/2823134204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 msgr2=0x55c82f0c05a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f607e7fc640 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0c05a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f607e7fc640 1 -- 172.21.15.112:0/2823134204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0aedb0 msgr2=0x55c82f0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f607e7fc640 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0aedb0 0x55c82f0b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.908+0000 7f607e7fc640 1 -- 172.21.15.112:0/2823134204 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82f0a9cd0 con 0x55c82f0af7a0 2024-05-10T10:47:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.909+0000 7f607e7fc640 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b0ec0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6070005c40 tx=0x7f6070005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f024b589b3859eb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.909+0000 7f607cff9640 1 -- 172.21.15.112:0/2823134204 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070035030 con 0x55c82f0af7a0 2024-05-10T10:47:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.909+0000 7f607cff9640 1 -- 172.21.15.112:0/2823134204 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f607003c730 con 0x55c82f0af7a0 2024-05-10T10:47:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.909+0000 7f607cff9640 1 -- 172.21.15.112:0/2823134204 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60700035f0 con 0x55c82f0af7a0 2024-05-10T10:47:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.910+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 msgr2=0x55c82f0b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.910+0000 7f609044a740 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b0ec0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6070005c40 tx=0x7f6070005a20 comp rx=0 tx=0).stop 2024-05-10T10:47:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.911+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 shutdown_connections 2024-05-10T10:47:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.911+0000 7f609044a740 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.911+0000 7f609044a740 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0af7a0 0x55c82f0b0ec0 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6070005c40 tx=0x7f6070005a20 comp rx=0 tx=0).stop 2024-05-10T10:47:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.911+0000 7f609044a740 1 --2- 172.21.15.112:0/2823134204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0aedb0 0x55c82f0b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.911+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 >> 172.21.15.112:0/2823134204 conn(0x55c82f0aa420 msgr2=0x55c82f0a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.912+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 shutdown_connections 2024-05-10T10:47:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.913+0000 7f609044a740 1 -- 172.21.15.112:0/2823134204 wait complete. 2024-05-10T10:47:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.913+0000 7f609044a740 1 -- start start 2024-05-10T10:47:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.913+0000 7f609044a740 1 Processor -- start 2024-05-10T10:47:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f609044a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f609044a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f609044a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0c07f0 0x55c82f12dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f60877fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f60877fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0bd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:57162/0 (socket says 172.21.15.112:57162) 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f60877fe640 1 -- 172.21.15.112:0/3688034338 learned_addr learned my addr 172.21.15.112:0/3688034338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f607e7fc640 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6068011100 con 0x55c82f0c07f0 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6068010a30 con 0x55c82f0b4910 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.914+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f606800dc70 con 0x55c82f0aedb0 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f609044a740 10 client.0 _ll_register_callbacks cb 0x7f60800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f60877fe640 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0c07f0 0x55c82f12dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f607e7fc640 1 -- 172.21.15.112:0/3688034338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 msgr2=0x55c82f0bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f607e7fc640 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0bd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f607e7fc640 1 -- 172.21.15.112:0/3688034338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0c07f0 msgr2=0x55c82f12dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f607e7fc640 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0c07f0 0x55c82f12dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f607e7fc640 1 -- 172.21.15.112:0/3688034338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82f12e0d0 con 0x55c82f0b4910 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f607e7fc640 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0bd7d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c82f0b00c0 tx=0x7f6070005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4bf8944229f366e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f606e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f606e7fc640 1 -- 172.21.15.112:0/3688034338 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607005d020 con 0x55c82f0b4910 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.915+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f607003c730 con 0x55c82f0b4910 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f606e7fc640 1 -- 172.21.15.112:0/3688034338 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f607003ca30 con 0x55c82f0b4910 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f606e7fc640 1 -- 172.21.15.112:0/3688034338 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6070065050 con 0x55c82f0b4910 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f609044a740 10 client.26441 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c82f12f3a0 con 0x55c82f0b4910 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f606e7fc640 1 -- 172.21.15.112:0/3688034338 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f607003c730 con 0x55c82f0b4910 2024-05-10T10:47:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f606e7fc640 1 -- 172.21.15.112:0/3688034338 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f6070051430 con 0x55c82f0b4910 2024-05-10T10:47:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f609044a740 10 client.26441 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f609044a740 10 client.26441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f609044a740 20 client.26441 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.916+0000 7f609044a740 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f12fc20 0x55c82f132040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.917+0000 7f609044a740 4 client.26441 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:47:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.917+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c82f01b050 con 0x55c82f12fc20 2024-05-10T10:47:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.917+0000 7f60877fe640 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f12fc20 0x55c82f132040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.917+0000 7f60877fe640 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f12fc20 0x55c82f132040 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d448bcba9823e4 server_cookie=e0be8088f2dab99f in_seq=0 out_seq=0 2024-05-10T10:47:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.917+0000 7f606e7fc640 10 client.26441 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f606e7fc640 1 -- 172.21.15.112:0/3688034338 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c82f01b050 con 0x55c82f12fc20 2024-05-10T10:47:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f606e7fc640 10 client.26441 handle_command_reply: tid=0 2024-05-10T10:47:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f609044a740 1 client.26441 shutdown 2024-05-10T10:47:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 msgr2=0x55c82f0bd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f609044a740 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0bd7d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c82f0b00c0 tx=0x7f6070005e60 comp rx=0 tx=0).stop 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 shutdown_connections 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f609044a740 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82f0c07f0 0x55c82f12dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f609044a740 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55c82f12fc20 0x55c82f132040 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f609044a740 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c82f0b4910 0x55c82f0bd7d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.935+0000 7f606e7fc640 0 client.26441 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.936+0000 7f609044a740 1 --2- 172.21.15.112:0/3688034338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c82f0aedb0 0x55c82f0bd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.936+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 >> 172.21.15.112:0/3688034338 conn(0x55c82f0aa420 msgr2=0x55c82f0b8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.936+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 shutdown_connections 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.936+0000 7f609044a740 1 -- 172.21.15.112:0/3688034338 wait complete. 2024-05-10T10:47:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.936+0000 7f609044a740 20 client.26441 trim_cache size 0 max 16384 2024-05-10T10:47:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:47:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:47:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:47:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:47:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.937+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60740b2a70 msgr2=0x7f60740b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.937+0000 7f608d72b640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60740b2a70 0x7f60740b4f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6078000c00 tx=0x7f60780a5040 comp rx=0 tx=0).stop 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.937+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 msgr2=0x7f60881015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.937+0000 7f608d72b640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f60881015b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f607800d950 tx=0x7f607800de10 comp rx=0 tx=0).stop 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 shutdown_connections 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60740b2a70 0x7f60740b4f30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6088102f80 0x7f60880736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6088102500 0x7f6088073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 --2- 172.21.15.112:0/4045272058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6088101b40 0x7f60881015b0 secure :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f607800d950 tx=0x7f607800de10 comp rx=0 tx=0).stop 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 >> 172.21.15.112:0/4045272058 conn(0x7f60880fc610 msgr2=0x7f6088106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 shutdown_connections 2024-05-10T10:47:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:21.938+0000 7f608d72b640 1 -- 172.21.15.112:0/4045272058 wait complete. 2024-05-10T10:47:23.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:23 smithi148 ceph-mon[33568]: pgmap v644: 129 pgs: 129 active+clean; 76 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 1.00k op/s 2024-05-10T10:47:23.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:23 smithi195 ceph-mon[33773]: pgmap v644: 129 pgs: 129 active+clean; 76 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 1.00k op/s 2024-05-10T10:47:23.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:23 smithi112 ceph-mon[30100]: pgmap v644: 129 pgs: 129 active+clean; 76 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 1.00k op/s 2024-05-10T10:47:24.873 INFO:tasks.workunit.client.0.smithi112.stdout:40000 falloc from 0x4cbeead to 0x4cc78f9 (0x8a4c bytes) 2024-05-10T10:47:25.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:25 smithi148 ceph-mon[33568]: pgmap v645: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 602 op/s 2024-05-10T10:47:25.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:25 smithi195 ceph-mon[33773]: pgmap v645: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 602 op/s 2024-05-10T10:47:25.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:25 smithi112 ceph-mon[30100]: pgmap v645: 129 pgs: 129 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 602 op/s 2024-05-10T10:47:27.381 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:27 smithi112 ceph-mon[30100]: pgmap v646: 129 pgs: 129 active+clean; 70 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 712 op/s 2024-05-10T10:47:27.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:27 smithi148 ceph-mon[33568]: pgmap v646: 129 pgs: 129 active+clean; 70 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 712 op/s 2024-05-10T10:47:27.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:27 smithi195 ceph-mon[33773]: pgmap v646: 129 pgs: 129 active+clean; 70 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 712 op/s 2024-05-10T10:47:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:29 smithi148 ceph-mon[33568]: pgmap v647: 129 pgs: 129 active+clean; 62 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 519 op/s 2024-05-10T10:47:29.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:29 smithi195 ceph-mon[33773]: pgmap v647: 129 pgs: 129 active+clean; 62 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 519 op/s 2024-05-10T10:47:29.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:29 smithi112 ceph-mon[30100]: pgmap v647: 129 pgs: 129 active+clean; 62 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 519 op/s 2024-05-10T10:47:31.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:31 smithi148 ceph-mon[33568]: pgmap v648: 129 pgs: 129 active+clean; 58 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 533 op/s 2024-05-10T10:47:31.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:31 smithi195 ceph-mon[33773]: pgmap v648: 129 pgs: 129 active+clean; 58 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 533 op/s 2024-05-10T10:47:31.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:31 smithi112 ceph-mon[30100]: pgmap v648: 129 pgs: 129 active+clean; 58 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 533 op/s 2024-05-10T10:47:33.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:33 smithi148 ceph-mon[33568]: pgmap v649: 129 pgs: 129 active+clean; 73 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 682 op/s 2024-05-10T10:47:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:33 smithi195 ceph-mon[33773]: pgmap v649: 129 pgs: 129 active+clean; 73 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 682 op/s 2024-05-10T10:47:33.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:33 smithi112 ceph-mon[30100]: pgmap v649: 129 pgs: 129 active+clean; 73 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 682 op/s 2024-05-10T10:47:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 -- 172.21.15.112:0/4278058771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 msgr2=0x7f0c24104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 --2- 172.21.15.112:0/4278058771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 0x7f0c24104090 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009900 tx=0x7f0c1802a410 comp rx=0 tx=0).stop 2024-05-10T10:47:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 -- 172.21.15.112:0/4278058771 shutdown_connections 2024-05-10T10:47:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 --2- 172.21.15.112:0/4278058771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c241045d0 0x7f0c2410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 --2- 172.21.15.112:0/4278058771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 0x7f0c24104090 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 --2- 172.21.15.112:0/4278058771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 0x7f0c24102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 -- 172.21.15.112:0/4278058771 >> 172.21.15.112:0/4278058771 conn(0x7f0c240fdf70 msgr2=0x7f0c241003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 -- 172.21.15.112:0/4278058771 shutdown_connections 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 -- 172.21.15.112:0/4278058771 wait complete. 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.056+0000 7f0c29390640 1 Processor -- start 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c29390640 1 -- start start 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c29390640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 0x7f0c24198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c29390640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 0x7f0c24199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c29390640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c241045d0 0x7f0c241a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c29390640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410e430 con 0x7f0c24103c10 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c29390640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410e2b0 con 0x7f0c241045d0 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c29390640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410e5b0 con 0x7f0c24102a20 2024-05-10T10:47:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 0x7f0c24198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 0x7f0c24198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:47184/0 (socket says 172.21.15.112:47184) 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3544275693 learned_addr learned my addr 172.21.15.112:0/3544275693 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c227fc640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 0x7f0c24199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c237fe640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c241045d0 0x7f0c241a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3544275693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c241045d0 msgr2=0x7f0c241a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c241045d0 0x7f0c241a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 msgr2=0x7f0c24199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 0x7f0c24199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.057+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3544275693 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c241a0700 con 0x7f0c24102a20 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.058+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 0x7f0c24198ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400e9c0 tx=0x7f0c1400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.058+0000 7f0c0bfff640 1 -- 172.21.15.112:0/3544275693 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400cd80 con 0x7f0c24102a20 2024-05-10T10:47:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.058+0000 7f0c0bfff640 1 -- 172.21.15.112:0/3544275693 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c140044b0 con 0x7f0c24102a20 2024-05-10T10:47:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.058+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c241a09f0 con 0x7f0c24102a20 2024-05-10T10:47:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.058+0000 7f0c0bfff640 1 -- 172.21.15.112:0/3544275693 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14010430 con 0x7f0c24102a20 2024-05-10T10:47:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.058+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241a0f10 con 0x7f0c24102a20 2024-05-10T10:47:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.060+0000 7f0c0bfff640 1 -- 172.21.15.112:0/3544275693 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0c14003f00 con 0x7f0c24102a20 2024-05-10T10:47:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 Processor -- start 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 -- start start 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f479661910 0x55f479663da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4795c05d0 con 0x55f479661910 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f479579890 con 0x55f47965bdb0 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c2c187740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4795bc590 con 0x55f47965c7a0 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c0bfff640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c0c0b28a0 0x7f0c0c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f479661910 0x55f479663da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c22ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f479661910 0x55f479663da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42052/0 (socket says 172.21.15.112:42052) 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 -- 172.21.15.112:0/1265470007 learned_addr learned my addr 172.21.15.112:0/1265470007 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c237fe640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c0c0b28a0 0x7f0c0c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c237fe640 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 -- 172.21.15.112:0/1265470007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 msgr2=0x55f4796612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 -- 172.21.15.112:0/1265470007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 msgr2=0x55f47965c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c227fc640 1 -- 172.21.15.112:0/1265470007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4792d2210 con 0x55f479661910 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c237fe640 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.061+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.062+0000 7f0c0bfff640 1 -- 172.21.15.112:0/3544275693 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c140e2990 con 0x7f0c24102a20 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.062+0000 7f0c237fe640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c0c0b28a0 0x7f0c0c0b4d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55f47965d000 tx=0x7f0c1c009bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.062+0000 7f0c227fc640 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f479661910 0x55f479663da0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0c18006010 tx=0x7f0c18004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db6c788fef21eea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.062+0000 7f0c097fa640 1 -- 172.21.15.112:0/1265470007 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18035030 con 0x55f479661910 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.062+0000 7f0c097fa640 1 -- 172.21.15.112:0/1265470007 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c18040070 con 0x55f479661910 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.062+0000 7f0c097fa640 1 -- 172.21.15.112:0/1265470007 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18031760 con 0x55f479661910 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 -- 172.21.15.112:0/1265470007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f479661910 msgr2=0x55f479663da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f479661910 0x55f479663da0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0c18006010 tx=0x7f0c18004040 comp rx=0 tx=0).stop 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 -- 172.21.15.112:0/1265470007 shutdown_connections 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f479661910 0x55f479663da0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 --2- 172.21.15.112:0/1265470007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 -- 172.21.15.112:0/1265470007 >> 172.21.15.112:0/1265470007 conn(0x55f479657420 msgr2=0x55f479657820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 -- 172.21.15.112:0/1265470007 shutdown_connections 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 -- 172.21.15.112:0/1265470007 wait complete. 2024-05-10T10:47:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 -- start start 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.063+0000 7f0c2c187740 1 Processor -- start 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f479665fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f479661910 0x55f47966d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f479664860 con 0x55f47965bdb0 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f47957d370 con 0x55f479661910 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4796649e0 con 0x55f47965c7a0 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f479665fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f479665fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42064/0 (socket says 172.21.15.112:42064) 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 10 client.0 _ll_register_callbacks cb 0x55f4792d1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c22ffd640 1 -- 172.21.15.112:0/2363456059 learned_addr learned my addr 172.21.15.112:0/2363456059 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c227fc640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c237fe640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f479661910 0x55f47966d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c227fc640 1 -- 172.21.15.112:0/2363456059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f479661910 msgr2=0x55f47966d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c227fc640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f479661910 0x55f47966d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c227fc640 1 -- 172.21.15.112:0/2363456059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 msgr2=0x55f479665fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c227fc640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f479665fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c227fc640 1 -- 172.21.15.112:0/2363456059 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47939e620 con 0x55f47965c7a0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c237fe640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f479661910 0x55f47966d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f479665fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c227fc640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796664e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005ab0 tx=0x7f0c18004460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cefe679e0493327c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c06ff5640 1 -- 172.21.15.112:0/2363456059 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18053060 con 0x55f47965c7a0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c06ff5640 1 -- 172.21.15.112:0/2363456059 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c18003a00 con 0x55f47965c7a0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.064+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c18040070 con 0x55f47965c7a0 2024-05-10T10:47:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c06ff5640 1 -- 172.21.15.112:0/2363456059 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0c1805a050 con 0x55f47965c7a0 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 10 client.26464 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4796648b0 con 0x55f47965c7a0 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c06ff5640 1 -- 172.21.15.112:0/2363456059 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c18040070 con 0x55f47965c7a0 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c06ff5640 1 -- 172.21.15.112:0/2363456059 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f0c1804d030 con 0x55f47965c7a0 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 10 client.26464 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 20 client.26464 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f479671780 0x55f479673b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 4 client.26464 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4795ce1e0 con 0x55f479671780 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.065+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f479671780 0x55f479673b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c06ff5640 10 client.26464 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f479671780 0x55f479673b70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3db4ffd175b2ac server_cookie=cf2dc09bcccce619 in_seq=0 out_seq=0 2024-05-10T10:47:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c06ff5640 1 -- 172.21.15.112:0/2363456059 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4795ce1e0 con 0x55f479671780 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c06ff5640 10 client.26464 handle_command_reply: tid=0 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 client.26464 shutdown 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 msgr2=0x55f4796664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796664e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005ab0 tx=0x7f0c18004460 comp rx=0 tx=0).stop 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 shutdown_connections 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f479671780 0x55f479673b70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f479661910 0x55f47966d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f47965c7a0 0x55f4796664e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c06ff5640 0 client.26464 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 --2- 172.21.15.112:0/2363456059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f479665fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.066+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 >> 172.21.15.112:0/2363456059 conn(0x55f479657420 msgr2=0x55f479656580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.067+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 shutdown_connections 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.067+0000 7f0c2c187740 1 -- 172.21.15.112:0/2363456059 wait complete. 2024-05-10T10:47:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.067+0000 7f0c2c187740 20 client.26464 trim_cache size 0 max 16384 2024-05-10T10:47:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.083+0000 7f0c2c187740 1 Processor -- start 2024-05-10T10:47:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.083+0000 7f0c2c187740 1 -- start start 2024-05-10T10:47:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965c7a0 0x55f47965a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f479678450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c2c187740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241ab350 con 0x55f47965c7a0 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c2c187740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241ace90 con 0x55f47965bdb0 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c2c187740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241aba20 con 0x55f479661910 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965c7a0 0x55f47965a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965c7a0 0x55f47965a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42072/0 (socket says 172.21.15.112:42072) 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 -- 172.21.15.112:0/855654643 learned_addr learned my addr 172.21.15.112:0/855654643 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c237fe640 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f479678450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 -- 172.21.15.112:0/855654643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 msgr2=0x55f479678450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f479678450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 -- 172.21.15.112:0/855654643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 msgr2=0x55f47965a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 -- 172.21.15.112:0/855654643 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f479673ee0 con 0x55f47965c7a0 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c227fc640 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965c7a0 0x55f47965a840 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0c1802a450 tx=0x7f0c18005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2681bd238583b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c08ff9640 1 -- 172.21.15.112:0/855654643 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18053040 con 0x55f47965c7a0 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c08ff9640 1 -- 172.21.15.112:0/855654643 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c18040070 con 0x55f47965c7a0 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.084+0000 7f0c08ff9640 1 -- 172.21.15.112:0/855654643 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18035070 con 0x55f47965c7a0 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 -- 172.21.15.112:0/855654643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965c7a0 msgr2=0x55f47965a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965c7a0 0x55f47965a840 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0c1802a450 tx=0x7f0c18005a20 comp rx=0 tx=0).stop 2024-05-10T10:47:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 -- 172.21.15.112:0/855654643 shutdown_connections 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f479678450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965c7a0 0x55f47965a840 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 --2- 172.21.15.112:0/855654643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965bdb0 0x55f47965a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 -- 172.21.15.112:0/855654643 >> 172.21.15.112:0/855654643 conn(0x55f479657420 msgr2=0x55f47965d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 -- 172.21.15.112:0/855654643 shutdown_connections 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 -- 172.21.15.112:0/855654643 wait complete. 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.085+0000 7f0c2c187740 1 -- start start 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 Processor -- start 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f47966a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965c7a0 0x55f47966a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f47966b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241ac7c0 con 0x55f47965bdb0 2024-05-10T10:47:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241ac0f0 con 0x55f47965c7a0 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241a9330 con 0x55f479661910 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 10 client.0 _ll_register_callbacks cb 0x7f0c18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f47966a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f47966a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42078/0 (socket says 172.21.15.112:42078) 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3296603796 learned_addr learned my addr 172.21.15.112:0/3296603796 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c227fc640 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965c7a0 0x55f47966a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c237fe640 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f47966b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 msgr2=0x55f47966b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f47966b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3296603796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965c7a0 msgr2=0x55f47966a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965c7a0 0x55f47966a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 -- 172.21.15.112:0/3296603796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47966be60 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c22ffd640 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f47966a320 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0c140a61c0 tx=0x7f0c140a5430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef9f38d1be399e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c06ff5640 1 -- 172.21.15.112:0/3296603796 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140a9730 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c06ff5640 1 -- 172.21.15.112:0/3296603796 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0c140a8e80 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c18040070 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c06ff5640 1 -- 172.21.15.112:0/3296603796 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0c140aa320 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.086+0000 7f0c2c187740 10 client.17115 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4796dc0e0 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c06ff5640 1 -- 172.21.15.112:0/3296603796 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c18040070 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c06ff5640 1 -- 172.21.15.112:0/3296603796 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f0c140c4140 con 0x55f47965bdb0 2024-05-10T10:47:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c2c187740 10 client.17115 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c2c187740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c2c187740 20 client.17115 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c2c187740 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f4796dca60 0x55f4796dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c2c187740 4 client.17115 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4795c8050 con 0x55f4796dca60 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c227fc640 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f4796dca60 0x55f4796dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c06ff5640 10 client.17115 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.087+0000 7f0c227fc640 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f4796dca60 0x55f4796dee80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3ab28f5059dd53 server_cookie=4ad7101eaf6b97ae in_seq=0 out_seq=0 2024-05-10T10:47:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c06ff5640 1 -- 172.21.15.112:0/3296603796 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4795c8050 con 0x55f4796dca60 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c06ff5640 10 client.17115 handle_command_reply: tid=0 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 client.17115 shutdown 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 msgr2=0x55f47966a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f47966a320 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0c140a61c0 tx=0x7f0c140a5430 comp rx=0 tx=0).stop 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 shutdown_connections 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f479661910 0x55f47966b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x55f4796dca60 0x55f4796dee80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c06ff5640 0 client.17115 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f47965c7a0 0x55f47966a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 --2- 172.21.15.112:0/3296603796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f47965bdb0 0x55f47966a320 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 >> 172.21.15.112:0/3296603796 conn(0x55f479657420 msgr2=0x55f47965a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 shutdown_connections 2024-05-10T10:47:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 1 -- 172.21.15.112:0/3296603796 wait complete. 2024-05-10T10:47:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.112+0000 7f0c2c187740 20 client.17115 trim_cache size 0 max 16384 2024-05-10T10:47:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:47:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:47:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:47:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:47:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c0c0b28a0 msgr2=0x7f0c0c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c0c0b28a0 0x7f0c0c0b4d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55f47965d000 tx=0x7f0c1c009bf0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 msgr2=0x7f0c24198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 0x7f0c24198ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400e9c0 tx=0x7f0c1400ee80 comp rx=0 tx=0).stop 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 shutdown_connections 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f0c0c0b28a0 0x7f0c0c0b4d60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c241045d0 0x7f0c241a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c24103c10 0x7f0c24199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 --2- 172.21.15.112:0/3544275693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c24102a20 0x7f0c24198ae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 >> 172.21.15.112:0/3544275693 conn(0x7f0c240fdf70 msgr2=0x7f0c24106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 shutdown_connections 2024-05-10T10:47:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:35.113+0000 7f0c29390640 1 -- 172.21.15.112:0/3544275693 wait complete. 2024-05-10T10:47:35.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:35 smithi148 ceph-mon[33568]: pgmap v650: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 521 op/s 2024-05-10T10:47:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:35 smithi195 ceph-mon[33773]: pgmap v650: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 521 op/s 2024-05-10T10:47:35.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:35 smithi112 ceph-mon[30100]: pgmap v650: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 521 op/s 2024-05-10T10:47:37.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:37 smithi148 ceph-mon[33568]: pgmap v651: 129 pgs: 129 active+clean; 67 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 700 op/s 2024-05-10T10:47:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:37 smithi195 ceph-mon[33773]: pgmap v651: 129 pgs: 129 active+clean; 67 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 700 op/s 2024-05-10T10:47:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:37 smithi112 ceph-mon[30100]: pgmap v651: 129 pgs: 129 active+clean; 67 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 700 op/s 2024-05-10T10:47:39.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:39 smithi148 ceph-mon[33568]: pgmap v652: 129 pgs: 129 active+clean; 63 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 489 op/s 2024-05-10T10:47:39.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:39 smithi195 ceph-mon[33773]: pgmap v652: 129 pgs: 129 active+clean; 63 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 489 op/s 2024-05-10T10:47:39.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:39 smithi112 ceph-mon[30100]: pgmap v652: 129 pgs: 129 active+clean; 63 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 489 op/s 2024-05-10T10:47:41.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:41 smithi148 ceph-mon[33568]: pgmap v653: 129 pgs: 129 active+clean; 58 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.3 MiB/s wr, 529 op/s 2024-05-10T10:47:41.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:41 smithi195 ceph-mon[33773]: pgmap v653: 129 pgs: 129 active+clean; 58 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.3 MiB/s wr, 529 op/s 2024-05-10T10:47:41.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:41 smithi112 ceph-mon[30100]: pgmap v653: 129 pgs: 129 active+clean; 58 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.3 MiB/s wr, 529 op/s 2024-05-10T10:47:42.996 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T10:47:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:47:43.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:43 smithi112 ceph-mon[30100]: pgmap v654: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 663 op/s 2024-05-10T10:47:43.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:43 smithi148 ceph-mon[33568]: pgmap v654: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 663 op/s 2024-05-10T10:47:43.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:43 smithi195 ceph-mon[33773]: pgmap v654: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 663 op/s 2024-05-10T10:47:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.338+0000 7f1511d4a700 1 -- 172.21.15.112:0/1866826634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c107760 msgr2=0x7f150c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.338+0000 7f1511d4a700 1 --2- 172.21.15.112:0/1866826634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c107760 0x7f150c10f2e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1504009f90 tx=0x7f150400b330 comp rx=0 tx=0).stop 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.338+0000 7f1511d4a700 1 -- 172.21.15.112:0/1866826634 shutdown_connections 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.338+0000 7f1511d4a700 1 --2- 172.21.15.112:0/1866826634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c107760 0x7f150c10f2e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.338+0000 7f1511d4a700 1 --2- 172.21.15.112:0/1866826634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f150c102e00 0x7f150c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.338+0000 7f1511d4a700 1 --2- 172.21.15.112:0/1866826634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c1037b0 0x7f150c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.338+0000 7f1511d4a700 1 -- 172.21.15.112:0/1866826634 >> 172.21.15.112:0/1866826634 conn(0x7f150c078a10 msgr2=0x7f150c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.339+0000 7f1511d4a700 1 -- 172.21.15.112:0/1866826634 shutdown_connections 2024-05-10T10:47:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.339+0000 7f1511d4a700 1 -- 172.21.15.112:0/1866826634 wait complete. 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 Processor -- start 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 -- start start 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c102e00 0x7f150c114750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f150c1037b0 0x7f150c114c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c107760 0x7f150c113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f150b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c102e00 0x7f150c114750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f150c112430 con 0x7f150c107760 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1122b0 con 0x7f150c1037b0 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f1511d4a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1125b0 con 0x7f150c102e00 2024-05-10T10:47:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f150b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c102e00 0x7f150c114750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:47164/0 (socket says 172.21.15.112:47164) 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.340+0000 7f150b7fe700 1 -- 172.21.15.112:0/4244720728 learned_addr learned my addr 172.21.15.112:0/4244720728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150affd700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f150c1037b0 0x7f150c114c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150bfff700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c107760 0x7f150c113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150b7fe700 1 -- 172.21.15.112:0/4244720728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f150c1037b0 msgr2=0x7f150c114c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150b7fe700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f150c1037b0 0x7f150c114c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150b7fe700 1 -- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c107760 msgr2=0x7f150c113260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150b7fe700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c107760 0x7f150c113260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150b7fe700 1 -- 172.21.15.112:0/4244720728 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c113b60 con 0x7f150c102e00 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f150b7fe700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c102e00 0x7f150c114750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00d950 tx=0x7f14fc00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f1508ff9700 1 -- 172.21.15.112:0/4244720728 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc016070 con 0x7f150c102e00 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f1508ff9700 1 -- 172.21.15.112:0/4244720728 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14fc009dc0 con 0x7f150c102e00 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.341+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c113e50 con 0x7f150c102e00 2024-05-10T10:47:45.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.342+0000 7f1508ff9700 1 -- 172.21.15.112:0/4244720728 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00f3c0 con 0x7f150c102e00 2024-05-10T10:47:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.342+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c1a87d0 con 0x7f150c102e00 2024-05-10T10:47:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.345+0000 7f1508ff9700 1 -- 172.21.15.112:0/4244720728 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f14fc019070 con 0x7f150c102e00 2024-05-10T10:47:45.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.346+0000 7f151c5ddb80 1 Processor -- start 2024-05-10T10:47:45.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.346+0000 7f1508ff9700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14f40b29c0 0x7f14f40b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f151c5ddb80 1 -- start start 2024-05-10T10:47:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x56448385b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x56448385c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5644837a9860 con 0x56448385bc80 2024-05-10T10:47:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5644837487c0 con 0x5644838574d0 2024-05-10T10:47:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644837a5820 con 0x564483857df0 2024-05-10T10:47:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f150bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x56448385c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f150bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x56448385c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44586/0 (socket says 172.21.15.112:44586) 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.347+0000 7f150bfff700 1 -- 172.21.15.112:0/3400515523 learned_addr learned my addr 172.21.15.112:0/3400515523 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.348+0000 7f150bfff700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14f40b29c0 0x7f14f40b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.348+0000 7f150bfff700 1 -- 172.21.15.112:0/3400515523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 msgr2=0x56448385b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.348+0000 7f150bfff700 1 --2- 172.21.15.112:0/3400515523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x56448385b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.348+0000 7f150bfff700 1 -- 172.21.15.112:0/3400515523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 msgr2=0x5644838578b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.348+0000 7f150bfff700 1 --2- 172.21.15.112:0/3400515523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838578b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.348+0000 7f150bfff700 1 -- 172.21.15.112:0/3400515523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644836f68d0 con 0x56448385bc80 2024-05-10T10:47:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.348+0000 7f150bfff700 1 --2- 172.21.15.112:0/3400515523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x56448385c100 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f15040060b0 tx=0x7f150400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d6b8341855a040 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.349+0000 7f14f1ffb700 1 -- 172.21.15.112:0/3400515523 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150402d070 con 0x56448385bc80 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.349+0000 7f14f1ffb700 1 -- 172.21.15.112:0/3400515523 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1504007b70 con 0x56448385bc80 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.349+0000 7f14f1ffb700 1 -- 172.21.15.112:0/3400515523 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150401fc90 con 0x56448385bc80 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.349+0000 7f150bfff700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14f40b29c0 0x7f14f40b4e80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1504015000 tx=0x7f1504013000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.349+0000 7f1508ff9700 1 -- 172.21.15.112:0/4244720728 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14fc0e2470 con 0x7f150c102e00 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.350+0000 7f151c5ddb80 1 -- 172.21.15.112:0/3400515523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 msgr2=0x56448385c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.350+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/3400515523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x56448385c100 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f15040060b0 tx=0x7f150400bc00 comp rx=0 tx=0).stop 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.350+0000 7f151c5ddb80 1 -- 172.21.15.112:0/3400515523 shutdown_connections 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.350+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/3400515523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x56448385c100 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.350+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/3400515523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.350+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/3400515523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x56448385b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.350+0000 7f151c5ddb80 1 -- 172.21.15.112:0/3400515523 >> 172.21.15.112:0/3400515523 conn(0x564483783100 msgr2=0x564483851c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.351+0000 7f151c5ddb80 1 -- 172.21.15.112:0/3400515523 shutdown_connections 2024-05-10T10:47:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.351+0000 7f151c5ddb80 1 -- 172.21.15.112:0/3400515523 wait complete. 2024-05-10T10:47:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.352+0000 7f151c5ddb80 1 -- start start 2024-05-10T10:47:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.352+0000 7f151c5ddb80 1 Processor -- start 2024-05-10T10:47:45.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644838574d0 0x564483863150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564483857df0 0x564483860540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483860a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564483863e30 con 0x56448385bc80 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56448370fc30 con 0x564483857df0 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564483863fb0 con 0x5644838574d0 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f150b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644838574d0 0x564483863150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f150b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644838574d0 0x564483863150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:47190/0 (socket says 172.21.15.112:47190) 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f150b7fe700 1 -- 172.21.15.112:0/2783013756 learned_addr learned my addr 172.21.15.112:0/2783013756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f150bfff700 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483860a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f150affd700 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564483857df0 0x564483860540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f151c5ddb80 10 client.0 _ll_register_callbacks cb 0x564483741920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f150b7fe700 1 -- 172.21.15.112:0/2783013756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564483857df0 msgr2=0x564483860540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.353+0000 7f150b7fe700 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564483857df0 0x564483860540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f150b7fe700 1 -- 172.21.15.112:0/2783013756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 msgr2=0x564483860a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f150b7fe700 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483860a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f150b7fe700 1 -- 172.21.15.112:0/2783013756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564483745690 con 0x5644838574d0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f150b7fe700 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644838574d0 0x564483863150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0a9630 tx=0x7f14fc0a90e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e4d5b52f831301 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f14eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f14eaffd700 1 -- 172.21.15.112:0/2783013756 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc020030 con 0x5644838574d0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1504007b70 con 0x5644838574d0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f14eaffd700 1 -- 172.21.15.112:0/2783013756 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14fc010660 con 0x5644838574d0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f14eaffd700 1 -- 172.21.15.112:0/2783013756 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f14fc010460 con 0x5644838574d0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f151c5ddb80 10 client.26479 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.354+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56448386e780 con 0x5644838574d0 2024-05-10T10:47:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f14eaffd700 1 -- 172.21.15.112:0/2783013756 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1504007b70 con 0x5644838574d0 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f14eaffd700 1 -- 172.21.15.112:0/2783013756 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f14fc0910f0 con 0x5644838574d0 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f151c5ddb80 10 client.26479 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f151c5ddb80 10 client.26479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f151c5ddb80 20 client.26479 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838710d0 0x564483873510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f151c5ddb80 4 client.26479 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.355+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564483823650 con 0x5644838710d0 2024-05-10T10:47:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.356+0000 7f150affd700 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838710d0 0x564483873510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.357+0000 7f14eaffd700 10 client.26479 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.357+0000 7f150affd700 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838710d0 0x564483873510 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7244f5cc51b87012 server_cookie=1dd53e313e77af10 in_seq=0 out_seq=0 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f14eaffd700 1 -- 172.21.15.112:0/2783013756 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564483823650 con 0x5644838710d0 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f14eaffd700 10 client.26479 handle_command_reply: tid=0 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 client.26479 shutdown 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644838574d0 msgr2=0x564483863150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644838574d0 0x564483863150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0a9630 tx=0x7f14fc0a90e0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 shutdown_connections 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838710d0 0x564483873510 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483860a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644838574d0 0x564483863150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/2783013756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564483857df0 0x564483860540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 >> 172.21.15.112:0/2783013756 conn(0x564483783100 msgr2=0x564483851c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:45.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.358+0000 7f14eaffd700 0 client.26479 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:45.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.359+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 shutdown_connections 2024-05-10T10:47:45.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.359+0000 7f151c5ddb80 1 -- 172.21.15.112:0/2783013756 wait complete. 2024-05-10T10:47:45.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.359+0000 7f151c5ddb80 20 client.26479 trim_cache size 0 max 16384 2024-05-10T10:47:45.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.390+0000 7f151c5ddb80 1 Processor -- start 2024-05-10T10:47:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.390+0000 7f151c5ddb80 1 -- start start 2024-05-10T10:47:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.391+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x56448386cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.391+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483878e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.392 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:45 smithi112 ceph-mon[30100]: pgmap v655: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 488 op/s 2024-05-10T10:47:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.391+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483879360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.391+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b1420 con 0x56448385bc80 2024-05-10T10:47:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.391+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b2f60 con 0x5644838574d0 2024-05-10T10:47:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.391+0000 7f151c5ddb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b1af0 con 0x564483857df0 2024-05-10T10:47:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150affd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483878e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483879360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150affd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483878e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:47206/0 (socket says 172.21.15.112:47206) 2024-05-10T10:47:45.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150affd700 1 -- 172.21.15.112:0/1859469741 learned_addr learned my addr 172.21.15.112:0/1859469741 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:45.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.391+0000 7f150b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x56448386cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150bfff700 1 -- 172.21.15.112:0/1859469741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 msgr2=0x564483878e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150bfff700 1 --2- 172.21.15.112:0/1859469741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483878e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150bfff700 1 -- 172.21.15.112:0/1859469741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 msgr2=0x56448386cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150bfff700 1 --2- 172.21.15.112:0/1859469741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x56448386cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150bfff700 1 -- 172.21.15.112:0/1859469741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448384a2d0 con 0x56448385bc80 2024-05-10T10:47:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.392+0000 7f150bfff700 1 --2- 172.21.15.112:0/1859469741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483879360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1504009960 tx=0x7f1504004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c83762263762926 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.393+0000 7f14f17fa700 1 -- 172.21.15.112:0/1859469741 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150402d050 con 0x56448385bc80 2024-05-10T10:47:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.393+0000 7f14f17fa700 1 -- 172.21.15.112:0/1859469741 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1504037070 con 0x56448385bc80 2024-05-10T10:47:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.393+0000 7f14f17fa700 1 -- 172.21.15.112:0/1859469741 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504029410 con 0x56448385bc80 2024-05-10T10:47:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.394+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1859469741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 msgr2=0x564483879360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.394+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1859469741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483879360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1504009960 tx=0x7f1504004ab0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.396+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1859469741 shutdown_connections 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.396+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1859469741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x564483879360 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.396+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1859469741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x56448386cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.396+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1859469741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483878e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.396+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1859469741 >> 172.21.15.112:0/1859469741 conn(0x564483783100 msgr2=0x564483856b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.396+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1859469741 shutdown_connections 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.396+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1859469741 wait complete. 2024-05-10T10:47:45.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.397+0000 7f151c5ddb80 1 -- start start 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.397+0000 7f151c5ddb80 1 Processor -- start 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.398+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.398+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483866d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.398+0000 7f150b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.398+0000 7f150b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:45856/0 (socket says 172.21.15.112:45856) 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.398+0000 7f151c5ddb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x5644838674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.398+0000 7f150b7fe700 1 -- 172.21.15.112:0/1011427719 learned_addr learned my addr 172.21.15.112:0/1011427719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.398+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b2890 con 0x56448385bc80 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1b21c0 con 0x5644838574d0 2024-05-10T10:47:45.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1af3c0 con 0x564483857df0 2024-05-10T10:47:45.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f151c5ddb80 10 client.0 _ll_register_callbacks cb 0x564483741920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:45.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f150b7fe700 1 -- 172.21.15.112:0/1011427719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 msgr2=0x564483866d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T10:47:45.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f150b7fe700 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483866d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f150b7fe700 1 -- 172.21.15.112:0/1011427719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 msgr2=0x5644838674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:45.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f150b7fe700 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x5644838674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.399+0000 7f150b7fe700 1 -- 172.21.15.112:0/1011427719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644838da9e0 con 0x5644838574d0 2024-05-10T10:47:45.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.400+0000 7f150b7fe700 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838699d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b2f0 tx=0x7f14fc0a9220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab994d810d2eedf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.403+0000 7f14eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.403+0000 7f14eaffd700 1 -- 172.21.15.112:0/1011427719 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc020040 con 0x5644838574d0 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.403+0000 7f14eaffd700 1 -- 172.21.15.112:0/1011427719 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14fc0b2070 con 0x5644838574d0 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.403+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1504037070 con 0x5644838574d0 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.403+0000 7f14eaffd700 1 -- 172.21.15.112:0/1011427719 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f14fc0b2500 con 0x5644838574d0 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.403+0000 7f151c5ddb80 10 client.26474 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.403+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5644838dbbc0 con 0x5644838574d0 2024-05-10T10:47:45.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f14eaffd700 1 -- 172.21.15.112:0/1011427719 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1504037070 con 0x5644838574d0 2024-05-10T10:47:45.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f14eaffd700 1 -- 172.21.15.112:0/1011427719 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f14fc0a8900 con 0x5644838574d0 2024-05-10T10:47:45.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f151c5ddb80 10 client.26474 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:45.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f151c5ddb80 10 client.26474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:45.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f151c5ddb80 20 client.26474 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:45.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838dc0c0 0x5644838de540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:45.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f151c5ddb80 4 client.26474 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:47:45.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644837b7240 con 0x5644838dc0c0 2024-05-10T10:47:45.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.404+0000 7f150affd700 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838dc0c0 0x5644838de540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:45.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.405+0000 7f150affd700 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838dc0c0 0x5644838de540 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deec21b1c583489b server_cookie=1bd40e2f0e8e84c8 in_seq=0 out_seq=0 2024-05-10T10:47:45.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.405+0000 7f14eaffd700 10 client.26474 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:45.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f14eaffd700 1 -- 172.21.15.112:0/1011427719 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5644837b7240 con 0x5644838dc0c0 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f14eaffd700 10 client.26474 handle_command_reply: tid=0 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 client.26474 shutdown 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 msgr2=0x5644838699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838699d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b2f0 tx=0x7f14fc0a9220 comp rx=0 tx=0).stop 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 shutdown_connections 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x5644838dc0c0 0x5644838de540 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56448385bc80 0x5644838674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5644838574d0 0x5644838699d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 --2- 172.21.15.112:0/1011427719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564483857df0 0x564483866d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 >> 172.21.15.112:0/1011427719 conn(0x564483783100 msgr2=0x56448385a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f14eaffd700 0 client.26474 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 shutdown_connections 2024-05-10T10:47:45.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.406+0000 7f151c5ddb80 1 -- 172.21.15.112:0/1011427719 wait complete. 2024-05-10T10:47:45.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.407+0000 7f151c5ddb80 20 client.26474 trim_cache size 0 max 16384 2024-05-10T10:47:45.410 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:47:45.410 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-10T10:47:45.410 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-10T10:47:45.410 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.407+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14f40b29c0 msgr2=0x7f14f40b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.407+0000 7f1511d4a700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14f40b29c0 0x7f14f40b4e80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1504015000 tx=0x7f1504013000 comp rx=0 tx=0).stop 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.407+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c102e00 msgr2=0x7f150c114750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.407+0000 7f1511d4a700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c102e00 0x7f150c114750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00d950 tx=0x7f14fc00dd10 comp rx=0 tx=0).stop 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 shutdown_connections 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c107760 0x7f150c113260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14f40b29c0 0x7f14f40b4e80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f150c102e00 0x7f150c114750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 --2- 172.21.15.112:0/4244720728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f150c1037b0 0x7f150c114c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:45.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 >> 172.21.15.112:0/4244720728 conn(0x7f150c078a10 msgr2=0x7f150c1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:45.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 shutdown_connections 2024-05-10T10:47:45.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:45.408+0000 7f1511d4a700 1 -- 172.21.15.112:0/4244720728 wait complete. 2024-05-10T10:47:45.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:45 smithi148 ceph-mon[33568]: pgmap v655: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 488 op/s 2024-05-10T10:47:45.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:45 smithi195 ceph-mon[33773]: pgmap v655: 129 pgs: 129 active+clean; 74 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 488 op/s 2024-05-10T10:47:46.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:47:46.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:47:46.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:46 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:47:46.557 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:47:46.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:46 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:47:46.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:46 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:47:47.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:47 smithi112 ceph-mon[30100]: pgmap v656: 129 pgs: 129 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 665 op/s 2024-05-10T10:47:47.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:47:47.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:47:47.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:47:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:47 smithi148 ceph-mon[33568]: pgmap v656: 129 pgs: 129 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 665 op/s 2024-05-10T10:47:47.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:47:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:47:47.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:47:47.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:47 smithi195 ceph-mon[33773]: pgmap v656: 129 pgs: 129 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 665 op/s 2024-05-10T10:47:47.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:47:47.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:47:47.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2130040830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf41034e0 msgr2=0x7f3cf4109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2130040830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf41034e0 0x7f3cf4109d70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009ca0 tx=0x7f3cec02a8c0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2130040830 shutdown_connections 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2130040830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf41034e0 0x7f3cf4109d70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2130040830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf4102b20 0x7f3cf4102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2130040830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4101930 0x7f3cf4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2130040830 >> 172.21.15.112:0/2130040830 conn(0x7f3cf40fcec0 msgr2=0x7f3cf40ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2130040830 shutdown_connections 2024-05-10T10:47:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.218+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2130040830 wait complete. 2024-05-10T10:47:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 Processor -- start 2024-05-10T10:47:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 -- start start 2024-05-10T10:47:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4101930 0x7f3cf4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4102b20 0x7f3cf4198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41034e0 0x7f3cf41a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf410e490 con 0x7f3cf4101930 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf410e310 con 0x7f3cf41034e0 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfbc75640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf410e610 con 0x7f3cf4102b20 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cf9210640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4102b20 0x7f3cf4198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cf9a11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4101930 0x7f3cf4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cf9210640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4102b20 0x7f3cf4198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:36362/0 (socket says 172.21.15.112:36362) 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cf9210640 1 -- 172.21.15.112:0/2425615759 learned_addr learned my addr 172.21.15.112:0/2425615759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.219+0000 7f3cfa212640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41034e0 0x7f3cf41a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cf9210640 1 -- 172.21.15.112:0/2425615759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41034e0 msgr2=0x7f3cf41a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cf9210640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41034e0 0x7f3cf41a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cf9210640 1 -- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4101930 msgr2=0x7f3cf4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cf9210640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4101930 0x7f3cf4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cf9210640 1 -- 172.21.15.112:0/2425615759 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf41a0690 con 0x7f3cf4102b20 2024-05-10T10:47:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4101930 0x7f3cf4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cf9210640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4102b20 0x7f3cf4198fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400e9f0 tx=0x7f3ce400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3ceaffd640 1 -- 172.21.15.112:0/2425615759 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce400cd80 con 0x7f3cf4102b20 2024-05-10T10:47:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf410a790 con 0x7f3cf4102b20 2024-05-10T10:47:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf410ad30 con 0x7f3cf4102b20 2024-05-10T10:47:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3ceaffd640 1 -- 172.21.15.112:0/2425615759 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ce40108c0 con 0x7f3cf4102b20 2024-05-10T10:47:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.220+0000 7f3ceaffd640 1 -- 172.21.15.112:0/2425615759 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40187c0 con 0x7f3cf4102b20 2024-05-10T10:47:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 Processor -- start 2024-05-10T10:47:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 -- start start 2024-05-10T10:47:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d20d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d2122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d214da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56335d1715d0 con 0x56335d212910 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56335d12a890 con 0x56335d20cdb0 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3cfe996740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56335d16d590 con 0x56335d20d7a0 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.223+0000 7f3ceaffd640 1 -- 172.21.15.112:0/2425615759 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3ce4010430 con 0x7f3cf4102b20 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d20d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d20d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:49116/0 (socket says 172.21.15.112:49116) 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 -- 172.21.15.112:0/2560580994 learned_addr learned my addr 172.21.15.112:0/2560580994 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d2122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cf9210640 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d214da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 -- 172.21.15.112:0/2560580994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 msgr2=0x56335d2122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d2122a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 -- 172.21.15.112:0/2560580994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 msgr2=0x56335d214da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d214da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 -- 172.21.15.112:0/2560580994 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335ce83210 con 0x56335d20cdb0 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cf9210640 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d214da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d2122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cfa212640 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d20d1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009ca0 tx=0x7f3cec009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161123ac2635ac01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cdffff640 1 -- 172.21.15.112:0/2560580994 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec031a80 con 0x56335d20cdb0 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cdffff640 1 -- 172.21.15.112:0/2560580994 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cec036030 con 0x56335d20cdb0 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3cdffff640 1 -- 172.21.15.112:0/2560580994 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec004460 con 0x56335d20cdb0 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.224+0000 7f3ceaffd640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ce00b28a0 0x7f3ce00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ce00b28a0 0x7f3ce00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3ceaffd640 1 -- 172.21.15.112:0/2425615759 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3ce40e2910 con 0x7f3cf4102b20 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ce00b28a0 0x7f3ce00b4d60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x56335d20dba0 tx=0x7f3cf0012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 -- 172.21.15.112:0/2560580994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 msgr2=0x56335d20d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d20d1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009ca0 tx=0x7f3cec009a20 comp rx=0 tx=0).stop 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 -- 172.21.15.112:0/2560580994 shutdown_connections 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d214da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d2122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2560580994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d20d1d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 -- 172.21.15.112:0/2560580994 >> 172.21.15.112:0/2560580994 conn(0x56335d208420 msgr2=0x56335d208820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 -- 172.21.15.112:0/2560580994 shutdown_connections 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.225+0000 7f3cfe996740 1 -- 172.21.15.112:0/2560580994 wait complete. 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 -- start start 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 Processor -- start 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20cdb0 0x56335d216df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d217330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d212910 0x56335d21e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56335d2156e0 con 0x56335d20cdb0 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56335d12e370 con 0x56335d212910 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56335d215860 con 0x56335d20d7a0 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfe996740 10 client.0 _ll_register_callbacks cb 0x56335ce82a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cf9a11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20cdb0 0x56335d216df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cf9a11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20cdb0 0x56335d216df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33740/0 (socket says 172.21.15.112:33740) 2024-05-10T10:47:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cf9a11640 1 -- 172.21.15.112:0/3626264922 learned_addr learned my addr 172.21.15.112:0/3626264922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cfa212640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d212910 0x56335d21e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.226+0000 7f3cf9210640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d217330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cf9210640 1 -- 172.21.15.112:0/3626264922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d212910 msgr2=0x56335d21e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cf9210640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d212910 0x56335d21e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cf9210640 1 -- 172.21.15.112:0/3626264922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20cdb0 msgr2=0x56335d216df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cf9210640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20cdb0 0x56335d216df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cf9210640 1 -- 172.21.15.112:0/3626264922 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335cf4f620 con 0x56335d20d7a0 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20cdb0 0x56335d216df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cf9210640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d217330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40e00d0 tx=0x7f3ce4010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d76c5ab2b7cfa7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/3626264922 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40a9760 con 0x56335d20d7a0 2024-05-10T10:47:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cec036030 con 0x56335d20d7a0 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/3626264922 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3ce40a8eb0 con 0x56335d20d7a0 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/3626264922 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3ce40a9c50 con 0x56335d20d7a0 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cfe996740 10 client.26494 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56335d221410 con 0x56335d20d7a0 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.227+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/3626264922 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cec036030 con 0x56335d20d7a0 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/3626264922 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f3ce40a89b0 con 0x56335d20d7a0 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cfe996740 10 client.26494 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cfe996740 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cfe996740 20 client.26494 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d222a90 0x56335d224ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cfe996740 4 client.26494 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56335d17f1e0 con 0x56335d222a90 2024-05-10T10:47:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.228+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d222a90 0x56335d224ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.229+0000 7f3cdd7fa640 10 client.26494 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.229+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d222a90 0x56335d224ea0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4fd43d86af02cec server_cookie=594369a4cc8d5adb in_seq=0 out_seq=0 2024-05-10T10:47:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.229+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/3626264922 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56335d17f1e0 con 0x56335d222a90 2024-05-10T10:47:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.229+0000 7f3cdd7fa640 10 client.26494 handle_command_reply: tid=0 2024-05-10T10:47:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.229+0000 7f3cfe996740 1 client.26494 shutdown 2024-05-10T10:47:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 msgr2=0x56335d217330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d217330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40e00d0 tx=0x7f3ce4010720 comp rx=0 tx=0).stop 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 shutdown_connections 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d222a90 0x56335d224ea0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d212910 0x56335d21e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d217330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3626264922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20cdb0 0x56335d216df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cdd7fa640 0 client.26494 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 >> 172.21.15.112:0/3626264922 conn(0x56335d208420 msgr2=0x56335d207580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 shutdown_connections 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 1 -- 172.21.15.112:0/3626264922 wait complete. 2024-05-10T10:47:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.230+0000 7f3cfe996740 20 client.26494 trim_cache size 0 max 16384 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.255+0000 7f3cfe996740 1 Processor -- start 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.255+0000 7f3cfe996740 1 -- start start 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d206cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20d7a0 0x56335d207230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d212910 0x56335d207770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cfe996740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf41a91b0 con 0x56335d20d7a0 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cfe996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf41aacf0 con 0x56335d20cdb0 2024-05-10T10:47:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cfe996740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf41a9880 con 0x56335d212910 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20d7a0 0x56335d207230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20d7a0 0x56335d207230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33750/0 (socket says 172.21.15.112:33750) 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 -- 172.21.15.112:0/3093876462 learned_addr learned my addr 172.21.15.112:0/3093876462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d206cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 -- 172.21.15.112:0/3093876462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d212910 msgr2=0x56335d207770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d212910 0x56335d207770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 -- 172.21.15.112:0/3093876462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 msgr2=0x56335d206cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d206cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 -- 172.21.15.112:0/3093876462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335d225520 con 0x56335d20d7a0 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d206cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.256+0000 7f3cf9210640 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20d7a0 0x56335d207230 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40a61f0 tx=0x7f3ce4010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c823e2583735d338 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.257+0000 7f3cdf7fe640 1 -- 172.21.15.112:0/3093876462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40adf50 con 0x56335d20d7a0 2024-05-10T10:47:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.257+0000 7f3cdf7fe640 1 -- 172.21.15.112:0/3093876462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ce40cae50 con 0x56335d20d7a0 2024-05-10T10:47:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.257+0000 7f3cdf7fe640 1 -- 172.21.15.112:0/3093876462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40af3c0 con 0x56335d20d7a0 2024-05-10T10:47:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 -- 172.21.15.112:0/3093876462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20d7a0 msgr2=0x56335d207230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20d7a0 0x56335d207230 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40a61f0 tx=0x7f3ce4010850 comp rx=0 tx=0).stop 2024-05-10T10:47:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 -- 172.21.15.112:0/3093876462 shutdown_connections 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d212910 0x56335d207770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d20d7a0 0x56335d207230 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 --2- 172.21.15.112:0/3093876462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d206cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 -- 172.21.15.112:0/3093876462 >> 172.21.15.112:0/3093876462 conn(0x56335d208420 msgr2=0x56335d20e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 -- 172.21.15.112:0/3093876462 shutdown_connections 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 -- 172.21.15.112:0/3093876462 wait complete. 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 -- start start 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.258+0000 7f3cfe996740 1 Processor -- start 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d21b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d21b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfe996740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d21c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfe996740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf41aa620 con 0x56335d212910 2024-05-10T10:47:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfe996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf41a9f50 con 0x56335d20cdb0 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfe996740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf41a7190 con 0x56335d20d7a0 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfe996740 10 client.0 _ll_register_callbacks cb 0x7f3ce400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d21c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d21c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33764/0 (socket says 172.21.15.112:33764) 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 -- 172.21.15.112:0/2840401385 learned_addr learned my addr 172.21.15.112:0/2840401385 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 -- 172.21.15.112:0/2840401385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 msgr2=0x56335d21b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d21b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 -- 172.21.15.112:0/2840401385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 msgr2=0x56335d21b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d21b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.259+0000 7f3cfa212640 1 -- 172.21.15.112:0/2840401385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335d21ccb0 con 0x56335d212910 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cfa212640 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d21c6c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009ca0 tx=0x7f3cec009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac3f24661a17c5b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/2840401385 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec053020 con 0x56335d212910 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/2840401385 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3cec003b70 con 0x56335d212910 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce40cae50 con 0x56335d212910 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/2840401385 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3cec04a040 con 0x56335d212910 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cfe996740 10 client.17148 fetch_fsmap learned FSMap version 42 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.260+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56335d28d480 con 0x56335d212910 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/2840401385 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cec036050 con 0x56335d212910 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/2840401385 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f3cec043070 con 0x56335d212910 2024-05-10T10:47:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cfe996740 10 client.17148 fetch_fsmap finished waiting for FSMap version 42 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cfe996740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cfe996740 20 client.17148 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d28dbf0 0x56335d290010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cfe996740 4 client.17148 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3ce40cae50 con 0x56335d28dbf0 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d28dbf0 0x56335d290010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.261+0000 7f3cdd7fa640 10 client.17148 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.262+0000 7f3cf9a11640 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d28dbf0 0x56335d290010 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae72e150b68412e6 server_cookie=9ef468b2843c4b56 in_seq=0 out_seq=0 2024-05-10T10:47:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cdd7fa640 1 -- 172.21.15.112:0/2840401385 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f3ce40cae50 con 0x56335d28dbf0 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cdd7fa640 10 client.17148 handle_command_reply: tid=0 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 client.17148 shutdown 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 msgr2=0x56335d21c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d21c6c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009ca0 tx=0x7f3cec009800 comp rx=0 tx=0).stop 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 shutdown_connections 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335d212910 0x56335d21c6c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56335d20d7a0 0x56335d21b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x56335d28dbf0 0x56335d290010 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 --2- 172.21.15.112:0/2840401385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335d20cdb0 0x56335d21b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cdd7fa640 0 client.17148 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 >> 172.21.15.112:0/2840401385 conn(0x56335d208420 msgr2=0x56335d20b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 shutdown_connections 2024-05-10T10:47:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 1 -- 172.21.15.112:0/2840401385 wait complete. 2024-05-10T10:47:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.275+0000 7f3cfe996740 20 client.17148 trim_cache size 0 max 16384 2024-05-10T10:47:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:47:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:47:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:47:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:47:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ce00b28a0 msgr2=0x7f3ce00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ce00b28a0 0x7f3ce00b4d60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x56335d20dba0 tx=0x7f3cf0012040 comp rx=0 tx=0).stop 2024-05-10T10:47:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4102b20 msgr2=0x7f3cf4198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4102b20 0x7f3cf4198fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400e9f0 tx=0x7f3ce400eeb0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 shutdown_connections 2024-05-10T10:47:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3ce00b28a0 0x7f3ce00b4d60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41034e0 0x7f3cf41a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cf4102b20 0x7f3cf4198fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 --2- 172.21.15.112:0/2425615759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4101930 0x7f3cf4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 >> 172.21.15.112:0/2425615759 conn(0x7f3cf40fcec0 msgr2=0x7f3cf4105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 shutdown_connections 2024-05-10T10:47:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:47:48.276+0000 7f3cfbc75640 1 -- 172.21.15.112:0/2425615759 wait complete. 2024-05-10T10:47:49.517 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:49 smithi112 ceph-mon[30100]: pgmap v657: 129 pgs: 129 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 476 op/s 2024-05-10T10:47:49.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:49 smithi148 ceph-mon[33568]: pgmap v657: 129 pgs: 129 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 476 op/s 2024-05-10T10:47:49.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:49 smithi195 ceph-mon[33773]: pgmap v657: 129 pgs: 129 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 476 op/s 2024-05-10T10:47:49.580 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-05-10T10:47:49.582 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-10T10:47:49.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.764+0000 7f8902d5f700 1 -- 172.21.15.112:0/4021146758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06bb40 msgr2=0x7f88fc110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.764+0000 7f8902d5f700 1 --2- 172.21.15.112:0/4021146758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06bb40 0x7f88fc110330 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b0a0 tx=0x7f88f400b3b0 comp rx=0 tx=0).stop 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.764+0000 7f8902d5f700 1 -- 172.21.15.112:0/4021146758 shutdown_connections 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.764+0000 7f8902d5f700 1 --2- 172.21.15.112:0/4021146758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06bb40 0x7f88fc110330 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.764+0000 7f8902d5f700 1 --2- 172.21.15.112:0/4021146758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88fc109510 0x7f88fc1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.764+0000 7f8902d5f700 1 --2- 172.21.15.112:0/4021146758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88fc06b180 0x7f88fc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.765+0000 7f8902d5f700 1 -- 172.21.15.112:0/4021146758 >> 172.21.15.112:0/4021146758 conn(0x7f88fc079350 msgr2=0x7f88fc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.765+0000 7f8902d5f700 1 -- 172.21.15.112:0/4021146758 shutdown_connections 2024-05-10T10:47:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.765+0000 7f8902d5f700 1 -- 172.21.15.112:0/4021146758 wait complete. 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.765+0000 7f8902d5f700 1 Processor -- start 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.765+0000 7f8902d5f700 1 -- start start 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8902d5f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06b180 0x7f88fc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8902d5f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88fc06bb40 0x7f88fc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8902d5f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88fc109510 0x7f88fc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8902d5f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc1134f0 con 0x7f88fc06b180 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8902d5f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc113370 con 0x7f88fc109510 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8902d5f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc113670 con 0x7f88fc06bb40 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06b180 0x7f88fc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06b180 0x7f88fc19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33782/0 (socket says 172.21.15.112:33782) 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 -- 172.21.15.112:0/822198596 learned_addr learned my addr 172.21.15.112:0/822198596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f89012fc700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88fc109510 0x7f88fc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f88fbfff700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88fc06bb40 0x7f88fc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 -- 172.21.15.112:0/822198596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88fc06bb40 msgr2=0x7f88fc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88fc06bb40 0x7f88fc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 -- 172.21.15.112:0/822198596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88fc109510 msgr2=0x7f88fc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88fc109510 0x7f88fc199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 -- 172.21.15.112:0/822198596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc19a030 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f8900afb700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06b180 0x7f88fc19f6d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f88f000e9c0 tx=0x7f88f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f88f9ffb700 1 -- 172.21.15.112:0/822198596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f000cbd0 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f88f9ffb700 1 -- 172.21.15.112:0/822198596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88f000cd70 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.766+0000 7f88f9ffb700 1 -- 172.21.15.112:0/822198596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0005540 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.767+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc19a2c0 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.767+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc1a6530 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.768+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc104bd0 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.770+0000 7f88f9ffb700 1 -- 172.21.15.112:0/822198596 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f88f0004750 con 0x7f88fc06b180 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.771+0000 7f88f9ffb700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88ec0b2880 0x7f88ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.771+0000 7f88fbfff700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88ec0b2880 0x7f88ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.771+0000 7f88f9ffb700 1 -- 172.21.15.112:0/822198596 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f88f00e2a80 con 0x7f88fc06b180 2024-05-10T10:47:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.771+0000 7f88fbfff700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88ec0b2880 0x7f88ec0b4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f88fc079a70 tx=0x7f88e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:49.774+0000 7f88f9ffb700 1 -- 172.21.15.112:0/822198596 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f88f0054d70 con 0x7f88fc06b180 2024-05-10T10:47:50.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.041+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88fc02e510 con 0x7f88fc06b180 2024-05-10T10:47:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.042+0000 7f88f9ffb700 1 -- 172.21.15.112:0/822198596 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11953 (secure 0 0 0) 0x7f88f00aa410 con 0x7f88fc06b180 2024-05-10T10:47:50.044 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:47:50.044 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:46:09.581136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24451,24446,24428,24445]},"id":1}]} 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88ec0b2880 msgr2=0x7f88ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88ec0b2880 0x7f88ec0b4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f88fc079a70 tx=0x7f88e8009260 comp rx=0 tx=0).stop 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06b180 msgr2=0x7f88fc19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06b180 0x7f88fc19f6d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f88f000e9c0 tx=0x7f88f000ecd0 comp rx=0 tx=0).stop 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 shutdown_connections 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88fc109510 0x7f88fc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc06b180 0x7f88fc19f6d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f88ec0b2880 0x7f88ec0b4d40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:50.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 --2- 172.21.15.112:0/822198596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88fc06bb40 0x7f88fc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:50.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.045+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 >> 172.21.15.112:0/822198596 conn(0x7f88fc079350 msgr2=0x7f88fc101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:50.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.046+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 shutdown_connections 2024-05-10T10:47:50.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:50.046+0000 7f8902d5f700 1 -- 172.21.15.112:0/822198596 wait complete. 2024-05-10T10:47:50.048 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 42 2024-05-10T10:47:50.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:50 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/822198596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:47:50.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:50 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/822198596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:47:50.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:50 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/822198596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:47:50.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-10T10:47:51.130 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:47:51.445 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:51 smithi148 ceph-mon[33568]: pgmap v658: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 511 op/s 2024-05-10T10:47:51.520 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:51 smithi112 ceph-mon[30100]: pgmap v658: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 511 op/s 2024-05-10T10:47:51.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:51 smithi195 ceph-mon[33773]: pgmap v658: 129 pgs: 129 active+clean; 82 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 511 op/s 2024-05-10T10:47:53.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1902910170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 msgr2=0x7f3be410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1902910170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 0x7f3be410f250 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00b0a0 tx=0x7f3bdc00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1902910170 shutdown_connections 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1902910170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 0x7f3be410f250 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1902910170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be4108410 0x7f3be41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1902910170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be41023a0 0x7f3be4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1902910170 >> 172.21.15.112:0/1902910170 conn(0x7f3be4079350 msgr2=0x7f3be4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1902910170 shutdown_connections 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1902910170 wait complete. 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.131+0000 7f3bec4f3700 1 Processor -- start 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bec4f3700 1 -- start start 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bec4f3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be41023a0 0x7f3be419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bec4f3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 0x7f3be419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bec4f3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be4108410 0x7f3be41998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bec4f3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112400 con 0x7f3be4102d60 2024-05-10T10:47:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bec4f3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112280 con 0x7f3be41023a0 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bec4f3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112580 con 0x7f3be4108410 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3be9a8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 0x7f3be419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3beaa90700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be4108410 0x7f3be41998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3be9a8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 0x7f3be419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33812/0 (socket says 172.21.15.112:33812) 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3be9a8e700 1 -- 172.21.15.112:0/1668542441 learned_addr learned my addr 172.21.15.112:0/1668542441 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bea28f700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be41023a0 0x7f3be419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3beaa90700 1 -- 172.21.15.112:0/1668542441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be41023a0 msgr2=0x7f3be419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3beaa90700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be41023a0 0x7f3be419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3beaa90700 1 -- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 msgr2=0x7f3be419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3beaa90700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 0x7f3be419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3beaa90700 1 -- 172.21.15.112:0/1668542441 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be419a010 con 0x7f3be4108410 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.132+0000 7f3bea28f700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be41023a0 0x7f3be419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.133+0000 7f3beaa90700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be4108410 0x7f3be41998b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00b0a0 tx=0x7f3bdc004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.133+0000 7f3bdb7fe700 1 -- 172.21.15.112:0/1668542441 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc021070 con 0x7f3be4108410 2024-05-10T10:47:53.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.133+0000 7f3bdb7fe700 1 -- 172.21.15.112:0/1668542441 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bdc007b80 con 0x7f3be4108410 2024-05-10T10:47:53.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.133+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be419a2a0 con 0x7f3be4108410 2024-05-10T10:47:53.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.133+0000 7f3bdb7fe700 1 -- 172.21.15.112:0/1668542441 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc005040 con 0x7f3be4108410 2024-05-10T10:47:53.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.133+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41a65c0 con 0x7f3be4108410 2024-05-10T10:47:53.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.135+0000 7f3bdb7fe700 1 -- 172.21.15.112:0/1668542441 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3bdc024070 con 0x7f3be4108410 2024-05-10T10:47:53.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.136+0000 7f3bdb7fe700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3bd00b2880 0x7f3bd00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.136+0000 7f3bea28f700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3bd00b2880 0x7f3bd00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bdb7fe700 1 -- 172.21.15.112:0/1668542441 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3bdc0e35b0 con 0x7f3be4108410 2024-05-10T10:47:53.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bea28f700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3bd00b2880 0x7f3bd00b4d40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3be419a9f0 tx=0x7f3be0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 Processor -- start 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 -- start start 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a094810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a095190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d719fe6940 con 0x55d71a09a1b0 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d719f4bda0 con 0x55d71a09aad0 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.137+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d719fe2900 con 0x55d71a094d50 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3bea28f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3be9a8e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a094810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3bea28f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33822/0 (socket says 172.21.15.112:33822) 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3bea28f700 1 -- 172.21.15.112:0/3972982410 learned_addr learned my addr 172.21.15.112:0/3972982410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:53.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3beaa90700 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a095190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3be9a8e700 1 -- 172.21.15.112:0/3972982410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 msgr2=0x55d71a095190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a095190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3be9a8e700 1 -- 172.21.15.112:0/3972982410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 msgr2=0x55d71a09a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3be9a8e700 1 -- 172.21.15.112:0/3972982410 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d719f338d0 con 0x55d71a09aad0 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3bea28f700 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a094810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3be406c260 tx=0x7f3bd4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0933afba1c125a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3bd8ff9700 1 -- 172.21.15.112:0/3972982410 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4011d80 con 0x55d71a09aad0 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3bd8ff9700 1 -- 172.21.15.112:0/3972982410 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd40044e0 con 0x55d71a09aad0 2024-05-10T10:47:53.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.138+0000 7f3bd8ff9700 1 -- 172.21.15.112:0/3972982410 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4005280 con 0x55d71a09aad0 2024-05-10T10:47:53.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3972982410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 msgr2=0x55d71a094810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a094810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3be406c260 tx=0x7f3bd4011440 comp rx=0 tx=0).stop 2024-05-10T10:47:53.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3972982410 shutdown_connections 2024-05-10T10:47:53.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a095190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3972982410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a094810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3972982410 >> 172.21.15.112:0/3972982410 conn(0x55d719fc0b70 msgr2=0x55d71a08f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3972982410 shutdown_connections 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.139+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3972982410 wait complete. 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 -- start start 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 Processor -- start 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a0a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a09dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d71a0a1450 con 0x55d71a09a1b0 2024-05-10T10:47:53.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d719f9fed0 con 0x55d71a09aad0 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d71a0a15d0 con 0x55d71a094d50 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bf6d86b80 10 client.0 _ll_register_callbacks cb 0x55d719f7a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3be9a8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bea28f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a0a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bea28f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a0a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:36452/0 (socket says 172.21.15.112:36452) 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3be9a8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33828/0 (socket says 172.21.15.112:33828) 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3bea28f700 1 -- 172.21.15.112:0/4066398693 learned_addr learned my addr 172.21.15.112:0/4066398693 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3beaa90700 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a09dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3be9a8e700 1 -- 172.21.15.112:0/4066398693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 msgr2=0x55d71a0a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a0a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3be9a8e700 1 -- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 msgr2=0x55d71a09dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a09dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.140+0000 7f3be9a8e700 1 -- 172.21.15.112:0/4066398693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d719f9d760 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09da90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55d71a09aeb0 tx=0x7f3bd4011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f2e5b271c1729b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bcdffb700 1 -- 172.21.15.112:0/4066398693 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4026020 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bcdffb700 1 -- 172.21.15.112:0/4066398693 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3bd4003680 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd40044e0 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bcdffb700 1 -- 172.21.15.112:0/4066398693 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3bd4031050 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bf6d86b80 10 client.17166 fetch_fsmap learned FSMap version 43 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d71a0abce0 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.141+0000 7f3bcdffb700 1 -- 172.21.15.112:0/4066398693 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd40044e0 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bcdffb700 1 -- 172.21.15.112:0/4066398693 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f3bd4018070 con 0x55d71a09a1b0 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bf6d86b80 10 client.17166 fetch_fsmap finished waiting for FSMap version 43 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bf6d86b80 10 client.17166 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bf6d86b80 20 client.17166 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:53.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a0ae2c0 0x55d71a0b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bf6d86b80 4 client.17166 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d719ff4320 con 0x55d71a0ae2c0 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bea28f700 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a0ae2c0 0x55d71a0b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bcdffb700 10 client.17166 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.142+0000 7f3bea28f700 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a0ae2c0 0x55d71a0b06b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0123072141050a server_cookie=23c8e2e0242adcc1 in_seq=0 out_seq=0 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bcdffb700 1 -- 172.21.15.112:0/4066398693 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d719ff4320 con 0x55d71a0ae2c0 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bcdffb700 10 client.17166 handle_command_reply: tid=0 2024-05-10T10:47:53.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 client.17166 shutdown 2024-05-10T10:47:53.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 msgr2=0x55d71a09da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09da90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55d71a09aeb0 tx=0x7f3bd4011b80 comp rx=0 tx=0).stop 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 shutdown_connections 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a09aad0 0x55d71a09dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a0ae2c0 0x55d71a0b06b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a094d50 0x55d71a0a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/4066398693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a09a1b0 0x55d71a09da90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 >> 172.21.15.112:0/4066398693 conn(0x55d719fc0b70 msgr2=0x55d71a089b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bcdffb700 0 client.17166 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 shutdown_connections 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.143+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/4066398693 wait complete. 2024-05-10T10:47:53.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.144+0000 7f3bf6d86b80 20 client.17166 trim_cache size 0 max 16384 2024-05-10T10:47:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 Processor -- start 2024-05-10T10:47:53.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 -- start start 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d719fc0b70 0x55d71a094810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a094d50 0x55d71a0aa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 0x55d71a089c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ace10 con 0x55d71a094d50 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ae9b0 con 0x55d719fc0b70 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ad540 con 0x55d71a09a1b0 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3be9a8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a094d50 0x55d71a0aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3be9a8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a094d50 0x55d71a0aa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33844/0 (socket says 172.21.15.112:33844) 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3be9a8e700 1 -- 172.21.15.112:0/2761811002 learned_addr learned my addr 172.21.15.112:0/2761811002 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.159+0000 7f3beaa90700 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 0x55d71a089c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3bea28f700 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d719fc0b70 0x55d71a094810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3be9a8e700 1 -- 172.21.15.112:0/2761811002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 msgr2=0x55d71a089c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 0x55d71a089c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3be9a8e700 1 -- 172.21.15.112:0/2761811002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d719fc0b70 msgr2=0x55d71a094810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d719fc0b70 0x55d71a094810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3be9a8e700 1 -- 172.21.15.112:0/2761811002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d71a09c020 con 0x55d71a094d50 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a094d50 0x55d71a0aa510 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4011480 tx=0x7f3bd40098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e46f61c1096fab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3bcffff700 1 -- 172.21.15.112:0/2761811002 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4026030 con 0x55d71a094d50 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3bcffff700 1 -- 172.21.15.112:0/2761811002 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd4011790 con 0x55d71a094d50 2024-05-10T10:47:53.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.160+0000 7f3bcffff700 1 -- 172.21.15.112:0/2761811002 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4003880 con 0x55d71a094d50 2024-05-10T10:47:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/2761811002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a094d50 msgr2=0x55d71a0aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a094d50 0x55d71a0aa510 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4011480 tx=0x7f3bd40098f0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/2761811002 shutdown_connections 2024-05-10T10:47:53.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 0x55d71a089c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d719fc0b70 0x55d71a094810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/2761811002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71a094d50 0x55d71a0aa510 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/2761811002 >> 172.21.15.112:0/2761811002 conn(0x55d71a0b0a00 msgr2=0x55d71a09c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/2761811002 shutdown_connections 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/2761811002 wait complete. 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- start start 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 Processor -- start 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d719fc0b70 0x55d71a0a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a094d50 0x55d71a0a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 0x55d71a0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ae2e0 con 0x55d719fc0b70 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41adc10 con 0x55d71a094d50 2024-05-10T10:47:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41aadb0 con 0x55d71a09a1b0 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bf6d86b80 10 client.0 _ll_register_callbacks cb 0x55d719f7a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bea28f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d719fc0b70 0x55d71a0a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bea28f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d719fc0b70 0x55d71a0a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33848/0 (socket says 172.21.15.112:33848) 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.161+0000 7f3bea28f700 1 -- 172.21.15.112:0/3642860297 learned_addr learned my addr 172.21.15.112:0/3642860297 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a094d50 0x55d71a0a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bea28f700 1 -- 172.21.15.112:0/3642860297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 msgr2=0x55d71a0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bea28f700 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 0x55d71a0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bea28f700 1 -- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a094d50 msgr2=0x55d71a0a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bea28f700 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a094d50 0x55d71a0a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bea28f700 1 -- 172.21.15.112:0/3642860297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d71a0b55a0 con 0x55d719fc0b70 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a094d50 0x55d71a0a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bea28f700 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d719fc0b70 0x55d71a0a6890 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3be001c740 tx=0x7f3be00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff5c07c5e8cbf4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:47:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bcdffb700 1 -- 172.21.15.112:0/3642860297 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be00118d0 con 0x55d719fc0b70 2024-05-10T10:47:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd4011790 con 0x55d719fc0b70 2024-05-10T10:47:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bcdffb700 1 -- 172.21.15.112:0/3642860297 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3be0011a70 con 0x55d719fc0b70 2024-05-10T10:47:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bcdffb700 1 -- 172.21.15.112:0/3642860297 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3be003c050 con 0x55d719fc0b70 2024-05-10T10:47:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bf6d86b80 10 client.17172 fetch_fsmap learned FSMap version 43 2024-05-10T10:47:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.162+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d71a118490 con 0x55d719fc0b70 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bcdffb700 1 -- 172.21.15.112:0/3642860297 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd4011790 con 0x55d719fc0b70 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bcdffb700 1 -- 172.21.15.112:0/3642860297 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f3be002d850 con 0x55d719fc0b70 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bf6d86b80 10 client.17172 fetch_fsmap finished waiting for FSMap version 43 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bf6d86b80 10 client.17172 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bf6d86b80 20 client.17172 populate_metadata read hostname 'smithi112' 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a09aad0 0x55d71a11ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bf6d86b80 4 client.17172 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 --> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d719fee3c0 con 0x55d71a09aad0 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.166+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a09aad0 0x55d71a11ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3be9a8e700 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a09aad0 0x55d71a11ac00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46de7af3553b3288 server_cookie=f79880619bcad0ca in_seq=0 out_seq=0 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bcdffb700 10 client.17172 ms_handle_connect on v2:172.21.15.148:6836/1474359623 2024-05-10T10:47:53.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bcdffb700 1 -- 172.21.15.112:0/3642860297 <== mds.4 v2:172.21.15.148:6836/1474359623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d719fee3c0 con 0x55d71a09aad0 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bcdffb700 10 client.17172 handle_command_reply: tid=0 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 client.17172 shutdown 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d719fc0b70 msgr2=0x55d71a0a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d719fc0b70 0x55d71a0a6890 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3be001c740 tx=0x7f3be00047c0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 shutdown_connections 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:6836/1474359623,v1:172.21.15.148:6837/1474359623] conn(0x55d71a09aad0 0x55d71a11ac00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d71a09a1b0 0x55d71a0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bcdffb700 0 client.17172 ms_handle_reset on v2:172.21.15.148:6836/1474359623 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d719fc0b70 0x55d71a0a6890 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 --2- 172.21.15.112:0/3642860297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71a094d50 0x55d71a0a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 >> 172.21.15.112:0/3642860297 conn(0x55d71a0b0a00 msgr2=0x55d71a09cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 shutdown_connections 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 1 -- 172.21.15.112:0/3642860297 wait complete. 2024-05-10T10:47:53.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.167+0000 7f3bf6d86b80 20 client.17172 trim_cache size 0 max 16384 2024-05-10T10:47:53.172 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:47:53.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3bd00b2880 msgr2=0x7f3bd00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3bd00b2880 0x7f3bd00b4d40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3be419a9f0 tx=0x7f3be0009260 comp rx=0 tx=0).stop 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be4108410 msgr2=0x7f3be41998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be4108410 0x7f3be41998b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00b0a0 tx=0x7f3bdc004d00 comp rx=0 tx=0).stop 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 shutdown_connections 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f3bd00b2880 0x7f3bd00b4d40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be4108410 0x7f3be41998b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3be41023a0 0x7f3be419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 --2- 172.21.15.112:0/1668542441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be4102d60 0x7f3be419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 >> 172.21.15.112:0/1668542441 conn(0x7f3be4079350 msgr2=0x7f3be41065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 shutdown_connections 2024-05-10T10:47:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:47:53.168+0000 7f3bec4f3700 1 -- 172.21.15.112:0/1668542441 wait complete. 2024-05-10T10:47:53.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:53 smithi112 ceph-mon[30100]: pgmap v659: 129 pgs: 129 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.7 MiB/s wr, 606 op/s 2024-05-10T10:47:53.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:53 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:53.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:53 smithi148 ceph-mon[33568]: pgmap v659: 129 pgs: 129 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.7 MiB/s wr, 606 op/s 2024-05-10T10:47:53.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:53 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:53.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:53 smithi195 ceph-mon[33773]: pgmap v659: 129 pgs: 129 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.7 MiB/s wr, 606 op/s 2024-05-10T10:47:53.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:53 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:53.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-10T10:47:54.127 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:47:55.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:55 smithi148 ceph-mon[33568]: pgmap v660: 129 pgs: 129 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-05-10T10:47:55.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:55 smithi195 ceph-mon[33773]: pgmap v660: 129 pgs: 129 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-05-10T10:47:55.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:55 smithi112 ceph-mon[30100]: pgmap v660: 129 pgs: 129 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 438 op/s 2024-05-10T10:47:57.480 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:57 smithi112 ceph-mon[30100]: pgmap v661: 129 pgs: 129 active+clean; 70 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 465 op/s 2024-05-10T10:47:57.481 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:57 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:57 smithi148 ceph-mon[33568]: pgmap v661: 129 pgs: 129 active+clean; 70 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 465 op/s 2024-05-10T10:47:57.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:57 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:57.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:57 smithi195 ceph-mon[33773]: pgmap v661: 129 pgs: 129 active+clean; 70 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 465 op/s 2024-05-10T10:47:57.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:57 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:58.487 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:58 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:58.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:58 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:58.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:58 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:47:59.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:47:59 smithi112 ceph-mon[30100]: pgmap v662: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 304 op/s 2024-05-10T10:47:59.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:47:59 smithi148 ceph-mon[33568]: pgmap v662: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 304 op/s 2024-05-10T10:47:59.761 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:47:59 smithi195 ceph-mon[33773]: pgmap v662: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 304 op/s 2024-05-10T10:48:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 -- 172.21.15.112:0/3432988224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14481023a0 msgr2=0x7f1448102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 --2- 172.21.15.112:0/3432988224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14481023a0 0x7f1448102820 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f143c009990 tx=0x7f143c009ca0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 -- 172.21.15.112:0/3432988224 shutdown_connections 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 --2- 172.21.15.112:0/3432988224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1448102d60 0x7f144810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 --2- 172.21.15.112:0/3432988224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1448108410 0x7f14481087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 --2- 172.21.15.112:0/3432988224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14481023a0 0x7f1448102820 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 -- 172.21.15.112:0/3432988224 >> 172.21.15.112:0/3432988224 conn(0x7f1448079350 msgr2=0x7f1448079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 -- 172.21.15.112:0/3432988224 shutdown_connections 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.396+0000 7f144de55700 1 -- 172.21.15.112:0/3432988224 wait complete. 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 Processor -- start 2024-05-10T10:48:00.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 -- start start 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14481023a0 0x7f144819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1448102d60 0x7f144819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1448108410 0x7f14481998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1448112410 con 0x7f1448108410 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1448112290 con 0x7f1448102d60 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f144de55700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1448112590 con 0x7f14481023a0 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14481023a0 0x7f144819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14481023a0 0x7f144819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:58482/0 (socket says 172.21.15.112:58482) 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 -- 172.21.15.112:0/1092640529 learned_addr learned my addr 172.21.15.112:0/1092640529 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1448102d60 0x7f144819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f1447fff700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1448108410 0x7f14481998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 -- 172.21.15.112:0/1092640529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1448102d60 msgr2=0x7f144819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1448102d60 0x7f144819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 -- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1448108410 msgr2=0x7f14481998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1448108410 0x7f14481998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.397+0000 7f14477fe700 1 -- 172.21.15.112:0/1092640529 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144819a020 con 0x7f14481023a0 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.398+0000 7f14477fe700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14481023a0 0x7f144819f700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9a0 tx=0x7f143800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.398+0000 7f1444ff9700 1 -- 172.21.15.112:0/1092640529 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143800ca10 con 0x7f14481023a0 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.398+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144819a310 con 0x7f14481023a0 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.398+0000 7f1444ff9700 1 -- 172.21.15.112:0/1092640529 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f143800cbb0 con 0x7f14481023a0 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.398+0000 7f1444ff9700 1 -- 172.21.15.112:0/1092640529 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438018680 con 0x7f14481023a0 2024-05-10T10:48:00.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.398+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481a65d0 con 0x7f14481023a0 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.400+0000 7f1444ff9700 1 -- 172.21.15.112:0/1092640529 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1438010430 con 0x7f14481023a0 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.401+0000 7f1444ff9700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14300b2880 0x7f14300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.401+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14300b2880 0x7f14300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.401+0000 7f1444ff9700 1 -- 172.21.15.112:0/1092640529 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14380e26e0 con 0x7f14481023a0 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14300b2880 0x7f14300b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f14480798e0 tx=0x7f143c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 Processor -- start 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 -- start start 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464e4d0 0x55de0464e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464edf0 0x55de046526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de04652c80 0x55de04653100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de045a0860 con 0x55de0464e4d0 2024-05-10T10:48:00.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55de0453f7c0 con 0x55de0464edf0 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14586e8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55de0459c820 con 0x55de04652c80 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14477fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464e4d0 0x55de0464e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14477fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464e4d0 0x55de0464e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42256/0 (socket says 172.21.15.112:42256) 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f14477fe700 1 -- 172.21.15.112:0/3082779457 learned_addr learned my addr 172.21.15.112:0/3082779457 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f1447fff700 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de04652c80 0x55de04653100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.402+0000 7f1446ffd700 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464edf0 0x55de046526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f1446ffd700 1 -- 172.21.15.112:0/3082779457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de04652c80 msgr2=0x55de04653100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f1446ffd700 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de04652c80 0x55de04653100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f1446ffd700 1 -- 172.21.15.112:0/3082779457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464e4d0 msgr2=0x55de0464e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f1446ffd700 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464e4d0 0x55de0464e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f1446ffd700 1 -- 172.21.15.112:0/3082779457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de044ed8d0 con 0x55de0464edf0 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f14477fe700 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464e4d0 0x55de0464e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:00.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f1447fff700 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de04652c80 0x55de04653100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:48:00.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f1446ffd700 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464edf0 0x55de046526f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b650 tx=0x7f143c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f191fd4588d19084 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:00.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f142dffb700 1 -- 172.21.15.112:0/3082779457 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00b6b0 con 0x55de0464edf0 2024-05-10T10:48:00.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f142dffb700 1 -- 172.21.15.112:0/3082779457 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f143c00bce0 con 0x55de0464edf0 2024-05-10T10:48:00.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.403+0000 7f142dffb700 1 -- 172.21.15.112:0/3082779457 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c025c30 con 0x55de0464edf0 2024-05-10T10:48:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 -- 172.21.15.112:0/3082779457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464edf0 msgr2=0x55de046526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464edf0 0x55de046526f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b650 tx=0x7f143c01f000 comp rx=0 tx=0).stop 2024-05-10T10:48:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 -- 172.21.15.112:0/3082779457 shutdown_connections 2024-05-10T10:48:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de04652c80 0x55de04653100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464e4d0 0x55de0464e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3082779457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464edf0 0x55de046526f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 -- 172.21.15.112:0/3082779457 >> 172.21.15.112:0/3082779457 conn(0x55de0457a100 msgr2=0x55de04648c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 -- 172.21.15.112:0/3082779457 shutdown_connections 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 -- 172.21.15.112:0/3082779457 wait complete. 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.404+0000 7f14586e8b80 1 -- start start 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 1 Processor -- start 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de0465a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de04657450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de04657990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de0465ae30 con 0x55de0464edf0 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55de04506c30 con 0x55de04652c80 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55de0465afb0 con 0x55de0464e4d0 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14586e8b80 10 client.0 _ll_register_callbacks cb 0x55de04538920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de04657450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de0465a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de04657450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42260/0 (socket says 172.21.15.112:42260) 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f14477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de0465a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:58486/0 (socket says 172.21.15.112:58486) 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f1446ffd700 1 -- 172.21.15.112:0/1936060472 learned_addr learned my addr 172.21.15.112:0/1936060472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.405+0000 7f1447fff700 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de04657990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1446ffd700 1 -- 172.21.15.112:0/1936060472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 msgr2=0x55de0465a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de0465a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1446ffd700 1 -- 172.21.15.112:0/1936060472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 msgr2=0x55de04657990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de04657990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1446ffd700 1 -- 172.21.15.112:0/1936060472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0453c690 con 0x55de0464edf0 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de04657450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f143c009720 tx=0x7f143c0256a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4669fb78968cdf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1426ffd700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1426ffd700 1 -- 172.21.15.112:0/1936060472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00be90 con 0x55de0464edf0 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1426ffd700 1 -- 172.21.15.112:0/1936060472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f143c003dc0 con 0x55de0464edf0 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143c00bce0 con 0x55de0464edf0 2024-05-10T10:48:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1426ffd700 1 -- 172.21.15.112:0/1936060472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f143c025a20 con 0x55de0464edf0 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f14586e8b80 10 client.17184 fetch_fsmap learned FSMap version 45 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55de0465ae30 con 0x55de0464edf0 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.406+0000 7f1426ffd700 1 -- 172.21.15.112:0/1936060472 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f143c00bce0 con 0x55de0464edf0 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f1426ffd700 1 -- 172.21.15.112:0/1936060472 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f143c02e020 con 0x55de0464edf0 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14586e8b80 10 client.17184 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14586e8b80 10 client.17184 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14586e8b80 20 client.17184 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de04669d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14586e8b80 4 client.17184 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:48:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de0461a650 con 0x55de04667970 2024-05-10T10:48:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14477fe700 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de04669d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f14477fe700 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de04669d60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7afb83e2c1330f03 server_cookie=8d84a4544870d8fc in_seq=0 out_seq=0 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.407+0000 7f1426ffd700 10 client.17184 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.408+0000 7f1426ffd700 1 -- 172.21.15.112:0/1936060472 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de0461a650 con 0x55de04667970 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.408+0000 7f1426ffd700 10 client.17184 handle_command_reply: tid=0 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.408+0000 7f14586e8b80 1 client.17184 shutdown 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.408+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 msgr2=0x55de04657450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.408+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de04657450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f143c009720 tx=0x7f143c0256a0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 shutdown_connections 2024-05-10T10:48:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de04669d60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de04657990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f1426ffd700 0 client.17184 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de0465a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1936060472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de04657450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 >> 172.21.15.112:0/1936060472 conn(0x55de0457a100 msgr2=0x55de04643720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 shutdown_connections 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 1 -- 172.21.15.112:0/1936060472 wait complete. 2024-05-10T10:48:00.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.409+0000 7f14586e8b80 20 client.17184 trim_cache size 0 max 16384 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 Processor -- start 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 -- start start 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464e4d0 0x55de04663ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464edf0 0x55de046435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de04652c80 0x55de0466f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14481af1b0 con 0x55de04652c80 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14481b0de0 con 0x55de0464e4d0 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14586e8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14481af940 con 0x55de0464edf0 2024-05-10T10:48:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14477fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464e4d0 0x55de04663ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464edf0 0x55de046435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14477fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464e4d0 0x55de04663ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.112:39614/0 (socket says 172.21.15.112:39614) 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f1447fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de04652c80 0x55de0466f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14477fe700 1 -- 172.21.15.112:0/1957987372 learned_addr learned my addr 172.21.15.112:0/1957987372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f1446ffd700 1 -- 172.21.15.112:0/1957987372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464e4d0 msgr2=0x55de04663ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464e4d0 0x55de04663ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f1446ffd700 1 -- 172.21.15.112:0/1957987372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de04652c80 msgr2=0x55de0466f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de04652c80 0x55de0466f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f1446ffd700 1 -- 172.21.15.112:0/1957987372 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de04657d10 con 0x55de0464edf0 2024-05-10T10:48:00.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.425+0000 7f14477fe700 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464e4d0 0x55de04663ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:00.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f1446ffd700 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464edf0 0x55de046435a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f143c000c00 tx=0x7f143c00b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ed1382c9850887 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f142d7fa700 1 -- 172.21.15.112:0/1957987372 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c005bc0 con 0x55de0464edf0 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f142d7fa700 1 -- 172.21.15.112:0/1957987372 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f143c025bc0 con 0x55de0464edf0 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f142d7fa700 1 -- 172.21.15.112:0/1957987372 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c030710 con 0x55de0464edf0 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f14586e8b80 1 -- 172.21.15.112:0/1957987372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464edf0 msgr2=0x55de046435a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464edf0 0x55de046435a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f143c000c00 tx=0x7f143c00b610 comp rx=0 tx=0).stop 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f14586e8b80 1 -- 172.21.15.112:0/1957987372 shutdown_connections 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de04652c80 0x55de0466f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de0464e4d0 0x55de04663ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f14586e8b80 1 --2- 172.21.15.112:0/1957987372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464edf0 0x55de046435a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.426+0000 7f14586e8b80 1 -- 172.21.15.112:0/1957987372 >> 172.21.15.112:0/1957987372 conn(0x55de0457a100 msgr2=0x55de04642e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 -- 172.21.15.112:0/1957987372 shutdown_connections 2024-05-10T10:48:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 -- 172.21.15.112:0/1957987372 wait complete. 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 -- start start 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 Processor -- start 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de046ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de046ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de0465f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14481b0710 con 0x55de0464edf0 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14481b0040 con 0x55de04652c80 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14481ad150 con 0x55de0464e4d0 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f14586e8b80 10 client.0 _ll_register_callbacks cb 0x55de04538920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.427+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de046ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de046ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f1447fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de0465f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de046ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:58506/0 (socket says 172.21.15.112:58506) 2024-05-10T10:48:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 -- 172.21.15.112:0/3832461869 learned_addr learned my addr 172.21.15.112:0/3832461869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 -- 172.21.15.112:0/3832461869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 msgr2=0x55de0465f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de0465f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 -- 172.21.15.112:0/3832461869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 msgr2=0x55de046ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de046ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 -- 172.21.15.112:0/3832461869 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0465f770 con 0x55de0464e4d0 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f1446ffd700 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de046ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14477fe700 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de046ca3f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14380a6be0 tx=0x7f14380cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e099298f4c8fb47 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f1426ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f1426ffd700 1 -- 172.21.15.112:0/3832461869 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14380c2020 con 0x55de0464e4d0 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143c025bc0 con 0x55de0464e4d0 2024-05-10T10:48:00.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f1426ffd700 1 -- 172.21.15.112:0/3832461869 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14380cc0b0 con 0x55de0464e4d0 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.428+0000 7f1426ffd700 1 -- 172.21.15.112:0/3832461869 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f14380cc2b0 con 0x55de0464e4d0 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 10 client.26533 fetch_fsmap learned FSMap version 45 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55de04660aa0 con 0x55de0464e4d0 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f1426ffd700 1 -- 172.21.15.112:0/3832461869 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f143c025bc0 con 0x55de0464e4d0 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f1426ffd700 1 -- 172.21.15.112:0/3832461869 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f14380b2340 con 0x55de0464e4d0 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 10 client.26533 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 10 client.26533 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 20 client.26533 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de046d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 4 client.26533 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 --> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de045ae240 con 0x55de04667970 2024-05-10T10:48:00.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f1446ffd700 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de046d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f1446ffd700 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de046d85e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c7539daae7dd7161 server_cookie=c9552624c525130e in_seq=0 out_seq=0 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.429+0000 7f1426ffd700 10 client.26533 ms_handle_connect on v2:172.21.15.195:6834/3491127541 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f1426ffd700 1 -- 172.21.15.112:0/3832461869 <== mds.3 v2:172.21.15.195:6834/3491127541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de045ae240 con 0x55de04667970 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f1426ffd700 10 client.26533 handle_command_reply: tid=0 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 client.26533 shutdown 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 msgr2=0x55de046ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de046ca3f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14380a6be0 tx=0x7f14380cc040 comp rx=0 tx=0).stop 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 shutdown_connections 2024-05-10T10:48:00.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:6834/3491127541,v1:172.21.15.195:6835/3491127541] conn(0x55de04667970 0x55de046d85e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55de04652c80 0x55de0465f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55de0464e4d0 0x55de046ca3f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f1426ffd700 0 client.26533 ms_handle_reset on v2:172.21.15.195:6834/3491127541 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 --2- 172.21.15.112:0/3832461869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de0464edf0 0x55de046ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 >> 172.21.15.112:0/3832461869 conn(0x55de0457a100 msgr2=0x55de04643720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 shutdown_connections 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 1 -- 172.21.15.112:0/3832461869 wait complete. 2024-05-10T10:48:00.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f14586e8b80 20 client.26533 trim_cache size 0 max 16384 2024-05-10T10:48:00.442 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:48:00.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.430+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14300b2880 msgr2=0x7f14300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14300b2880 0x7f14300b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f14480798e0 tx=0x7f143c005fb0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14481023a0 msgr2=0x7f144819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:00.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14481023a0 0x7f144819f700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9a0 tx=0x7f143800ecb0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 shutdown_connections 2024-05-10T10:48:00.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1448108410 0x7f14481998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f14300b2880 0x7f14300b4d40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14481023a0 0x7f144819f700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 --2- 172.21.15.112:0/1092640529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1448102d60 0x7f144819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:00.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 >> 172.21.15.112:0/1092640529 conn(0x7f1448079350 msgr2=0x7f144806b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:00.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 shutdown_connections 2024-05-10T10:48:00.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:00.431+0000 7f144de55700 1 -- 172.21.15.112:0/1092640529 wait complete. 2024-05-10T10:48:01.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T10:48:01.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:01 smithi112 ceph-mon[30100]: pgmap v663: 129 pgs: 129 active+clean; 68 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 354 op/s 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 -- 172.21.15.112:0/4291924267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8301034c0 msgr2=0x7fe830109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 --2- 172.21.15.112:0/4291924267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8301034c0 0x7fe830109d50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe828009ca0 tx=0x7fe82802a8c0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 -- 172.21.15.112:0/4291924267 shutdown_connections 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 --2- 172.21.15.112:0/4291924267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8301034c0 0x7fe830109d50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 --2- 172.21.15.112:0/4291924267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe830102b00 0x7fe830102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 --2- 172.21.15.112:0/4291924267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe830101910 0x7fe830101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 -- 172.21.15.112:0/4291924267 >> 172.21.15.112:0/4291924267 conn(0x7fe8300fce80 msgr2=0x7fe8300ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 -- 172.21.15.112:0/4291924267 shutdown_connections 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.372+0000 7fe8385da640 1 -- 172.21.15.112:0/4291924267 wait complete. 2024-05-10T10:48:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 Processor -- start 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 -- start start 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe830101910 0x7fe830198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe830102b00 0x7fe830198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8301034c0 0x7fe8301a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010d320 con 0x7fe830101910 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010d1a0 con 0x7fe8301034c0 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe8385da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010d4a0 con 0x7fe830102b00 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe836376640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe830101910 0x7fe830198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe836376640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe830101910 0x7fe830198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42306/0 (socket says 172.21.15.112:42306) 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe836376640 1 -- 172.21.15.112:0/1395223273 learned_addr learned my addr 172.21.15.112:0/1395223273 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe835b75640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe830102b00 0x7fe830198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe836b77640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8301034c0 0x7fe8301a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe835b75640 1 -- 172.21.15.112:0/1395223273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8301034c0 msgr2=0x7fe8301a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe835b75640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8301034c0 0x7fe8301a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe835b75640 1 -- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe830101910 msgr2=0x7fe830198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe835b75640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe830101910 0x7fe830198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe835b75640 1 -- 172.21.15.112:0/1395223273 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8301a06b0 con 0x7fe830102b00 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe836376640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe830101910 0x7fe830198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.373+0000 7fe835b75640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe830102b00 0x7fe830198fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe82000e9f0 tx=0x7fe82000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.374+0000 7fe8277fe640 1 -- 172.21.15.112:0/1395223273 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82000cd80 con 0x7fe830102b00 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.374+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8301a09a0 con 0x7fe830102b00 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.374+0000 7fe8277fe640 1 -- 172.21.15.112:0/1395223273 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8200035f0 con 0x7fe830102b00 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.374+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8301a0f40 con 0x7fe830102b00 2024-05-10T10:48:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.374+0000 7fe8277fe640 1 -- 172.21.15.112:0/1395223273 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820018670 con 0x7fe830102b00 2024-05-10T10:48:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe8277fe640 1 -- 172.21.15.112:0/1395223273 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe8200188d0 con 0x7fe830102b00 2024-05-10T10:48:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 Processor -- start 2024-05-10T10:48:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 -- start start 2024-05-10T10:48:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038baf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bb46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bb4d20 0x557038bb71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557038b0f5d0 con 0x557038bafbb0 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557038ac8890 con 0x557038baf1c0 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557038b0b590 con 0x557038bb4d20 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe836376640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bb46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe835b75640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bb4d20 0x557038bb71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe836376640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bb46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42322/0 (socket says 172.21.15.112:42322) 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe836376640 1 -- 172.21.15.112:0/856090390 learned_addr learned my addr 172.21.15.112:0/856090390 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.377+0000 7fe836b77640 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038baf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe836b77640 1 -- 172.21.15.112:0/856090390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bb4d20 msgr2=0x557038bb71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe836b77640 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bb4d20 0x557038bb71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe836b77640 1 -- 172.21.15.112:0/856090390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 msgr2=0x557038bb46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe836b77640 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bb46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe836b77640 1 -- 172.21.15.112:0/856090390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557038821210 con 0x557038baf1c0 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe8277fe640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe81c0b28a0 0x7fe81c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe836b77640 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038baf5e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe828006010 tx=0x7fe8280093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2154ddf410452e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe836376640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe81c0b28a0 0x7fe81c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe8277fe640 1 -- 172.21.15.112:0/1395223273 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe8200e28a0 con 0x7fe830102b00 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe824bfe640 1 -- 172.21.15.112:0/856090390 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8280316d0 con 0x557038baf1c0 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe824bfe640 1 -- 172.21.15.112:0/856090390 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe828036040 con 0x557038baf1c0 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.378+0000 7fe824bfe640 1 -- 172.21.15.112:0/856090390 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8280043b0 con 0x557038baf1c0 2024-05-10T10:48:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe83b3cc740 1 -- 172.21.15.112:0/856090390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 msgr2=0x557038baf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe836376640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe81c0b28a0 0x7fe81c0b4d60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x557038baffb0 tx=0x7fe82c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038baf5e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe828006010 tx=0x7fe8280093e0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe83b3cc740 1 -- 172.21.15.112:0/856090390 shutdown_connections 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bb4d20 0x557038bb71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bb46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/856090390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038baf5e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.379+0000 7fe83b3cc740 1 -- 172.21.15.112:0/856090390 >> 172.21.15.112:0/856090390 conn(0x557038ba6420 msgr2=0x557038ba6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 -- 172.21.15.112:0/856090390 shutdown_connections 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 -- 172.21.15.112:0/856090390 wait complete. 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 -- start start 2024-05-10T10:48:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 Processor -- start 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bb9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bb4d20 0x557038bc0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557038bb7c70 con 0x557038bb4d20 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557038acc370 con 0x557038baf1c0 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557038bb7df0 con 0x557038bafbb0 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe83b3cc740 10 client.0 _ll_register_callbacks cb 0x557038820a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe836b77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bb4d20 0x557038bc0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.380+0000 7fe836b77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bb4d20 0x557038bc0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42324/0 (socket says 172.21.15.112:42324) 2024-05-10T10:48:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836b77640 1 -- 172.21.15.112:0/303122619 learned_addr learned my addr 172.21.15.112:0/303122619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836376640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe835b75640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bb9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836b77640 1 -- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 msgr2=0x557038bb9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836b77640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bb9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836b77640 1 -- 172.21.15.112:0/303122619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 msgr2=0x557038bb93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836b77640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836b77640 1 -- 172.21.15.112:0/303122619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570388ed620 con 0x557038bb4d20 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe835b75640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bb9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe836b77640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bb4d20 0x557038bc0960 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe828009620 tx=0x7fe828009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc77e68a55b5c23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe819ffb640 1 -- 172.21.15.112:0/303122619 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828052020 con 0x557038bb4d20 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe819ffb640 1 -- 172.21.15.112:0/303122619 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe828003970 con 0x557038bb4d20 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe828036040 con 0x557038bb4d20 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe819ffb640 1 -- 172.21.15.112:0/303122619 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe82805d050 con 0x557038bb4d20 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe83b3cc740 10 client.17202 fetch_fsmap learned FSMap version 45 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.381+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557038bb7c70 con 0x557038bb4d20 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe819ffb640 1 -- 172.21.15.112:0/303122619 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe828036040 con 0x557038bb4d20 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe819ffb640 1 -- 172.21.15.112:0/303122619 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7fe82804d030 con 0x557038bb4d20 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe83b3cc740 10 client.17202 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe83b3cc740 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:48:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe83b3cc740 20 client.17202 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038bc7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe83b3cc740 4 client.17202 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557038b1d1e0 con 0x557038bc4c60 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.382+0000 7fe836376640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038bc7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.383+0000 7fe836376640 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038bc7050 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825d6e1afe5bf612 server_cookie=3618474e8f8282fa in_seq=0 out_seq=0 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.383+0000 7fe819ffb640 10 client.17202 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.383+0000 7fe819ffb640 1 -- 172.21.15.112:0/303122619 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557038b1d1e0 con 0x557038bc4c60 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.383+0000 7fe819ffb640 10 client.17202 handle_command_reply: tid=0 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.383+0000 7fe83b3cc740 1 client.17202 shutdown 2024-05-10T10:48:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bb4d20 msgr2=0x557038bc0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bb4d20 0x557038bc0960 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe828009620 tx=0x7fe828009800 comp rx=0 tx=0).stop 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 shutdown_connections 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bb4d20 0x557038bc0960 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bb9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038bc7050 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe819ffb640 0 client.17202 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/303122619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 >> 172.21.15.112:0/303122619 conn(0x557038ba6420 msgr2=0x557038ba5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 shutdown_connections 2024-05-10T10:48:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 1 -- 172.21.15.112:0/303122619 wait complete. 2024-05-10T10:48:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.384+0000 7fe83b3cc740 20 client.17202 trim_cache size 0 max 16384 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.403+0000 7fe83b3cc740 1 Processor -- start 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.403+0000 7fe83b3cc740 1 -- start start 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.403+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.403+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bc0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.403+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bb9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.403+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8301a9280 con 0x557038baf1c0 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8301aaeb0 con 0x557038bb4d20 2024-05-10T10:48:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe8301a9a10 con 0x557038bafbb0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42328/0 (socket says 172.21.15.112:42328) 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 -- 172.21.15.112:0/797206307 learned_addr learned my addr 172.21.15.112:0/797206307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe835b75640 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bc0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836b77640 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bb9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 -- 172.21.15.112:0/797206307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 msgr2=0x557038bc0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bc0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 -- 172.21.15.112:0/797206307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 msgr2=0x557038bb9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bb9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 -- 172.21.15.112:0/797206307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557038ba70f0 con 0x557038baf1c0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe836376640 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038baf1c0 0x557038bb93e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x557038bb0620 tx=0x7fe82c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0066c17ab2f0da2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe81bfff640 1 -- 172.21.15.112:0/797206307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c02b530 con 0x557038baf1c0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe81bfff640 1 -- 172.21.15.112:0/797206307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe82c031070 con 0x557038baf1c0 2024-05-10T10:48:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.404+0000 7fe81bfff640 1 -- 172.21.15.112:0/797206307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011d40 con 0x557038baf1c0 2024-05-10T10:48:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.405+0000 7fe83b3cc740 1 -- 172.21.15.112:0/797206307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038baf1c0 msgr2=0x557038bb93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.405+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038baf1c0 0x557038bb93e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x557038bb0620 tx=0x7fe82c017000 comp rx=0 tx=0).stop 2024-05-10T10:48:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.405+0000 7fe83b3cc740 1 -- 172.21.15.112:0/797206307 shutdown_connections 2024-05-10T10:48:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.405+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bb9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.405+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038bafbb0 0x557038bc0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.405+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/797206307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038baf1c0 0x557038bb93e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.405+0000 7fe83b3cc740 1 -- 172.21.15.112:0/797206307 >> 172.21.15.112:0/797206307 conn(0x557038ba6420 msgr2=0x557038bb4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 -- 172.21.15.112:0/797206307 shutdown_connections 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 -- 172.21.15.112:0/797206307 wait complete. 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 -- start start 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 Processor -- start 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038baf1c0 0x557038bbdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bbe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bbf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8301aa7e0 con 0x557038bafbb0 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8301aa110 con 0x557038bb4d20 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe8301a7260 con 0x557038baf1c0 2024-05-10T10:48:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe83b3cc740 10 client.0 _ll_register_callbacks cb 0x7fe82c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.406+0000 7fe835b75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bbe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bbe520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42336/0 (socket says 172.21.15.112:42336) 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 -- 172.21.15.112:0/359510933 learned_addr learned my addr 172.21.15.112:0/359510933 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe836376640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038baf1c0 0x557038bbdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe836b77640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bbf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 -- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038baf1c0 msgr2=0x557038bbdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038baf1c0 0x557038bbdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 -- 172.21.15.112:0/359510933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 msgr2=0x557038bbf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bbf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 -- 172.21.15.112:0/359510933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557038bbfa40 con 0x557038bafbb0 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe836376640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038baf1c0 0x557038bbdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe835b75640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bbe520 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe8200a9830 tx=0x7fe8200a9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49255c2743207b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe819ffb640 1 -- 172.21.15.112:0/359510933 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8200aae60 con 0x557038bafbb0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe819ffb640 1 -- 172.21.15.112:0/359510933 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe820009310 con 0x557038bafbb0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82c031070 con 0x557038bafbb0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe819ffb640 1 -- 172.21.15.112:0/359510933 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe8200c3050 con 0x557038bafbb0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe83b3cc740 10 client.17208 fetch_fsmap learned FSMap version 45 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.407+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557038c2f510 con 0x557038bafbb0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe819ffb640 1 -- 172.21.15.112:0/359510933 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe82c031070 con 0x557038bafbb0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe819ffb640 1 -- 172.21.15.112:0/359510933 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7fe8200a5dc0 con 0x557038bafbb0 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe83b3cc740 10 client.17208 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe83b3cc740 10 client.17208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe83b3cc740 20 client.17208 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038c31e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe83b3cc740 4 client.17208 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:48:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557038b17050 con 0x557038bc4c60 2024-05-10T10:48:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.408+0000 7fe836376640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038c31e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.409+0000 7fe836376640 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038c31e60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff945b7ccbdbed3 server_cookie=1d3231a09fe81b24 in_seq=0 out_seq=0 2024-05-10T10:48:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.409+0000 7fe819ffb640 10 client.17208 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.425+0000 7fe819ffb640 1 -- 172.21.15.112:0/359510933 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557038b17050 con 0x557038bc4c60 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.425+0000 7fe819ffb640 10 client.17208 handle_command_reply: tid=0 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.425+0000 7fe83b3cc740 1 client.17208 shutdown 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 msgr2=0x557038bbe520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bbe520 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe8200a9830 tx=0x7fe8200a9540 comp rx=0 tx=0).stop 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 shutdown_connections 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x557038bc4c60 0x557038c31e60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557038bb4d20 0x557038bbf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557038bafbb0 0x557038bbe520 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 --2- 172.21.15.112:0/359510933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557038baf1c0 0x557038bbdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe819ffb640 0 client.17208 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 >> 172.21.15.112:0/359510933 conn(0x557038ba6420 msgr2=0x557038bb92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 shutdown_connections 2024-05-10T10:48:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 1 -- 172.21.15.112:0/359510933 wait complete. 2024-05-10T10:48:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.426+0000 7fe83b3cc740 20 client.17208 trim_cache size 0 max 16384 2024-05-10T10:48:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-05-10T10:48:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-05-10T10:48:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-05-10T10:48:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-05-10T10:48:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe81c0b28a0 msgr2=0x7fe81c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe81c0b28a0 0x7fe81c0b4d60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x557038baffb0 tx=0x7fe82c005f90 comp rx=0 tx=0).stop 2024-05-10T10:48:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe830102b00 msgr2=0x7fe830198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe830102b00 0x7fe830198fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe82000e9f0 tx=0x7fe82000eeb0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 shutdown_connections 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fe81c0b28a0 0x7fe81c0b4d60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8301034c0 0x7fe8301a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe830102b00 0x7fe830198fd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 --2- 172.21.15.112:0/1395223273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe830101910 0x7fe830198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 >> 172.21.15.112:0/1395223273 conn(0x7fe8300fce80 msgr2=0x7fe8300fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 shutdown_connections 2024-05-10T10:48:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-05-10T10:48:01.427+0000 7fe8385da640 1 -- 172.21.15.112:0/1395223273 wait complete. 2024-05-10T10:48:01.448 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:01.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:01 smithi148 ceph-mon[33568]: pgmap v663: 129 pgs: 129 active+clean; 68 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 354 op/s 2024-05-10T10:48:01.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:01 smithi195 ceph-mon[33773]: pgmap v663: 129 pgs: 129 active+clean; 68 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 354 op/s 2024-05-10T10:48:03.486 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:03 smithi112 ceph-mon[30100]: pgmap v664: 129 pgs: 129 active+clean; 64 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-05-10T10:48:03.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:03 smithi148 ceph-mon[33568]: pgmap v664: 129 pgs: 129 active+clean; 64 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-05-10T10:48:03.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:03 smithi195 ceph-mon[33773]: pgmap v664: 129 pgs: 129 active+clean; 64 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-05-10T10:48:03.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 -- 172.21.15.112:0/658988918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 msgr2=0x7f2534102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 --2- 172.21.15.112:0/658988918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f2534102820 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2528009990 tx=0x7f2528009ca0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 -- 172.21.15.112:0/658988918 shutdown_connections 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 --2- 172.21.15.112:0/658988918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534102d60 0x7f253410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 --2- 172.21.15.112:0/658988918 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2534108410 0x7f25341087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 --2- 172.21.15.112:0/658988918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f2534102820 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 -- 172.21.15.112:0/658988918 >> 172.21.15.112:0/658988918 conn(0x7f2534079350 msgr2=0x7f2534079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 -- 172.21.15.112:0/658988918 shutdown_connections 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 -- 172.21.15.112:0/658988918 wait complete. 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 Processor -- start 2024-05-10T10:48:03.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.594+0000 7f253a386700 1 -- start start 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f253a386700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f253419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f253a386700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2534102d60 0x7f253419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f253a386700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534108410 0x7f25341a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f253a386700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2534112450 con 0x7f25341023a0 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f253a386700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25341122d0 con 0x7f2534108410 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f253a386700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f25341125d0 con 0x7f2534102d60 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f253419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f253419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42350/0 (socket says 172.21.15.112:42350) 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 -- 172.21.15.112:0/278897612 learned_addr learned my addr 172.21.15.112:0/278897612 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f25337fe700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2534102d60 0x7f253419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2538923700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534108410 0x7f25341a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 -- 172.21.15.112:0/278897612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2534102d60 msgr2=0x7f253419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2534102d60 0x7f253419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 -- 172.21.15.112:0/278897612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534108410 msgr2=0x7f25341a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534108410 0x7f25341a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f2533fff700 1 -- 172.21.15.112:0/278897612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25341a5ed0 con 0x7f25341023a0 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.595+0000 7f25337fe700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2534102d60 0x7f253419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.596+0000 7f2533fff700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f253419a910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9c0 tx=0x7f252400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.596+0000 7f25317fa700 1 -- 172.21.15.112:0/278897612 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252400cbd0 con 0x7f25341023a0 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.596+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25341a6100 con 0x7f25341023a0 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.596+0000 7f25317fa700 1 -- 172.21.15.112:0/278897612 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f252400cd70 con 0x7f25341023a0 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.596+0000 7f25317fa700 1 -- 172.21.15.112:0/278897612 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524005540 con 0x7f25341023a0 2024-05-10T10:48:03.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.596+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25341a6610 con 0x7f25341023a0 2024-05-10T10:48:03.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.598+0000 7f25317fa700 1 -- 172.21.15.112:0/278897612 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2524004750 con 0x7f25341023a0 2024-05-10T10:48:03.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.599+0000 7f25317fa700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f251c0b2880 0x7f251c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.599+0000 7f25317fa700 1 -- 172.21.15.112:0/278897612 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f25240e27f0 con 0x7f25341023a0 2024-05-10T10:48:03.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.599+0000 7f25337fe700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f251c0b2880 0x7f251c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 Processor -- start 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 -- start start 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f25337fe700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f251c0b2880 0x7f251c0b4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f253406bcf0 tx=0x7f252801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74590 0x556c33a74970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74eb0 0x556c33a6ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a6f1b0 0x556c33a6f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556c339bc940 con 0x556c33a74eb0 2024-05-10T10:48:03.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556c33921da0 con 0x556c33a6f1b0 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2544c19b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556c339b8900 con 0x556c33a74590 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f25337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74eb0 0x556c33a6ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f25337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74eb0 0x556c33a6ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42366/0 (socket says 172.21.15.112:42366) 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f25337fe700 1 -- 172.21.15.112:0/1648195677 learned_addr learned my addr 172.21.15.112:0/1648195677 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2533fff700 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74590 0x556c33a74970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2538923700 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a6f1b0 0x556c33a6f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2533fff700 1 -- 172.21.15.112:0/1648195677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a6f1b0 msgr2=0x556c33a6f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2533fff700 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a6f1b0 0x556c33a6f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2533fff700 1 -- 172.21.15.112:0/1648195677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74eb0 msgr2=0x556c33a6ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2533fff700 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74eb0 0x556c33a6ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f2533fff700 1 -- 172.21.15.112:0/1648195677 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c339098d0 con 0x556c33a74590 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.600+0000 7f25337fe700 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74eb0 0x556c33a6ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.601+0000 7f2533fff700 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74590 0x556c33a74970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25240dff60 tx=0x7f25240a94f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac716a09e214aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.601+0000 7f251a7fc700 1 -- 172.21.15.112:0/1648195677 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25240c3070 con 0x556c33a74590 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.601+0000 7f251a7fc700 1 -- 172.21.15.112:0/1648195677 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25240b2230 con 0x556c33a74590 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.601+0000 7f251a7fc700 1 -- 172.21.15.112:0/1648195677 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25240a9ee0 con 0x556c33a74590 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.601+0000 7f2544c19b80 1 -- 172.21.15.112:0/1648195677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74590 msgr2=0x556c33a74970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.601+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74590 0x556c33a74970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25240dff60 tx=0x7f25240a94f0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 -- 172.21.15.112:0/1648195677 shutdown_connections 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a6f1b0 0x556c33a6f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74590 0x556c33a74970 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1648195677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74eb0 0x556c33a6ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 -- 172.21.15.112:0/1648195677 >> 172.21.15.112:0/1648195677 conn(0x556c33996b70 msgr2=0x556c33a652a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 -- 172.21.15.112:0/1648195677 shutdown_connections 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 -- 172.21.15.112:0/1648195677 wait complete. 2024-05-10T10:48:03.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 -- start start 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.602+0000 7f2544c19b80 1 Processor -- start 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a7ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a78550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556c33a7b880 con 0x556c33a6f1b0 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556c33975ed0 con 0x556c33a74590 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556c33a7ba00 con 0x556c33a74eb0 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 10 client.0 _ll_register_callbacks cb 0x556c339505b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2533fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a7ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a78550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a78550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:58594/0 (socket says 172.21.15.112:58594) 2024-05-10T10:48:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 -- 172.21.15.112:0/2729040824 learned_addr learned my addr 172.21.15.112:0/2729040824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f25337fe700 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 -- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 msgr2=0x556c33a78010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a78010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 -- 172.21.15.112:0/2729040824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 msgr2=0x556c33a7ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a7ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 -- 172.21.15.112:0/2729040824 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c33973760 con 0x556c33a74eb0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2538923700 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a78550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f252c008bf0 tx=0x7f252c008f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ec38efe36f11b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f25137fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f25137fe700 1 -- 172.21.15.112:0/2729040824 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c010bc0 con 0x556c33a74eb0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25240b2230 con 0x556c33a74eb0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.603+0000 7f25137fe700 1 -- 172.21.15.112:0/2729040824 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f252c010d60 con 0x556c33a74eb0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f25137fe700 1 -- 172.21.15.112:0/2729040824 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f252c024050 con 0x556c33a74eb0 2024-05-10T10:48:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 10 client.26560 fetch_fsmap learned FSMap version 45 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556c33a86260 con 0x556c33a74eb0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f25137fe700 1 -- 172.21.15.112:0/2729040824 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25240b2230 con 0x556c33a74eb0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f25137fe700 1 -- 172.21.15.112:0/2729040824 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f252c015440 con 0x556c33a74eb0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 10 client.26560 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 10 client.26560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 20 client.26560 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33a888a0 0x556c33a8acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 4 client.26560 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c339ca320 con 0x556c33a888a0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.604+0000 7f2533fff700 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33a888a0 0x556c33a8acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.605+0000 7f25137fe700 10 client.26560 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.605+0000 7f2533fff700 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33a888a0 0x556c33a8acf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22f94f362f9a2ec8 server_cookie=5fd2b44f852e7804 in_seq=0 out_seq=0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.605+0000 7f25137fe700 1 -- 172.21.15.112:0/2729040824 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c339ca320 con 0x556c33a888a0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.605+0000 7f25137fe700 10 client.26560 handle_command_reply: tid=0 2024-05-10T10:48:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 client.26560 shutdown 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 msgr2=0x556c33a78550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a78550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f252c008bf0 tx=0x7f252c008f00 comp rx=0 tx=0).stop 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 shutdown_connections 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a78550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33a888a0 0x556c33a8acf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a7ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f25137fe700 0 client.26560 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 --2- 172.21.15.112:0/2729040824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a78010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 >> 172.21.15.112:0/2729040824 conn(0x556c33996b70 msgr2=0x556c33a5fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 shutdown_connections 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 1 -- 172.21.15.112:0/2729040824 wait complete. 2024-05-10T10:48:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.606+0000 7f2544c19b80 20 client.26560 trim_cache size 0 max 16384 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.621+0000 7f2544c19b80 1 Processor -- start 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.621+0000 7f2544c19b80 1 -- start start 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a6f1b0 0x556c33a75be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74590 0x556c33a84910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74eb0 0x556c33a5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2544c19b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25341ad040 con 0x556c33a74590 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2544c19b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25341aec70 con 0x556c33a74eb0 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2544c19b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f25341ad7d0 con 0x556c33a6f1b0 2024-05-10T10:48:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a6f1b0 0x556c33a75be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f25337fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74590 0x556c33a84910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a6f1b0 0x556c33a75be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.112:58604/0 (socket says 172.21.15.112:58604) 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 -- 172.21.15.112:0/3892704345 learned_addr learned my addr 172.21.15.112:0/3892704345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2538923700 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74eb0 0x556c33a5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 -- 172.21.15.112:0/3892704345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74eb0 msgr2=0x556c33a5fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74eb0 0x556c33a5fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 -- 172.21.15.112:0/3892704345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74590 msgr2=0x556c33a84910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74590 0x556c33a84910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 -- 172.21.15.112:0/3892704345 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c33a7a860 con 0x556c33a6f1b0 2024-05-10T10:48:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.622+0000 7f2533fff700 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a6f1b0 0x556c33a75be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25240a9a10 tx=0x7f252400cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472d21c0ab5292c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2519ffb700 1 -- 172.21.15.112:0/3892704345 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252406e070 con 0x556c33a6f1b0 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2519ffb700 1 -- 172.21.15.112:0/3892704345 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25240b2230 con 0x556c33a6f1b0 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2519ffb700 1 -- 172.21.15.112:0/3892704345 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25240c3040 con 0x556c33a6f1b0 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 -- 172.21.15.112:0/3892704345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a6f1b0 msgr2=0x556c33a75be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a6f1b0 0x556c33a75be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25240a9a10 tx=0x7f252400cf40 comp rx=0 tx=0).stop 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 -- 172.21.15.112:0/3892704345 shutdown_connections 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74eb0 0x556c33a5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a6f1b0 0x556c33a75be0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 --2- 172.21.15.112:0/3892704345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a74590 0x556c33a84910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 -- 172.21.15.112:0/3892704345 >> 172.21.15.112:0/3892704345 conn(0x556c33996b70 msgr2=0x556c33a76900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 -- 172.21.15.112:0/3892704345 shutdown_connections 2024-05-10T10:48:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.623+0000 7f2544c19b80 1 -- 172.21.15.112:0/3892704345 wait complete. 2024-05-10T10:48:03.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 -- start start 2024-05-10T10:48:03.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 Processor -- start 2024-05-10T10:48:03.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a8ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a90490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25341ae5a0 con 0x556c33a6f1b0 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25341aded0 con 0x556c33a74590 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f25341aafe0 con 0x556c33a74eb0 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2544c19b80 10 client.0 _ll_register_callbacks cb 0x556c33a67260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2533fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a8ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2533fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a8ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42396/0 (socket says 172.21.15.112:42396) 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2533fff700 1 -- 172.21.15.112:0/1603025506 learned_addr learned my addr 172.21.15.112:0/1603025506 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f25337fe700 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a90490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.624+0000 7f2538923700 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2533fff700 1 -- 172.21.15.112:0/1603025506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 msgr2=0x556c33a909d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2533fff700 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a909d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2533fff700 1 -- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 msgr2=0x556c33a90490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2533fff700 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a90490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2533fff700 1 -- 172.21.15.112:0/1603025506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c33af2310 con 0x556c33a6f1b0 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2533fff700 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a8ff50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2524009230 tx=0x7f2524005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea26b014fb4d6763 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f25137fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f25137fe700 1 -- 172.21.15.112:0/1603025506 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252406e030 con 0x556c33a6f1b0 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f25137fe700 1 -- 172.21.15.112:0/1603025506 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f25240a6070 con 0x556c33a6f1b0 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25240b2230 con 0x556c33a6f1b0 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f25137fe700 1 -- 172.21.15.112:0/1603025506 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f25240c3040 con 0x556c33a6f1b0 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2544c19b80 10 client.17226 fetch_fsmap learned FSMap version 45 2024-05-10T10:48:03.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556c33af35d0 con 0x556c33a6f1b0 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f25137fe700 1 -- 172.21.15.112:0/1603025506 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25240b2230 con 0x556c33a6f1b0 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.625+0000 7f25137fe700 1 -- 172.21.15.112:0/1603025506 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f25240b2450 con 0x556c33a6f1b0 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f2544c19b80 10 client.17226 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f2544c19b80 10 client.17226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f2544c19b80 20 client.17226 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33af3800 0x556c33af5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f2544c19b80 4 client.17226 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 --> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c339c43c0 con 0x556c33af3800 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f25337fe700 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33af3800 0x556c33af5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f25137fe700 10 client.17226 ms_handle_connect on v2:172.21.15.148:6838/1008412635 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f25337fe700 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33af3800 0x556c33af5bf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1df957fec21b4d1c server_cookie=b9c7795dc6c6faed in_seq=0 out_seq=0 2024-05-10T10:48:03.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f25137fe700 1 -- 172.21.15.112:0/1603025506 <== mds.2 v2:172.21.15.148:6838/1008412635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c339c43c0 con 0x556c33af3800 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f25137fe700 10 client.17226 handle_command_reply: tid=0 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.626+0000 7f2544c19b80 1 client.17226 shutdown 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 msgr2=0x556c33a8ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a8ff50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2524009230 tx=0x7f2524005760 comp rx=0 tx=0).stop 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 shutdown_connections 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c33a74eb0 0x556c33a909d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:6838/1008412635,v1:172.21.15.148:6839/1008412635] conn(0x556c33af3800 0x556c33af5bf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c33a6f1b0 0x556c33a8ff50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 --2- 172.21.15.112:0/1603025506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556c33a74590 0x556c33a90490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 >> 172.21.15.112:0/1603025506 conn(0x556c33996b70 msgr2=0x556c33a6a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:03.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f25137fe700 0 client.17226 ms_handle_reset on v2:172.21.15.148:6838/1008412635 2024-05-10T10:48:03.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 shutdown_connections 2024-05-10T10:48:03.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 1 -- 172.21.15.112:0/1603025506 wait complete. 2024-05-10T10:48:03.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f2544c19b80 20 client.17226 trim_cache size 0 max 16384 2024-05-10T10:48:03.636 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:48:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f251c0b2880 msgr2=0x7f251c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f253a386700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f251c0b2880 0x7f251c0b4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f253406bcf0 tx=0x7f252801a040 comp rx=0 tx=0).stop 2024-05-10T10:48:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 msgr2=0x7f253419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.627+0000 7f253a386700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f253419a910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9c0 tx=0x7f252400ecd0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 shutdown_connections 2024-05-10T10:48:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534108410 0x7f25341a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25341023a0 0x7f253419a910 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f251c0b2880 0x7f251c0b4d40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 --2- 172.21.15.112:0/278897612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2534102d60 0x7f253419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:03.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 >> 172.21.15.112:0/278897612 conn(0x7f2534079350 msgr2=0x7f2534101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:03.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 shutdown_connections 2024-05-10T10:48:03.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:03.628+0000 7f253a386700 1 -- 172.21.15.112:0/278897612 wait complete. 2024-05-10T10:48:04.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T10:48:04.947 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:05.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:05 smithi112 ceph-mon[30100]: pgmap v665: 129 pgs: 129 active+clean; 63 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 644 KiB/s wr, 348 op/s 2024-05-10T10:48:05.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:05 smithi148 ceph-mon[33568]: pgmap v665: 129 pgs: 129 active+clean; 63 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 644 KiB/s wr, 348 op/s 2024-05-10T10:48:05.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:05 smithi195 ceph-mon[33773]: pgmap v665: 129 pgs: 129 active+clean; 63 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 644 KiB/s wr, 348 op/s 2024-05-10T10:48:06.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 -- 172.21.15.112:0/903235072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048108410 msgr2=0x7f40481087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 --2- 172.21.15.112:0/903235072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048108410 0x7f40481087f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4038009990 tx=0x7f4038009ca0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 -- 172.21.15.112:0/903235072 shutdown_connections 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 --2- 172.21.15.112:0/903235072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4048102d60 0x7f404810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 --2- 172.21.15.112:0/903235072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048108410 0x7f40481087f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 --2- 172.21.15.112:0/903235072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40481023a0 0x7f4048102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 -- 172.21.15.112:0/903235072 >> 172.21.15.112:0/903235072 conn(0x7f4048079350 msgr2=0x7f4048079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 -- 172.21.15.112:0/903235072 shutdown_connections 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.687+0000 7f404c975700 1 -- 172.21.15.112:0/903235072 wait complete. 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 Processor -- start 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 -- start start 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40481023a0 0x7f404819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048102d60 0x7f404819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4048108410 0x7f4048199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4048112410 con 0x7f4048102d60 2024-05-10T10:48:06.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4048112290 con 0x7f40481023a0 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404c975700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4048112590 con 0x7f4048108410 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f4045d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048102d60 0x7f404819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f404659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40481023a0 0x7f404819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f4045d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048102d60 0x7f404819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36342/0 (socket says 172.21.15.112:36342) 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f4045d9b700 1 -- 172.21.15.112:0/2619102233 learned_addr learned my addr 172.21.15.112:0/2619102233 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.688+0000 7f4046d9d700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4048108410 0x7f4048199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f4045d9b700 1 -- 172.21.15.112:0/2619102233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4048108410 msgr2=0x7f4048199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4048108410 0x7f4048199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f4045d9b700 1 -- 172.21.15.112:0/2619102233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40481023a0 msgr2=0x7f404819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40481023a0 0x7f404819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f4045d9b700 1 -- 172.21.15.112:0/2619102233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404819a020 con 0x7f4048102d60 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f404659c700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40481023a0 0x7f404819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:06.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f4046d9d700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4048108410 0x7f4048199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:06.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048102d60 0x7f404819fc00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f403c00e9a0 tx=0x7f403c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:06.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f40377fe700 1 -- 172.21.15.112:0/2619102233 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c00ca10 con 0x7f4048102d60 2024-05-10T10:48:06.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f40377fe700 1 -- 172.21.15.112:0/2619102233 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c00cbb0 con 0x7f4048102d60 2024-05-10T10:48:06.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f40377fe700 1 -- 172.21.15.112:0/2619102233 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c018680 con 0x7f4048102d60 2024-05-10T10:48:06.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404819a310 con 0x7f4048102d60 2024-05-10T10:48:06.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.689+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40481a6520 con 0x7f4048102d60 2024-05-10T10:48:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.691+0000 7f40377fe700 1 -- 172.21.15.112:0/2619102233 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f403c010430 con 0x7f4048102d60 2024-05-10T10:48:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.692+0000 7f40377fe700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f40300b2880 0x7f40300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.692+0000 7f404659c700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f40300b2880 0x7f40300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.692+0000 7f40377fe700 1 -- 172.21.15.112:0/2619102233 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f403c0e2740 con 0x7f4048102d60 2024-05-10T10:48:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 Processor -- start 2024-05-10T10:48:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f404659c700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f40300b2880 0x7f40300b4d40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f403800b2e0 tx=0x7f403801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:06.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 -- start start 2024-05-10T10:48:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc896f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc8a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630acbd7860 con 0x5630acc85df0 2024-05-10T10:48:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630acb767c0 con 0x5630acc89c80 2024-05-10T10:48:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4057208b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5630acbd3820 con 0x5630acc854d0 2024-05-10T10:48:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4045d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc896f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4045d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc896f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36348/0 (socket says 172.21.15.112:36348) 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f4045d9b700 1 -- 172.21.15.112:0/2512023697 learned_addr learned my addr 172.21.15.112:0/2512023697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.693+0000 7f404659c700 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4046d9d700 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc8a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4045d9b700 1 -- 172.21.15.112:0/2512023697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 msgr2=0x5630acc858b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc858b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4045d9b700 1 -- 172.21.15.112:0/2512023697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 msgr2=0x5630acc8a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc8a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4045d9b700 1 -- 172.21.15.112:0/2512023697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630acb248d0 con 0x5630acc85df0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4046d9d700 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc8a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc896f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f403c0a96d0 tx=0x7f403c0a95b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec27b0cf12f6cd35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4034ff9700 1 -- 172.21.15.112:0/2512023697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0a99d0 con 0x5630acc85df0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4034ff9700 1 -- 172.21.15.112:0/2512023697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c010e50 con 0x5630acc85df0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.694+0000 7f4034ff9700 1 -- 172.21.15.112:0/2512023697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0b23a0 con 0x5630acc85df0 2024-05-10T10:48:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 -- 172.21.15.112:0/2512023697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 msgr2=0x5630acc896f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc896f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f403c0a96d0 tx=0x7f403c0a95b0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 -- 172.21.15.112:0/2512023697 shutdown_connections 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc8a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc858b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 --2- 172.21.15.112:0/2512023697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc896f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 -- 172.21.15.112:0/2512023697 >> 172.21.15.112:0/2512023697 conn(0x5630acbb1100 msgr2=0x5630acc7fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 -- 172.21.15.112:0/2512023697 shutdown_connections 2024-05-10T10:48:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 -- 172.21.15.112:0/2512023697 wait complete. 2024-05-10T10:48:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 -- start start 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.695+0000 7f4057208b80 1 Processor -- start 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4057208b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4057208b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc85df0 0x5630acc8e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4057208b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc89c80 0x5630acc8e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4057208b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630acc91e90 con 0x5630acc89c80 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4057208b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630acb3dc30 con 0x5630acc85df0 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4057208b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5630acc92010 con 0x5630acc854d0 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4057208b80 10 client.0 _ll_register_callbacks cb 0x5630acb6f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f404659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc89c80 0x5630acc8e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc89c80 0x5630acc8e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36354/0 (socket says 172.21.15.112:36354) 2024-05-10T10:48:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 -- 172.21.15.112:0/761860664 learned_addr learned my addr 172.21.15.112:0/761860664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4045d9b700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc85df0 0x5630acc8e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 -- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 msgr2=0x5630acc910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 -- 172.21.15.112:0/761860664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc85df0 msgr2=0x5630acc8e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc85df0 0x5630acc8e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4046d9d700 1 -- 172.21.15.112:0/761860664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630acb73690 con 0x5630acc89c80 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f4045d9b700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc85df0 0x5630acc8e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.696+0000 7f404659c700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f4046d9d700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc89c80 0x5630acc8e9f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4040009ec0 tx=0x7f4040004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcb118e06b023da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f402dffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f402dffb700 1 -- 172.21.15.112:0/761860664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404001a020 con 0x5630acc89c80 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c010e50 con 0x5630acc89c80 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f402dffb700 1 -- 172.21.15.112:0/761860664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f4040007a00 con 0x5630acc89c80 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f402dffb700 1 -- 172.21.15.112:0/761860664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4040025050 con 0x5630acc89c80 2024-05-10T10:48:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f4057208b80 10 client.17238 fetch_fsmap learned FSMap version 46 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5630acc9c6c0 con 0x5630acc89c80 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f402dffb700 1 -- 172.21.15.112:0/761860664 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c010e50 con 0x5630acc89c80 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f402dffb700 1 -- 172.21.15.112:0/761860664 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 46) ==== 8402+0+0 (secure 0 0 0) 0x7f4040016460 con 0x5630acc89c80 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f4057208b80 10 client.17238 fetch_fsmap finished waiting for FSMap version 46 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.697+0000 7f4057208b80 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.698+0000 7f4057208b80 20 client.17238 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.698+0000 7f4057208b80 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acc9ed00 0x5630acca1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.698+0000 7f4057208b80 4 client.17238 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.698+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630acc51650 con 0x5630acc9ed00 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.698+0000 7f404659c700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acc9ed00 0x5630acca1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.698+0000 7f402dffb700 10 client.17238 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.698+0000 7f404659c700 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acc9ed00 0x5630acca1170 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dfa034f01f69d4e5 server_cookie=41661bde27c7f1da in_seq=0 out_seq=0 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f402dffb700 1 -- 172.21.15.112:0/761860664 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630acc51650 con 0x5630acc9ed00 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f402dffb700 10 client.17238 handle_command_reply: tid=0 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 client.17238 shutdown 2024-05-10T10:48:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc89c80 msgr2=0x5630acc8e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc89c80 0x5630acc8e9f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4040009ec0 tx=0x7f4040004650 comp rx=0 tx=0).stop 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 shutdown_connections 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acc9ed00 0x5630acca1170 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc89c80 0x5630acc8e9f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 --2- 172.21.15.112:0/761860664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc85df0 0x5630acc8e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 >> 172.21.15.112:0/761860664 conn(0x5630acbb1100 msgr2=0x5630acc7a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f402dffb700 0 client.17238 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 shutdown_connections 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 1 -- 172.21.15.112:0/761860664 wait complete. 2024-05-10T10:48:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.700+0000 7f4057208b80 20 client.17238 trim_cache size 0 max 16384 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 Processor -- start 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 -- start start 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc9af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc8ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40481af2d0 con 0x5630acc85df0 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40481b0f60 con 0x5630acc89c80 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4057208b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40481afa60 con 0x5630acc854d0 2024-05-10T10:48:06.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc8ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc8ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36370/0 (socket says 172.21.15.112:36370) 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 -- 172.21.15.112:0/1431454118 learned_addr learned my addr 172.21.15.112:0/1431454118 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f404659c700 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc9af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 -- 172.21.15.112:0/1431454118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 msgr2=0x5630acc9af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc9af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 -- 172.21.15.112:0/1431454118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 msgr2=0x5630acc97680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f4045d9b700 1 -- 172.21.15.112:0/1431454118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630acc91430 con 0x5630acc85df0 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.717+0000 7f404659c700 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc9af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4045d9b700 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc8ddb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f403c000c00 tx=0x7f403c0a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62476f30f68da292 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f402ffff700 1 -- 172.21.15.112:0/1431454118 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0add80 con 0x5630acc85df0 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f402ffff700 1 -- 172.21.15.112:0/1431454118 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c0b2830 con 0x5630acc85df0 2024-05-10T10:48:06.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f402ffff700 1 -- 172.21.15.112:0/1431454118 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0b2600 con 0x5630acc85df0 2024-05-10T10:48:06.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 -- 172.21.15.112:0/1431454118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 msgr2=0x5630acc8ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc8ddb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f403c000c00 tx=0x7f403c0a5890 comp rx=0 tx=0).stop 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 -- 172.21.15.112:0/1431454118 shutdown_connections 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acc97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc854d0 0x5630acc9af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 --2- 172.21.15.112:0/1431454118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc85df0 0x5630acc8ddb0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 -- 172.21.15.112:0/1431454118 >> 172.21.15.112:0/1431454118 conn(0x5630acbb1100 msgr2=0x5630acc7a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 -- 172.21.15.112:0/1431454118 shutdown_connections 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.718+0000 7f4057208b80 1 -- 172.21.15.112:0/1431454118 wait complete. 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 -- start start 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 Processor -- start 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc854d0 0x5630acd05d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc85df0 0x5630acd062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acca6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40481b0890 con 0x5630acc854d0 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40481b01c0 con 0x5630acc89c80 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40481ad270 con 0x5630acc85df0 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4057208b80 10 client.0 _ll_register_callbacks cb 0x5630acb6f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f404659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc854d0 0x5630acd05d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4045d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc85df0 0x5630acd062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f404659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc854d0 0x5630acd05d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36384/0 (socket says 172.21.15.112:36384) 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f404659c700 1 -- 172.21.15.112:0/2435736422 learned_addr learned my addr 172.21.15.112:0/2435736422 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.719+0000 7f4046d9d700 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acca6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f404659c700 1 -- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc85df0 msgr2=0x5630acd062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f404659c700 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc85df0 0x5630acd062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f404659c700 1 -- 172.21.15.112:0/2435736422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 msgr2=0x5630acca6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f404659c700 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acca6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f404659c700 1 -- 172.21.15.112:0/2435736422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630acca6980 con 0x5630acc854d0 2024-05-10T10:48:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc85df0 0x5630acd062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f404659c700 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc854d0 0x5630acd05d60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4038000c00 tx=0x7f4038009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafa741995d15871 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f402dffb700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f402dffb700 1 -- 172.21.15.112:0/2435736422 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380360b0 con 0x5630acc854d0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f402dffb700 1 -- 172.21.15.112:0/2435736422 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f4038036250 con 0x5630acc854d0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630acbe5240 con 0x5630acc854d0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f402dffb700 1 -- 172.21.15.112:0/2435736422 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4038045050 con 0x5630acc854d0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f4057208b80 10 client.17244 fetch_fsmap learned FSMap version 46 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5630acd0d490 con 0x5630acc854d0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f402dffb700 1 -- 172.21.15.112:0/2435736422 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5630acbe5240 con 0x5630acc854d0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f402dffb700 1 -- 172.21.15.112:0/2435736422 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 46) ==== 8402+0+0 (secure 0 0 0) 0x7f40380266e0 con 0x5630acc854d0 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f4057208b80 10 client.17244 fetch_fsmap finished waiting for FSMap version 46 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.720+0000 7f4057208b80 10 client.17244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 20 client.17244 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acd0dbc0 0x5630acd0ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 4 client.17244 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 --> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630acbdf2e0 con 0x5630acd0dbc0 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acd0dbc0 0x5630acd0ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f402dffb700 10 client.17244 ms_handle_connect on v2:172.21.15.195:6836/3223375652 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4045d9b700 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acd0dbc0 0x5630acd0ffe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2137317ed848af6f server_cookie=28d28e130b9177aa in_seq=0 out_seq=0 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f402dffb700 1 -- 172.21.15.112:0/2435736422 <== mds.1 v2:172.21.15.195:6836/3223375652 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630acbdf2e0 con 0x5630acd0dbc0 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f402dffb700 10 client.17244 handle_command_reply: tid=0 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 client.17244 shutdown 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc854d0 msgr2=0x5630acd05d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc854d0 0x5630acd05d60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4038000c00 tx=0x7f4038009310 comp rx=0 tx=0).stop 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 shutdown_connections 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:6836/3223375652,v1:172.21.15.195:6837/3223375652] conn(0x5630acd0dbc0 0x5630acd0ffe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630acc89c80 0x5630acca6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f402dffb700 0 client.17244 ms_handle_reset on v2:172.21.15.195:6836/3223375652 2024-05-10T10:48:06.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630acc854d0 0x5630acd05d60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 --2- 172.21.15.112:0/2435736422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630acc85df0 0x5630acd062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.721+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 >> 172.21.15.112:0/2435736422 conn(0x5630acbb1100 msgr2=0x5630acc842e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:06.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 shutdown_connections 2024-05-10T10:48:06.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f4057208b80 1 -- 172.21.15.112:0/2435736422 wait complete. 2024-05-10T10:48:06.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f4057208b80 20 client.17244 trim_cache size 0 max 16384 2024-05-10T10:48:06.729 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f40300b2880 msgr2=0x7f40300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f40300b2880 0x7f40300b4d40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f403800b2e0 tx=0x7f403801a040 comp rx=0 tx=0).stop 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048102d60 msgr2=0x7f404819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048102d60 0x7f404819fc00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f403c00e9a0 tx=0x7f403c00ecb0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 shutdown_connections 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f40300b2880 0x7f40300b4d40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4048108410 0x7f4048199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40481023a0 0x7f404819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 --2- 172.21.15.112:0/2619102233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4048102d60 0x7f404819fc00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 >> 172.21.15.112:0/2619102233 conn(0x7f4048079350 msgr2=0x7f404806b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 shutdown_connections 2024-05-10T10:48:06.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:06.722+0000 7f404c975700 1 -- 172.21.15.112:0/2619102233 wait complete. 2024-05-10T10:48:07.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:07 smithi148 ceph-mon[33568]: pgmap v666: 129 pgs: 129 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 448 op/s 2024-05-10T10:48:07.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:07 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:07.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:07 smithi195 ceph-mon[33773]: pgmap v666: 129 pgs: 129 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 448 op/s 2024-05-10T10:48:07.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:07 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:07.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:07 smithi112 ceph-mon[30100]: pgmap v666: 129 pgs: 129 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 448 op/s 2024-05-10T10:48:07.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:07 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:07.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T10:48:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:09.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:09 smithi148 ceph-mon[33568]: pgmap v667: 129 pgs: 129 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 810 KiB/s wr, 458 op/s 2024-05-10T10:48:09.561 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:09 smithi112 ceph-mon[30100]: pgmap v667: 129 pgs: 129 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 810 KiB/s wr, 458 op/s 2024-05-10T10:48:09.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:09 smithi195 ceph-mon[33773]: pgmap v667: 129 pgs: 129 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 810 KiB/s wr, 458 op/s 2024-05-10T10:48:09.784 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-05-10T10:48:09.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-10T10:48:10.119 DEBUG:teuthology.parallel:result is None 2024-05-10T10:48:10.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T10:48:10.157 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T10:48:10.157 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-10T10:48:10.184 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:48:10.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 -- 172.21.15.112:0/3405085259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc108450 msgr2=0x7fd7cc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 --2- 172.21.15.112:0/3405085259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc108450 0x7fd7cc108830 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800b0a0 tx=0x7fd7c800b3b0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 -- 172.21.15.112:0/3405085259 shutdown_connections 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 --2- 172.21.15.112:0/3405085259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc102d90 0x7fd7cc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 --2- 172.21.15.112:0/3405085259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc108450 0x7fd7cc108830 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 --2- 172.21.15.112:0/3405085259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7cc1023d0 0x7fd7cc102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 -- 172.21.15.112:0/3405085259 >> 172.21.15.112:0/3405085259 conn(0x7fd7cc078410 msgr2=0x7fd7cc078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 -- 172.21.15.112:0/3405085259 shutdown_connections 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.161+0000 7fd7d3154700 1 -- 172.21.15.112:0/3405085259 wait complete. 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 Processor -- start 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 -- start start 2024-05-10T10:48:10.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc1023d0 0x7fd7cc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7cc102d90 0x7fd7cc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc108450 0x7fd7cc1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc1123f0 con 0x7fd7cc1023d0 2024-05-10T10:48:10.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc112270 con 0x7fd7cc108450 2024-05-10T10:48:10.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d3154700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc112570 con 0x7fd7cc102d90 2024-05-10T10:48:10.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d0ef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc1023d0 0x7fd7cc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d0ef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc1023d0 0x7fd7cc19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36408/0 (socket says 172.21.15.112:36408) 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/442020594 learned_addr learned my addr 172.21.15.112:0/442020594 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7cc102d90 0x7fd7cc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.162+0000 7fd7d16f1700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc108450 0x7fd7cc1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/442020594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7cc102d90 msgr2=0x7fd7cc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7cc102d90 0x7fd7cc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/442020594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc108450 msgr2=0x7fd7cc1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc108450 0x7fd7cc1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/442020594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc1a5e90 con 0x7fd7cc1023d0 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d16f1700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc108450 0x7fd7cc1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:48:10.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc1023d0 0x7fd7cc19a8d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8006070 tx=0x7fd7c8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7c1ffb700 1 -- 172.21.15.112:0/442020594 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8021070 con 0x7fd7cc1023d0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7c1ffb700 1 -- 172.21.15.112:0/442020594 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7c8003c00 con 0x7fd7cc1023d0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc1a6060 con 0x7fd7cc1023d0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7c1ffb700 1 -- 172.21.15.112:0/442020594 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8004730 con 0x7fd7cc1023d0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.163+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc1a6510 con 0x7fd7cc1023d0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.166+0000 7fd7c1ffb700 1 -- 172.21.15.112:0/442020594 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd7c8007980 con 0x7fd7cc1023d0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.166+0000 7fd7c1ffb700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd7b40b2880 0x7fd7b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.166+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd7b40b2880 0x7fd7b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.166+0000 7fd7c1ffb700 1 -- 172.21.15.112:0/442020594 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd7c800f020 con 0x7fd7cc1023d0 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 Processor -- start 2024-05-10T10:48:10.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 -- start start 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd7b40b2880 0x7fd7b40b4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc078b60 tx=0x7fd7bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0b4d0 0x562e81b0b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0bdf0 0x562e81b0f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b10100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e81a5d860 con 0x562e81b0bdf0 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562e819fc7c0 con 0x562e81b0fc80 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562e81a59820 con 0x562e81b0b4d0 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0bdf0 0x562e81b0f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0bdf0 0x562e81b0f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36424/0 (socket says 172.21.15.112:36424) 2024-05-10T10:48:10.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 -- 172.21.15.112:0/1033909234 learned_addr learned my addr 172.21.15.112:0/1033909234 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0b4d0 0x562e81b0b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7d16f1700 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 -- 172.21.15.112:0/1033909234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0b4d0 msgr2=0x562e81b0b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0b4d0 0x562e81b0b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 -- 172.21.15.112:0/1033909234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 msgr2=0x562e81b10100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.167+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b10100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.168+0000 7fd7c3fff700 1 -- 172.21.15.112:0/1033909234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e819aa8d0 con 0x562e81b0bdf0 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.168+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0bdf0 0x562e81b0f6f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011d70 tx=0x7fd7bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bdb376b1ae39ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.168+0000 7fd7b2ffd700 1 -- 172.21.15.112:0/1033909234 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc004a30 con 0x562e81b0bdf0 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.168+0000 7fd7b2ffd700 1 -- 172.21.15.112:0/1033909234 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7bc0054b0 con 0x562e81b0bdf0 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.168+0000 7fd7b2ffd700 1 -- 172.21.15.112:0/1033909234 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc003af0 con 0x562e81b0bdf0 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/1033909234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0bdf0 msgr2=0x562e81b0f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0bdf0 0x562e81b0f6f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011d70 tx=0x7fd7bc014000 comp rx=0 tx=0).stop 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/1033909234 shutdown_connections 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b10100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0b4d0 0x562e81b0b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/1033909234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0bdf0 0x562e81b0f6f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/1033909234 >> 172.21.15.112:0/1033909234 conn(0x562e81a37100 msgr2=0x562e81b05c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/1033909234 shutdown_connections 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/1033909234 wait complete. 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 -- start start 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.169+0000 7fd7dd9e7b80 1 Processor -- start 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b17020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b14900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e81b17e90 con 0x562e81b0b4d0 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562e819c3c30 con 0x562e81b0fc80 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562e81b18010 con 0x562e81b0bdf0 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 10 client.0 _ll_register_callbacks cb 0x562e819f5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b17020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b17020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36432/0 (socket says 172.21.15.112:36432) 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/3313374365 learned_addr learned my addr 172.21.15.112:0/3313374365 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d16f1700 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b14900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 msgr2=0x562e81b143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/3313374365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 msgr2=0x562e81b14900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b14900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/3313374365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e819f9690 con 0x562e81b0b4d0 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b17020 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80e1580 tx=0x7fd7c80ab4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79b9575107648b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7abfff700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7abfff700 1 -- 172.21.15.112:0/3313374365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c802d560 con 0x562e81b0b4d0 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7bc0054b0 con 0x562e81b0b4d0 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7abfff700 1 -- 172.21.15.112:0/3313374365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd7c80ab020 con 0x562e81b0b4d0 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7abfff700 1 -- 172.21.15.112:0/3313374365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd7c80b3b90 con 0x562e81b0b4d0 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 10 client.17256 fetch_fsmap learned FSMap version 46 2024-05-10T10:48:10.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.170+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562e81b22650 con 0x562e81b0b4d0 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7abfff700 1 -- 172.21.15.112:0/3313374365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7bc0054b0 con 0x562e81b0b4d0 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7abfff700 1 -- 172.21.15.112:0/3313374365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 46) ==== 8402+0+0 (secure 0 0 0) 0x7fd7c8011070 con 0x562e81b0b4d0 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7dd9e7b80 10 client.17256 fetch_fsmap finished waiting for FSMap version 46 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7dd9e7b80 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7dd9e7b80 20 client.17256 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b24c10 0x562e81b270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7dd9e7b80 4 client.17256 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e81ad7650 con 0x562e81b24c10 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b24c10 0x562e81b270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b24c10 0x562e81b270b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f21edd3fca54be server_cookie=aa6f27ee259310cf in_seq=0 out_seq=0 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.171+0000 7fd7abfff700 10 client.17256 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7abfff700 1 -- 172.21.15.112:0/3313374365 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e81ad7650 con 0x562e81b24c10 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7abfff700 10 client.17256 handle_command_reply: tid=0 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 client.17256 shutdown 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 msgr2=0x562e81b17020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b17020 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80e1580 tx=0x7fd7c80ab4c0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 shutdown_connections 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b24c10 0x562e81b270b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0fc80 0x562e81b14900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7abfff700 0 client.17256 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b17020 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/3313374365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 >> 172.21.15.112:0/3313374365 conn(0x562e81a37100 msgr2=0x562e81b00720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 shutdown_connections 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/3313374365 wait complete. 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.172+0000 7fd7dd9e7b80 20 client.17256 trim_cache size 0 max 16384 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 Processor -- start 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 -- start start 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b20f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0bdf0 0x562e81b005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0fc80 0x562e81b0d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc1af190 con 0x562e81b0b4d0 2024-05-10T10:48:10.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc1b0e20 con 0x562e81b0bdf0 2024-05-10T10:48:10.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc1af920 con 0x562e81b0fc80 2024-05-10T10:48:10.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7d0ef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b20f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7d0ef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b20f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36446/0 (socket says 172.21.15.112:36446) 2024-05-10T10:48:10.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.187+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/707276690 learned_addr learned my addr 172.21.15.112:0/707276690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:10.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0bdf0 0x562e81b005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/707276690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0fc80 msgr2=0x562e81b0d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0fc80 0x562e81b0d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/707276690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0bdf0 msgr2=0x562e81b005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0bdf0 0x562e81b005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7d0ef0700 1 -- 172.21.15.112:0/707276690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e81b07520 con 0x562e81b0b4d0 2024-05-10T10:48:10.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b20f20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009a20 tx=0x7fd7c80a6d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43457f81025729e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:10.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7b27fc700 1 -- 172.21.15.112:0/707276690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c802d560 con 0x562e81b0b4d0 2024-05-10T10:48:10.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7b27fc700 1 -- 172.21.15.112:0/707276690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7c80c1070 con 0x562e81b0b4d0 2024-05-10T10:48:10.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7b27fc700 1 -- 172.21.15.112:0/707276690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80b1370 con 0x562e81b0b4d0 2024-05-10T10:48:10.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/707276690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 msgr2=0x562e81b20f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.188+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b20f20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009a20 tx=0x7fd7c80a6d90 comp rx=0 tx=0).stop 2024-05-10T10:48:10.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/707276690 shutdown_connections 2024-05-10T10:48:10.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0fc80 0x562e81b0d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0b4d0 0x562e81b20f20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/707276690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0bdf0 0x562e81b005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/707276690 >> 172.21.15.112:0/707276690 conn(0x562e81a37100 msgr2=0x562e81affe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:10.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/707276690 shutdown_connections 2024-05-10T10:48:10.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/707276690 wait complete. 2024-05-10T10:48:10.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- start start 2024-05-10T10:48:10.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 Processor -- start 2024-05-10T10:48:10.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0b4d0 0x562e81b1d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0fc80 0x562e81b18e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc1b0750 con 0x562e81b0fc80 2024-05-10T10:48:10.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc1b0080 con 0x562e81b0b4d0 2024-05-10T10:48:10.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc1ad130 con 0x562e81b0bdf0 2024-05-10T10:48:10.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7dd9e7b80 10 client.0 _ll_register_callbacks cb 0x562e819f5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:48:10.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7d16f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0fc80 0x562e81b18e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7d16f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0fc80 0x562e81b18e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36460/0 (socket says 172.21.15.112:36460) 2024-05-10T10:48:10.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7d16f1700 1 -- 172.21.15.112:0/919206186 learned_addr learned my addr 172.21.15.112:0/919206186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:10.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.189+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0b4d0 0x562e81b1d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7c3fff700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7d16f1700 1 -- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 msgr2=0x562e81b188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7d16f1700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7d16f1700 1 -- 172.21.15.112:0/919206186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0b4d0 msgr2=0x562e81b1d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:48:10.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7d16f1700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0b4d0 0x562e81b1d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7d16f1700 1 -- 172.21.15.112:0/919206186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e81b193e0 con 0x562e81b0fc80 2024-05-10T10:48:10.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0b4d0 0x562e81b1d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:10.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7d16f1700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0fc80 0x562e81b18e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400deb0 tx=0x7fd7c4012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8106652392dd44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:10.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7abfff700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-10T10:48:10.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7abfff700 1 -- 172.21.15.112:0/919206186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012df0 con 0x562e81b0fc80 2024-05-10T10:48:10.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c80c1070 con 0x562e81b0fc80 2024-05-10T10:48:10.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7abfff700 1 -- 172.21.15.112:0/919206186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd7c4004830 con 0x562e81b0fc80 2024-05-10T10:48:10.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7abfff700 1 -- 172.21.15.112:0/919206186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd7c4025050 con 0x562e81b0fc80 2024-05-10T10:48:10.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7dd9e7b80 10 client.17262 fetch_fsmap learned FSMap version 46 2024-05-10T10:48:10.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562e81b8f2f0 con 0x562e81b0fc80 2024-05-10T10:48:10.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.190+0000 7fd7abfff700 1 -- 172.21.15.112:0/919206186 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7c80c1070 con 0x562e81b0fc80 2024-05-10T10:48:10.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7abfff700 1 -- 172.21.15.112:0/919206186 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 46) ==== 8402+0+0 (secure 0 0 0) 0x7fd7c4011440 con 0x562e81b0fc80 2024-05-10T10:48:10.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 10 client.17262 fetch_fsmap finished waiting for FSMap version 46 2024-05-10T10:48:10.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T10:48:10.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 20 client.17262 populate_metadata read hostname 'smithi112' 2024-05-10T10:48:10.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b8fa50 0x562e81b91e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:10.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 4 client.17262 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T10:48:10.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 --> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e81a6b240 con 0x562e81b8fa50 2024-05-10T10:48:10.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b8fa50 0x562e81b91e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:10.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7d0ef0700 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b8fa50 0x562e81b91e70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcadfa735176f2bd server_cookie=a1f87020f33589d in_seq=0 out_seq=0 2024-05-10T10:48:10.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7abfff700 10 client.17262 ms_handle_connect on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:10.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7abfff700 1 -- 172.21.15.112:0/919206186 <== mds.0 v2:172.21.15.195:6838/2238321568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e81a6b240 con 0x562e81b8fa50 2024-05-10T10:48:10.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7abfff700 10 client.17262 handle_command_reply: tid=0 2024-05-10T10:48:10.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 1 client.17262 shutdown 2024-05-10T10:48:10.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0fc80 msgr2=0x562e81b18e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.191+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0fc80 0x562e81b18e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400deb0 tx=0x7fd7c4012730 comp rx=0 tx=0).stop 2024-05-10T10:48:10.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 shutdown_connections 2024-05-10T10:48:10.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:6838/2238321568,v1:172.21.15.195:6839/2238321568] conn(0x562e81b8fa50 0x562e81b91e70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e81b0fc80 0x562e81b18e20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e81b0b4d0 0x562e81b1d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 --2- 172.21.15.112:0/919206186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562e81b0bdf0 0x562e81b188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 >> 172.21.15.112:0/919206186 conn(0x562e81a37100 msgr2=0x562e81b0e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:10.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7abfff700 0 client.17262 ms_handle_reset on v2:172.21.15.195:6838/2238321568 2024-05-10T10:48:10.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 shutdown_connections 2024-05-10T10:48:10.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 1 -- 172.21.15.112:0/919206186 wait complete. 2024-05-10T10:48:10.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.192+0000 7fd7dd9e7b80 20 client.17262 trim_cache size 0 max 16384 2024-05-10T10:48:10.220 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-10T10:48:10.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd7b40b2880 msgr2=0x7fd7b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd7b40b2880 0x7fd7b40b4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc078b60 tx=0x7fd7bc009260 comp rx=0 tx=0).stop 2024-05-10T10:48:10.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc1023d0 msgr2=0x7fd7cc19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:10.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc1023d0 0x7fd7cc19a8d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8006070 tx=0x7fd7c8004d00 comp rx=0 tx=0).stop 2024-05-10T10:48:10.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 shutdown_connections 2024-05-10T10:48:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc108450 0x7fd7cc1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7cc1023d0 0x7fd7cc19a8d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd7b40b2880 0x7fd7b40b4d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 --2- 172.21.15.112:0/442020594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7cc102d90 0x7fd7cc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 >> 172.21.15.112:0/442020594 conn(0x7fd7cc078410 msgr2=0x7fd7cc1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:10.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 shutdown_connections 2024-05-10T10:48:10.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:10.193+0000 7fd7d3154700 1 -- 172.21.15.112:0/442020594 wait complete. 2024-05-10T10:48:10.225 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-10T10:48:10.236 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-10T10:48:10.976 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-10T10:48:10.977 INFO:tasks.fwd_scrub:done joining 2024-05-10T10:48:10.977 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-10T10:48:10.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 -- 172.21.15.112:0/4229865543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 msgr2=0x7f60cc102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 --2- 172.21.15.112:0/4229865543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc102f50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009900 tx=0x7f60b802a410 comp rx=0 tx=0).stop 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 -- 172.21.15.112:0/4229865543 shutdown_connections 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 --2- 172.21.15.112:0/4229865543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60cc103490 0x7f60cc109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 --2- 172.21.15.112:0/4229865543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc102f50 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 --2- 172.21.15.112:0/4229865543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60cc1018e0 0x7f60cc101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 -- 172.21.15.112:0/4229865543 >> 172.21.15.112:0/4229865543 conn(0x7f60cc0fce70 msgr2=0x7f60cc0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 -- 172.21.15.112:0/4229865543 shutdown_connections 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.089+0000 7f60d3093640 1 -- 172.21.15.112:0/4229865543 wait complete. 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 Processor -- start 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 -- start start 2024-05-10T10:48:11.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60cc1018e0 0x7f60cc196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60cc103490 0x7f60cc19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10c1c0 con 0x7f60cc102ad0 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10c040 con 0x7f60cc103490 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d3093640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10c340 con 0x7f60cc1018e0 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36490/0 (socket says 172.21.15.112:36490) 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 -- 172.21.15.112:0/2991273588 learned_addr learned my addr 172.21.15.112:0/2991273588 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d0e2f640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60cc1018e0 0x7f60cc196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 -- 172.21.15.112:0/2991273588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60cc1018e0 msgr2=0x7f60cc196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60cc1018e0 0x7f60cc196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 -- 172.21.15.112:0/2991273588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60cc103490 msgr2=0x7f60cc19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60cc103490 0x7f60cc19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60cbfff640 1 -- 172.21.15.112:0/2991273588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc19e530 con 0x7f60cc102ad0 2024-05-10T10:48:11.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.090+0000 7f60d0e2f640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60cc1018e0 0x7f60cc196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:48:11.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.091+0000 7f60cbfff640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc196e50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009230 tx=0x7f60b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:11.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.091+0000 7f60c9ffb640 1 -- 172.21.15.112:0/2991273588 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b803f030 con 0x7f60cc102ad0 2024-05-10T10:48:11.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.091+0000 7f60c9ffb640 1 -- 172.21.15.112:0/2991273588 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60b8003c90 con 0x7f60cc102ad0 2024-05-10T10:48:11.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.091+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60cc19e7c0 con 0x7f60cc102ad0 2024-05-10T10:48:11.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.091+0000 7f60c9ffb640 1 -- 172.21.15.112:0/2991273588 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8035070 con 0x7f60cc102ad0 2024-05-10T10:48:11.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.091+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60cc19ecb0 con 0x7f60cc102ad0 2024-05-10T10:48:11.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.092+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60cc108d20 con 0x7f60cc102ad0 2024-05-10T10:48:11.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.093+0000 7f60c9ffb640 1 -- 172.21.15.112:0/2991273588 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f60b8042030 con 0x7f60cc102ad0 2024-05-10T10:48:11.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.094+0000 7f60c9ffb640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60bc0b4a00 0x7f60bc0b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.094+0000 7f60d0e2f640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60bc0b4a00 0x7f60bc0b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.094+0000 7f60c9ffb640 1 -- 172.21.15.112:0/2991273588 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f60b8102e20 con 0x7f60cc102ad0 2024-05-10T10:48:11.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.094+0000 7f60d0e2f640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60bc0b4a00 0x7f60bc0b6ec0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005d00 tx=0x7f60c0005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:11.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.099+0000 7f60c9ffb640 1 -- 172.21.15.112:0/2991273588 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f60b80ca810 con 0x7f60cc102ad0 2024-05-10T10:48:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.288+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60cc063650 con 0x7f60cc102ad0 2024-05-10T10:48:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.289+0000 7f60c9ffb640 1 -- 172.21.15.112:0/2991273588 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f60b80ca110 con 0x7f60cc102ad0 2024-05-10T10:48:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:48:11.291 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:48:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60bc0b4a00 msgr2=0x7f60bc0b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60bc0b4a00 0x7f60bc0b6ec0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005d00 tx=0x7f60c0005ef0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 msgr2=0x7f60cc196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc196e50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009230 tx=0x7f60b80042f0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 shutdown_connections 2024-05-10T10:48:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f60bc0b4a00 0x7f60bc0b6ec0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60cc103490 0x7f60cc19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60cc102ad0 0x7f60cc196e50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 --2- 172.21.15.112:0/2991273588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60cc1018e0 0x7f60cc196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 >> 172.21.15.112:0/2991273588 conn(0x7f60cc0fce70 msgr2=0x7f60cc105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 shutdown_connections 2024-05-10T10:48:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.292+0000 7f60d3093640 1 -- 172.21.15.112:0/2991273588 wait complete. 2024-05-10T10:48:11.305 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:30:55.269138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T10:35:46.170078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T10:35:49.831849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T10:35:59.956882+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T10:48:11.305 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:48:11.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 -- 172.21.15.112:0/2161971019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 msgr2=0x7f65c4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 --2- 172.21.15.112:0/2161971019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4101d20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009900 tx=0x7f65b402a410 comp rx=0 tx=0).stop 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 -- 172.21.15.112:0/2161971019 shutdown_connections 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 --2- 172.21.15.112:0/2161971019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c41034b0 0x7f65c4109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 --2- 172.21.15.112:0/2161971019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c4102af0 0x7f65c4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 --2- 172.21.15.112:0/2161971019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4101d20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 -- 172.21.15.112:0/2161971019 >> 172.21.15.112:0/2161971019 conn(0x7f65c40fce70 msgr2=0x7f65c40ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 -- 172.21.15.112:0/2161971019 shutdown_connections 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 -- 172.21.15.112:0/2161971019 wait complete. 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.424+0000 7f65c9779640 1 Processor -- start 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c9779640 1 -- start start 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c9779640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c9779640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c4102af0 0x7f65c41990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c9779640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c41034b0 0x7f65c41a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c9779640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65c410d2e0 con 0x7f65c4101900 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c9779640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65c410d160 con 0x7f65c41034b0 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c9779640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65c410d460 con 0x7f65c4102af0 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36514/0 (socket says 172.21.15.112:36514) 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 -- 172.21.15.112:0/2262888286 learned_addr learned my addr 172.21.15.112:0/2262888286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c27fc640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c4102af0 0x7f65c41990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c37fe640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c41034b0 0x7f65c41a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 -- 172.21.15.112:0/2262888286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c4102af0 msgr2=0x7f65c41990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c4102af0 0x7f65c41990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 -- 172.21.15.112:0/2262888286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c41034b0 msgr2=0x7f65c41a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c41034b0 0x7f65c41a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 -- 172.21.15.112:0/2262888286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c41a0790 con 0x7f65c4101900 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c37fe640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c41034b0 0x7f65c41a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65c2ffd640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4198b70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005ab0 tx=0x7f65b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.425+0000 7f65abfff640 1 -- 172.21.15.112:0/2262888286 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b403f030 con 0x7f65c4101900 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.426+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c41a0a20 con 0x7f65c4101900 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.426+0000 7f65abfff640 1 -- 172.21.15.112:0/2262888286 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65b4003c90 con 0x7f65c4101900 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.426+0000 7f65abfff640 1 -- 172.21.15.112:0/2262888286 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4035070 con 0x7f65c4101900 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.426+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a0f10 con 0x7f65c4101900 2024-05-10T10:48:11.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.426+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c41a1140 con 0x7f65c4101900 2024-05-10T10:48:11.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.429+0000 7f65abfff640 1 -- 172.21.15.112:0/2262888286 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f65b4042030 con 0x7f65c4101900 2024-05-10T10:48:11.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.429+0000 7f65abfff640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65ac0b27f0 0x7f65ac0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.429+0000 7f65c27fc640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65ac0b27f0 0x7f65ac0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.429+0000 7f65abfff640 1 -- 172.21.15.112:0/2262888286 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f65b4102f90 con 0x7f65c4101900 2024-05-10T10:48:11.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.430+0000 7f65c27fc640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65ac0b27f0 0x7f65ac0b4cb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65c419a080 tx=0x7f65b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:11.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.433+0000 7f65abfff640 1 -- 172.21.15.112:0/2262888286 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f65b40ca920 con 0x7f65c4101900 2024-05-10T10:48:11.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:11 smithi148 ceph-mon[33568]: pgmap v668: 129 pgs: 129 active+clean; 66 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 805 KiB/s wr, 514 op/s 2024-05-10T10:48:11.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:11 smithi195 ceph-mon[33773]: pgmap v668: 129 pgs: 129 active+clean; 66 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 805 KiB/s wr, 514 op/s 2024-05-10T10:48:11.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:11 smithi112 ceph-mon[30100]: pgmap v668: 129 pgs: 129 active+clean; 66 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 805 KiB/s wr, 514 op/s 2024-05-10T10:48:11.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.615+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65c41a1310 con 0x7f65c4101900 2024-05-10T10:48:11.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.615+0000 7f65abfff640 1 -- 172.21.15.112:0/2262888286 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f65b40ca220 con 0x7f65c4101900 2024-05-10T10:48:11.617 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65ac0b27f0 msgr2=0x7f65ac0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65ac0b27f0 0x7f65ac0b4cb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65c419a080 tx=0x7f65b800b410 comp rx=0 tx=0).stop 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 msgr2=0x7f65c4198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4198b70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005ab0 tx=0x7f65b40042f0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 shutdown_connections 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65ac0b27f0 0x7f65ac0b4cb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c41034b0 0x7f65c41a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c4102af0 0x7f65c41990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 --2- 172.21.15.112:0/2262888286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c4101900 0x7f65c4198b70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 >> 172.21.15.112:0/2262888286 conn(0x7f65c40fce70 msgr2=0x7f65c406af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:11.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 shutdown_connections 2024-05-10T10:48:11.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.617+0000 7f65c9779640 1 -- 172.21.15.112:0/2262888286 wait complete. 2024-05-10T10:48:11.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T10:48:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 -- 172.21.15.112:0/920836872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 msgr2=0x7f635c10ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 --2- 172.21.15.112:0/920836872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c10ae40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6354009ca0 tx=0x7f635402a8c0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 -- 172.21.15.112:0/920836872 shutdown_connections 2024-05-10T10:48:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 --2- 172.21.15.112:0/920836872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c10ae40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 --2- 172.21.15.112:0/920836872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635c103bf0 0x7f635c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 --2- 172.21.15.112:0/920836872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f635c102a00 0x7f635c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 -- 172.21.15.112:0/920836872 >> 172.21.15.112:0/920836872 conn(0x7f635c0fdf90 msgr2=0x7f635c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 -- 172.21.15.112:0/920836872 shutdown_connections 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.752+0000 7f6360e2a640 1 -- 172.21.15.112:0/920836872 wait complete. 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 Processor -- start 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 -- start start 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f635c102a00 0x7f635c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635c103bf0 0x7f635c199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f635c10e430 con 0x7f635c1045b0 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f635c10e2b0 con 0x7f635c103bf0 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6360e2a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f635c10e5b0 con 0x7f635c102a00 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f635ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f635ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c1a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36532/0 (socket says 172.21.15.112:36532) 2024-05-10T10:48:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f635ad9d640 1 -- 172.21.15.112:0/1242533958 learned_addr learned my addr 172.21.15.112:0/1242533958 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f635a59c640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f635c102a00 0x7f635c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.753+0000 7f6353fff640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635c103bf0 0x7f635c199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f635ad9d640 1 -- 172.21.15.112:0/1242533958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f635c102a00 msgr2=0x7f635c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f635ad9d640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f635c102a00 0x7f635c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f635ad9d640 1 -- 172.21.15.112:0/1242533958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635c103bf0 msgr2=0x7f635c199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f635ad9d640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635c103bf0 0x7f635c199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f635ad9d640 1 -- 172.21.15.112:0/1242533958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c1a0750 con 0x7f635c1045b0 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f635a59c640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f635c102a00 0x7f635c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f635ad9d640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c1a00e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6354031040 tx=0x7f6354003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f63537fe640 1 -- 172.21.15.112:0/1242533958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354003d40 con 0x7f635c1045b0 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f63537fe640 1 -- 172.21.15.112:0/1242533958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63540044d0 con 0x7f635c1045b0 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c1a09e0 con 0x7f635c1045b0 2024-05-10T10:48:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f63537fe640 1 -- 172.21.15.112:0/1242533958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354030ba0 con 0x7f635c1045b0 2024-05-10T10:48:11.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.754+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c1a0f30 con 0x7f635c1045b0 2024-05-10T10:48:11.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.755+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c109d90 con 0x7f635c1045b0 2024-05-10T10:48:11.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.756+0000 7f63537fe640 1 -- 172.21.15.112:0/1242533958 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6354004680 con 0x7f635c1045b0 2024-05-10T10:48:11.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.757+0000 7f63537fe640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63480b27f0 0x7f63480b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:11.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.757+0000 7f635a59c640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63480b27f0 0x7f63480b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:11.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.757+0000 7f63537fe640 1 -- 172.21.15.112:0/1242533958 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6354103a90 con 0x7f635c1045b0 2024-05-10T10:48:11.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.757+0000 7f635a59c640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63480b27f0 0x7f63480b4cb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f635c103a50 tx=0x7f634c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:11.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.760+0000 7f63537fe640 1 -- 172.21.15.112:0/1242533958 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f63540cb420 con 0x7f635c1045b0 2024-05-10T10:48:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.987+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f635c02dd10 con 0x7f635c1045b0 2024-05-10T10:48:11.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.988+0000 7f63537fe640 1 -- 172.21.15.112:0/1242533958 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f63540cad20 con 0x7f635c1045b0 2024-05-10T10:48:11.989 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-05-10T10:48:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63480b27f0 msgr2=0x7f63480b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63480b27f0 0x7f63480b4cb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f635c103a50 tx=0x7f634c005f90 comp rx=0 tx=0).stop 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 msgr2=0x7f635c1a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c1a00e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6354031040 tx=0x7f6354003ba0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 shutdown_connections 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f63480b27f0 0x7f63480b4cb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f635c1045b0 0x7f635c1a00e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635c103bf0 0x7f635c199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 --2- 172.21.15.112:0/1242533958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f635c102a00 0x7f635c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 >> 172.21.15.112:0/1242533958 conn(0x7f635c0fdf90 msgr2=0x7f635c106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 shutdown_connections 2024-05-10T10:48:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:11.990+0000 7f6360e2a640 1 -- 172.21.15.112:0/1242533958 wait complete. 2024-05-10T10:48:12.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T10:48:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 -- 172.21.15.112:0/3804115476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c101910 msgr2=0x7fda3c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 --2- 172.21.15.112:0/3804115476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c101910 0x7fda3c101d30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda38009900 tx=0x7fda3802a400 comp rx=0 tx=0).stop 2024-05-10T10:48:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 -- 172.21.15.112:0/3804115476 shutdown_connections 2024-05-10T10:48:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 --2- 172.21.15.112:0/3804115476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 0x7fda3c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 --2- 172.21.15.112:0/3804115476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda3c102b00 0x7fda3c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 --2- 172.21.15.112:0/3804115476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c101910 0x7fda3c101d30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 -- 172.21.15.112:0/3804115476 >> 172.21.15.112:0/3804115476 conn(0x7fda3c0fcea0 msgr2=0x7fda3c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 -- 172.21.15.112:0/3804115476 shutdown_connections 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.106+0000 7fda44264640 1 -- 172.21.15.112:0/3804115476 wait complete. 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 Processor -- start 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 -- start start 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda3c101910 0x7fda3c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c102b00 0x7fda3c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 0x7fda3c1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c10d2c0 con 0x7fda3c1034c0 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c10d140 con 0x7fda3c101910 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c10d440 con 0x7fda3c102b00 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 0x7fda3c1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 0x7fda3c1a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36536/0 (socket says 172.21.15.112:36536) 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda417ff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c102b00 0x7fda3c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 -- 172.21.15.112:0/1285126482 learned_addr learned my addr 172.21.15.112:0/1285126482 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42000640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda3c101910 0x7fda3c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 -- 172.21.15.112:0/1285126482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c102b00 msgr2=0x7fda3c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c102b00 0x7fda3c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 -- 172.21.15.112:0/1285126482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda3c101910 msgr2=0x7fda3c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda3c101910 0x7fda3c198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 -- 172.21.15.112:0/1285126482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c1a0750 con 0x7fda3c1034c0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42000640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda3c101910 0x7fda3c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda42801640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 0x7fda3c1a00e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fda3400efd0 tx=0x7fda3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda32ffd640 1 -- 172.21.15.112:0/1285126482 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34003b70 con 0x7fda3c1034c0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c1a0a40 con 0x7fda3c1034c0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.107+0000 7fda32ffd640 1 -- 172.21.15.112:0/1285126482 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda34003d10 con 0x7fda3c1034c0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.108+0000 7fda32ffd640 1 -- 172.21.15.112:0/1285126482 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340107a0 con 0x7fda3c1034c0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.108+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a0f90 con 0x7fda3c1034c0 2024-05-10T10:48:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.108+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c1a1220 con 0x7fda3c1034c0 2024-05-10T10:48:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.110+0000 7fda32ffd640 1 -- 172.21.15.112:0/1285126482 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fda34010940 con 0x7fda3c1034c0 2024-05-10T10:48:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.111+0000 7fda32ffd640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda280b27f0 0x7fda280b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.111+0000 7fda42000640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda280b27f0 0x7fda280b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.111+0000 7fda32ffd640 1 -- 172.21.15.112:0/1285126482 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fda340e2ec0 con 0x7fda3c1034c0 2024-05-10T10:48:12.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.111+0000 7fda42000640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda280b27f0 0x7fda280b4cb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fda380096c0 tx=0x7fda38009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:12.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.114+0000 7fda32ffd640 1 -- 172.21.15.112:0/1285126482 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fda340aa850 con 0x7fda3c1034c0 2024-05-10T10:48:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.300+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fda3c1a13f0 con 0x7fda3c1034c0 2024-05-10T10:48:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.300+0000 7fda32ffd640 1 -- 172.21.15.112:0/1285126482 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fda340aa150 con 0x7fda3c1034c0 2024-05-10T10:48:12.302 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda280b27f0 msgr2=0x7fda280b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda280b27f0 0x7fda280b4cb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fda380096c0 tx=0x7fda38009650 comp rx=0 tx=0).stop 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 msgr2=0x7fda3c1a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 0x7fda3c1a00e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fda3400efd0 tx=0x7fda3400c6a0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 shutdown_connections 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fda280b27f0 0x7fda280b4cb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c1034c0 0x7fda3c1a00e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fda3c102b00 0x7fda3c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 --2- 172.21.15.112:0/1285126482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda3c101910 0x7fda3c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 >> 172.21.15.112:0/1285126482 conn(0x7fda3c0fcea0 msgr2=0x7fda3c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 shutdown_connections 2024-05-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.302+0000 7fda44264640 1 -- 172.21.15.112:0/1285126482 wait complete. 2024-05-10T10:48:12.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:12 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:12.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:12 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2991273588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:48:12.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:12 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2262888286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:48:12.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:12 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1242533958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T10:48:12.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 -- 172.21.15.112:0/2177609773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba941018f0 msgr2=0x7fba94101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/2177609773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba941018f0 0x7fba94101d10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fba90009900 tx=0x7fba9002a410 comp rx=0 tx=0).stop 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 -- 172.21.15.112:0/2177609773 shutdown_connections 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/2177609773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba941034a0 0x7fba94109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/2177609773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94102ae0 0x7fba94102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/2177609773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba941018f0 0x7fba94101d10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 -- 172.21.15.112:0/2177609773 >> 172.21.15.112:0/2177609773 conn(0x7fba940fce60 msgr2=0x7fba940ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 -- 172.21.15.112:0/2177609773 shutdown_connections 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.417+0000 7fba9c4e8640 1 -- 172.21.15.112:0/2177609773 wait complete. 2024-05-10T10:48:12.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 Processor -- start 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 -- start start 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba941018f0 0x7fba94197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba94102ae0 0x7fba94197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba941034a0 0x7fba9419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba9410d2a0 con 0x7fba94102ae0 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba9410d120 con 0x7fba941018f0 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9c4e8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fba9410d420 con 0x7fba941034a0 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba94102ae0 0x7fba94197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba94102ae0 0x7fba94197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36562/0 (socket says 172.21.15.112:36562) 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 -- 172.21.15.112:0/3932170610 learned_addr learned my addr 172.21.15.112:0/3932170610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9a284640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba941018f0 0x7fba94197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba9aa85640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba941034a0 0x7fba9419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 -- 172.21.15.112:0/3932170610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba941034a0 msgr2=0x7fba9419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba941034a0 0x7fba9419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 -- 172.21.15.112:0/3932170610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba941018f0 msgr2=0x7fba94197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba941018f0 0x7fba94197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 -- 172.21.15.112:0/3932170610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9419f680 con 0x7fba94102ae0 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.418+0000 7fba99a83640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba94102ae0 0x7fba94197fa0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e9f0 tx=0x7fba8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.419+0000 7fba8b7fe640 1 -- 172.21.15.112:0/3932170610 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8400cd80 con 0x7fba94102ae0 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.419+0000 7fba8b7fe640 1 -- 172.21.15.112:0/3932170610 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba840108c0 con 0x7fba94102ae0 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.419+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9419f970 con 0x7fba94102ae0 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.419+0000 7fba8b7fe640 1 -- 172.21.15.112:0/3932170610 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba840187c0 con 0x7fba94102ae0 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.419+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9419fec0 con 0x7fba94102ae0 2024-05-10T10:48:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.420+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba94108cd0 con 0x7fba94102ae0 2024-05-10T10:48:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.421+0000 7fba8b7fe640 1 -- 172.21.15.112:0/3932170610 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fba84010430 con 0x7fba94102ae0 2024-05-10T10:48:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.422+0000 7fba8b7fe640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fba800b27f0 0x7fba800b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.422+0000 7fba9a284640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fba800b27f0 0x7fba800b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.422+0000 7fba8b7fe640 1 -- 172.21.15.112:0/3932170610 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fba840e2e80 con 0x7fba94102ae0 2024-05-10T10:48:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.422+0000 7fba9a284640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fba800b27f0 0x7fba800b4cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fba900057c0 tx=0x7fba90031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.425+0000 7fba8b7fe640 1 -- 172.21.15.112:0/3932170610 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fba840aa810 con 0x7fba94102ae0 2024-05-10T10:48:12.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:12 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:12.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:12 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2991273588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:48:12.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:12 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2262888286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:48:12.545 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:12 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1242533958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T10:48:12.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:12 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:12.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:12 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2991273588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:48:12.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:12 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2262888286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T10:48:12.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:12 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1242533958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T10:48:12.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.609+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fba9402dda0 con 0x7fba94102ae0 2024-05-10T10:48:12.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.609+0000 7fba8b7fe640 1 -- 172.21.15.112:0/3932170610 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fba840aa110 con 0x7fba94102ae0 2024-05-10T10:48:12.611 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-05-10T10:48:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fba800b27f0 msgr2=0x7fba800b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fba800b27f0 0x7fba800b4cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fba900057c0 tx=0x7fba90031400 comp rx=0 tx=0).stop 2024-05-10T10:48:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba94102ae0 msgr2=0x7fba94197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba94102ae0 0x7fba94197fa0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e9f0 tx=0x7fba8400eeb0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 shutdown_connections 2024-05-10T10:48:12.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fba800b27f0 0x7fba800b4cb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba941034a0 0x7fba9419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba94102ae0 0x7fba94197fa0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 --2- 172.21.15.112:0/3932170610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba941018f0 0x7fba94197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.611+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 >> 172.21.15.112:0/3932170610 conn(0x7fba940fce60 msgr2=0x7fba94105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.612+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 shutdown_connections 2024-05-10T10:48:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.612+0000 7fba9c4e8640 1 -- 172.21.15.112:0/3932170610 wait complete. 2024-05-10T10:48:12.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 -- 172.21.15.112:0/3863834218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e81045b0 msgr2=0x7f99e810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/3863834218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e81045b0 0x7f99e810ae40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009ca0 tx=0x7f99e002a8c0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 -- 172.21.15.112:0/3863834218 shutdown_connections 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/3863834218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e81045b0 0x7f99e810ae40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/3863834218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e8103bf0 0x7f99e8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/3863834218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e8102a00 0x7f99e8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 -- 172.21.15.112:0/3863834218 >> 172.21.15.112:0/3863834218 conn(0x7f99e80fdf90 msgr2=0x7f99e81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 -- 172.21.15.112:0/3863834218 shutdown_connections 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 -- 172.21.15.112:0/3863834218 wait complete. 2024-05-10T10:48:12.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 Processor -- start 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.725+0000 7f99ecb5f640 1 -- start start 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8102a00 0x7f99e819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e8103bf0 0x7f99e819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e81045b0 0x7f99e81a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99e810e4a0 con 0x7f99e8102a00 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99e810e320 con 0x7f99e8103bf0 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f99e810e620 con 0x7f99e81045b0 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8102a00 0x7f99e819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8102a00 0x7f99e819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36574/0 (socket says 172.21.15.112:36574) 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 -- 172.21.15.112:0/4007213989 learned_addr learned my addr 172.21.15.112:0/4007213989 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99dffff640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e8103bf0 0x7f99e819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e6d9d640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e81045b0 0x7f99e81a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 -- 172.21.15.112:0/4007213989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e81045b0 msgr2=0x7f99e81a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e81045b0 0x7f99e81a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 -- 172.21.15.112:0/4007213989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e8103bf0 msgr2=0x7f99e819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e8103bf0 0x7f99e819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 -- 172.21.15.112:0/4007213989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e81a2a20 con 0x7f99e8102a00 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99e659c640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8102a00 0x7f99e819ae00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f99d800b450 tx=0x7f99d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99df7fe640 1 -- 172.21.15.112:0/4007213989 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8015020 con 0x7f99e8102a00 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e81a2d10 con 0x7f99e8102a00 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99df7fe640 1 -- 172.21.15.112:0/4007213989 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d8011d20 con 0x7f99e8102a00 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99df7fe640 1 -- 172.21.15.112:0/4007213989 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8010e10 con 0x7f99e8102a00 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.726+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e81a31b0 con 0x7f99e8102a00 2024-05-10T10:48:12.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.727+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e81a33e0 con 0x7f99e8102a00 2024-05-10T10:48:12.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.729+0000 7f99df7fe640 1 -- 172.21.15.112:0/4007213989 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f99d8018020 con 0x7f99e8102a00 2024-05-10T10:48:12.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.729+0000 7f99df7fe640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99d40b27f0 0x7f99d40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:12.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.730+0000 7f99dffff640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99d40b27f0 0x7f99d40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:12.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.730+0000 7f99df7fe640 1 -- 172.21.15.112:0/4007213989 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f99d80e25a0 con 0x7f99e8102a00 2024-05-10T10:48:12.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.730+0000 7f99dffff640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99d40b27f0 0x7f99d40b4cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99e819c310 tx=0x7f99d000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:12.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.733+0000 7f99df7fe640 1 -- 172.21.15.112:0/4007213989 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99d801f020 con 0x7f99e8102a00 2024-05-10T10:48:12.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.966+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f99e81a3650 con 0x7f99e8102a00 2024-05-10T10:48:12.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.975+0000 7f99df7fe640 1 -- 172.21.15.112:0/4007213989 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815651 (secure 0 0 0) 0x7f99d800d040 con 0x7f99e8102a00 2024-05-10T10:48:12.987 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:48:12.987 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:48:12.987 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 18, 2024-05-10T10:48:12.987 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:12.987 INFO:teuthology.orchestra.run.smithi112.stdout: "active_gid": 14150, 2024-05-10T10:48:12.987 INFO:teuthology.orchestra.run.smithi112.stdout: "active_name": "x", 2024-05-10T10:48:12.987 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addrs": { 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6800", 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1898750131 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6801", 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1898750131 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.988 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addr": "172.21.15.112:6801/1898750131", 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "active_change": "2024-05-10T10:28:33.178773+0000", 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "active_mgr_features": 4540701547738038271, 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "available": true, 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "standbys": [ 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24109, 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "z", 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:12.989 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.990 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:12.991 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:12.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.995 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:12.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-05-10T10:48:12.997 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-05-10T10:48:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.000 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.001 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.002 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.005 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-05-10T10:48:13.006 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.007 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T10:48:13.008 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T10:48:13.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T10:48:13.011 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.015 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.017 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.020 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.026 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-05-10T10:48:13.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.029 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.031 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.034 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.035 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T10:48:13.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.042 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.046 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.047 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.052 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-05-10T10:48:13.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-05-10T10:48:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-05-10T10:48:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-05-10T10:48:13.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-05-10T10:48:13.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-05-10T10:48:13.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-05-10T10:48:13.060 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.061 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-05-10T10:48:13.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T10:48:13.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T10:48:13.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-05-10T10:48:13.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.070 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.071 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.072 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.078 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.094 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.095 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.097 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.098 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.099 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.100 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-05-10T10:48:13.101 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.111 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.113 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:13.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.122 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.127 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.134 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.136 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.142 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.145 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.150 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-05-10T10:48:13.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.155 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.156 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.157 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.161 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.163 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.169 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.171 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.174 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.175 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T10:48:13.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T10:48:13.177 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.180 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.182 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-05-10T10:48:13.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.184 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.185 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.192 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.196 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-05-10T10:48:13.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.201 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-05-10T10:48:13.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.205 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-05-10T10:48:13.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.210 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.211 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.212 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.215 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.218 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.220 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T10:48:13.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.228 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.229 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-05-10T10:48:13.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.233 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.239 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.243 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.244 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.245 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.248 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.250 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.251 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.252 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.253 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.257 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.260 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.261 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-10T10:48:13.264 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.266 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.271 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.273 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.274 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.276 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.277 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.278 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.281 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.286 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.287 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.288 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.290 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.291 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.293 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T10:48:13.294 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-05-10T10:48:13.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.297 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.298 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.300 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.301 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.304 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.305 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.306 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.309 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-05-10T10:48:13.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.314 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T10:48:13.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T10:48:13.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.315 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.315 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.315 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-05-10T10:48:13.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T10:48:13.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-10T10:48:13.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.328 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.329 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.330 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.332 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.334 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.335 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.336 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.339 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.341 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.344 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.347 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.353 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.354 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.357 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.358 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-05-10T10:48:13.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.364 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.365 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.371 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.377 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.379 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.386 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-05-10T10:48:13.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.391 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.393 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.394 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.395 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.397 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.398 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.430 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.445 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.446 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.449 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.450 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.451 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24116, 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "y", 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.477 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.488 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-05-10T10:48:13.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-05-10T10:48:13.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-10T10:48:13.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.501 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.502 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-05-10T10:48:13.505 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-05-10T10:48:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-05-10T10:48:13.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T10:48:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-05-10T10:48:13.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-05-10T10:48:13.522 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-05-10T10:48:13.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-05-10T10:48:13.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-05-10T10:48:13.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-05-10T10:48:13.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.538 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T10:48:13.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-05-10T10:48:13.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-05-10T10:48:13.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T10:48:13.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-05-10T10:48:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.556 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.557 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-10T10:48:13.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T10:48:13.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-05-10T10:48:13.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.576 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.582 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-05-10T10:48:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.585 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.594 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T10:48:13.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T10:48:13.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.597 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.598 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.599 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.600 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T10:48:13.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.602 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.603 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.605 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.606 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T10:48:13.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.608 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T10:48:13.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.610 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.611 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.612 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T10:48:13.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T10:48:13.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.615 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T10:48:13.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T10:48:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T10:48:13.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.619 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.620 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T10:48:13.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T10:48:13.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T10:48:13.640 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.641 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.649 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.652 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.657 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.659 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.661 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-10T10:48:13.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.663 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.664 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-05-10T10:48:13.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-05-10T10:48:13.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.674 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.676 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.677 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.678 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.681 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-05-10T10:48:13.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-05-10T10:48:13.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.687 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.689 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.691 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-05-10T10:48:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-05-10T10:48:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.707 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.712 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.715 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.717 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.722 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.723 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T10:48:13.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.726 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.727 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-05-10T10:48:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-05-10T10:48:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.755 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.758 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.762 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.763 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.766 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.767 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-05-10T10:48:13.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.770 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.771 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.773 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.774 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-10T10:48:13.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.778 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.779 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.782 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.784 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.786 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.787 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.790 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-05-10T10:48:13.793 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.794 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.795 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.797 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.800 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.803 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.804 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.806 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.814 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.820 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.823 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T10:48:13.826 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T10:48:13.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.828 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.831 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.834 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.835 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.838 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.840 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-05-10T10:48:13.843 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.845 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.848 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.849 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.850 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:13.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:13.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.853 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.858 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.859 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.861 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.862 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.863 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.866 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.867 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.871 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.872 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.875 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.881 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.882 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.885 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-05-10T10:48:13.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-05-10T10:48:13.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.888 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.889 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.890 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.895 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.896 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.898 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.899 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.901 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.904 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.905 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.908 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.909 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.911 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.915 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.918 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.919 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.922 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.925 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.929 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.933 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.936 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.938 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.939 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.942 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.943 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.945 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.946 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.951 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.954 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-05-10T10:48:13.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.960 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.961 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.962 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.966 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.969 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.971 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.972 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.973 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T10:48:13.974 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.976 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.977 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.980 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.981 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.982 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.983 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.984 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-05-10T10:48:13.987 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.988 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: "modules": [ 2024-05-10T10:48:13.991 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard", 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "iostat", 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "nfs", 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "restful" 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:13.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.994 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.996 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.997 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:13.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-05-10T10:48:14.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-05-10T10:48:14.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.004 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T10:48:14.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.008 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.010 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.014 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.015 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-05-10T10:48:14.016 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.018 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.023 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-05-10T10:48:14.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-10T10:48:14.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.026 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:14.030 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-05-10T10:48:14.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-05-10T10:48:14.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.034 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-05-10T10:48:14.039 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.042 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-05-10T10:48:14.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.046 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.054 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.058 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.059 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-05-10T10:48:14.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.063 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.064 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:14.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.069 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.070 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.072 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-05-10T10:48:14.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T10:48:14.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.086 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T10:48:14.090 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T10:48:14.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T10:48:14.092 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T10:48:14.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T10:48:14.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T10:48:14.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.103 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.105 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.107 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.111 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.112 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.115 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.116 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.118 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.122 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.123 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-05-10T10:48:14.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T10:48:14.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.145 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.147 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.150 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T10:48:14.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T10:48:14.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-05-10T10:48:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-05-10T10:48:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-10T10:48:14.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.172 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.174 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.175 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.177 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.178 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-05-10T10:48:14.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.180 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.191 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-05-10T10:48:14.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-05-10T10:48:14.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.195 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.197 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T10:48:14.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T10:48:14.199 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.202 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.203 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-05-10T10:48:14.212 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.220 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.221 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-05-10T10:48:14.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-10T10:48:14.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-05-10T10:48:14.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-05-10T10:48:14.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-05-10T10:48:14.228 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.230 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.232 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.239 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-05-10T10:48:14.242 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.243 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.244 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.245 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.252 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.255 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T10:48:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-05-10T10:48:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.260 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.262 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.266 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.268 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.270 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.273 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.276 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.279 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.281 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-05-10T10:48:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.286 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.291 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.293 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.294 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.297 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.298 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.300 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.300 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.300 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.308 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:14.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-05-10T10:48:14.313 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.316 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.321 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.322 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.327 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.329 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.330 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-05-10T10:48:14.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-10T10:48:14.338 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-10T10:48:14.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.339 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-05-10T10:48:14.339 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.346 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-05-10T10:48:14.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T10:48:14.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-05-10T10:48:14.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.355 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.359 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.360 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.361 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.364 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.369 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.370 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.373 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.377 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.378 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.380 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-05-10T10:48:14.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-05-10T10:48:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-05-10T10:48:14.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-10T10:48:14.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-10T10:48:14.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-05-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T10:48:14.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.405 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.412 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.429 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-05-10T10:48:14.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-05-10T10:48:14.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-05-10T10:48:14.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-05-10T10:48:14.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-10T10:48:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-05-10T10:48:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-05-10T10:48:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-05-10T10:48:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-05-10T10:48:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.463 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.464 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T10:48:14.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T10:48:14.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-10T10:48:14.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-10T10:48:14.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-10T10:48:14.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-10T10:48:14.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.481 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.482 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "services": { 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard": "https://172.21.15.112:8443/" 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "always_on_modules": { 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus": [ 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-10T10:48:14.484 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific": [ 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.485 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy": [ 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "reef": [ 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-10T10:48:14.486 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "squid": [ 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-10T10:48:14.487 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "last_failure_osd_epoch": 3, 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "active_clients": [ 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-10T10:48:14.488 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1078512171 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "libcephsqlite", 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3091790545 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3685786090 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-10T10:48:14.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-10T10:48:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-10T10:48:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4212086858 2024-05-10T10:48:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:14.491 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-10T10:48:14.491 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:13 smithi195 ceph-mon[33773]: pgmap v669: 129 pgs: 129 active+clean; 63 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 586 op/s 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:13 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1285126482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:13 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3932170610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:13 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/4007213989' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:13 smithi148 ceph-mon[33568]: pgmap v669: 129 pgs: 129 active+clean; 63 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 586 op/s 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:13 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1285126482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:13 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3932170610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T10:48:14.492 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:13 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/4007213989' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.980+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99d40b27f0 msgr2=0x7f99d40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99d40b27f0 0x7f99d40b4cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99e819c310 tx=0x7f99d000b390 comp rx=0 tx=0).stop 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8102a00 msgr2=0x7f99e819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8102a00 0x7f99e819ae00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f99d800b450 tx=0x7f99d800b910 comp rx=0 tx=0).stop 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 shutdown_connections 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f99d40b27f0 0x7f99d40b4cb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99e81045b0 0x7f99e81a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e8103bf0 0x7f99e819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 --2- 172.21.15.112:0/4007213989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8102a00 0x7f99e819ae00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 >> 172.21.15.112:0/4007213989 conn(0x7f99e80fdf90 msgr2=0x7f99e8074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 shutdown_connections 2024-05-10T10:48:14.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:12.981+0000 7f99ecb5f640 1 -- 172.21.15.112:0/4007213989 wait complete. 2024-05-10T10:48:14.501 DEBUG:tasks.check_counter:Getting stats from a 2024-05-10T10:48:14.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-10T10:48:14.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:13 smithi112 ceph-mon[30100]: pgmap v669: 129 pgs: 129 active+clean; 63 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 586 op/s 2024-05-10T10:48:14.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:13 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1285126482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T10:48:14.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:13 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3932170610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T10:48:14.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:13 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/4007213989' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T10:48:14.758 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:15.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:15 smithi148 ceph-mon[33568]: pgmap v670: 129 pgs: 129 active+clean; 52 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 847 KiB/s wr, 513 op/s 2024-05-10T10:48:15.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:15 smithi148 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:15.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:15 smithi195 ceph-mon[33773]: pgmap v670: 129 pgs: 129 active+clean; 52 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 847 KiB/s wr, 513 op/s 2024-05-10T10:48:15.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:15 smithi195 ceph-mon[33773]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:15.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:15 smithi112 ceph-mon[30100]: pgmap v670: 129 pgs: 129 active+clean; 52 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 847 KiB/s wr, 513 op/s 2024-05-10T10:48:15.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:15 smithi112 ceph-mon[30100]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T10:48:16.433 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1372, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1372, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1117184, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 435473, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.099998267, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.032263766, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.027212826, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.031600636, 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1371, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.034695822, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000025306 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 730, 2024-05-10T10:48:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 707, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 785690, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 284810, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.068546653, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.022755833, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.025249425, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.019200858, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 706, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.022111939, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000031320 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 663760, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 127968 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 157, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 428, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 333726, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.010607511, 2024-05-10T10:48:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.007941732, 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001404942, 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000017622, 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 155, 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.007544959, 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000048677 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 333408 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-05-10T10:48:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3177, 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.074649812, 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000023496 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 172, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 172, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 172, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-05-10T10:48:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 1, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-05-10T10:48:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T10:48:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-05-10T10:48:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-05-10T10:48:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 382, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-05-10T10:48:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 2, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 4900537, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 4899938, 2024-05-10T10:48:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 382 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 4, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 190, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 186, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 2, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 21, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 19, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 172, 2024-05-10T10:48:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 406, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 234, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 51176, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215416 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-05-10T10:48:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-05-10T10:48:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.456 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-05-10T10:48:16.457 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.457 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.457 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.457 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-05-10T10:48:16.457 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.457 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.458 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.461 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.462 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 1, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 0, 2024-05-10T10:48:16.464 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1669, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:16.466 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:16.467 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 189339, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 50, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 170, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:16.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 133144, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 717, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 1862, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 21, 2024-05-10T10:48:16.469 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 3, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 1862, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1862, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.332964368, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000715877 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1862, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1863 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 1862, 2024-05-10T10:48:16.470 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 1862, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 1164, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 696, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-05-10T10:48:16.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-05-10T10:48:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 90, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 2, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 5, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 5, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-05-10T10:48:16.474 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-05-10T10:48:16.475 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2100, 2024-05-10T10:48:16.476 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1699273, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2100, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2100, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1699273, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.477 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:16.478 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 1862, 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 706238, 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1862, 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 706238, 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:16.479 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 1862, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1862, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1862, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1862, 2024-05-10T10:48:16.480 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:16.481 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.482 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55b613c326e0": { 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:16.483 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:16.484 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:48:17.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:17 smithi112 ceph-mon[30100]: pgmap v671: 129 pgs: 129 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 865 KiB/s wr, 487 op/s 2024-05-10T10:48:17.346 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-10T10:48:17.346 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-10T10:48:17.346 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-10T10:48:17.346 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-10T10:48:17.347 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-05-10T10:48:17.347 DEBUG:tasks.check_counter:Getting stats from d 2024-05-10T10:48:17.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-10T10:48:17.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:17 smithi148 ceph-mon[33568]: pgmap v671: 129 pgs: 129 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 865 KiB/s wr, 487 op/s 2024-05-10T10:48:17.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:17 smithi195 ceph-mon[33773]: pgmap v671: 129 pgs: 129 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 865 KiB/s wr, 487 op/s 2024-05-10T10:48:17.605 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:19.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:19 smithi112 ceph-mon[30100]: pgmap v672: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 414 KiB/s wr, 375 op/s 2024-05-10T10:48:19.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:19 smithi148 ceph-mon[33568]: pgmap v672: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 414 KiB/s wr, 375 op/s 2024-05-10T10:48:19.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:19 smithi195 ceph-mon[33773]: pgmap v672: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 414 KiB/s wr, 375 op/s 2024-05-10T10:48:20.751 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:48:20.751 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:20.751 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 216, 2024-05-10T10:48:20.751 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 213, 2024-05-10T10:48:20.751 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 512848, 2024-05-10T10:48:20.751 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 130624, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.029467771, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.011983487, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.013597602, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.003755346, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 212, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.008064537, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000038040 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:20.752 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 512848, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 130624 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 162, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 160, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 107024, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-05-10T10:48:20.753 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.010903394, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.008576025, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000966394, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 160, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.006256262, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000039101 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:20.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 107024 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 2944, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.000635377, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000041319, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000575676, 2024-05-10T10:48:20.755 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000698099, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000232699 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.756 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:20.757 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:20.758 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:20.759 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:20.759 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:20.759 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:20.759 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:20.759 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148985, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 217, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 482324, 2024-05-10T10:48:20.762 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 217, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 217, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 482324, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:20.763 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:48:21.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:21 smithi112 ceph-mon[30100]: pgmap v673: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 414 KiB/s wr, 317 op/s 2024-05-10T10:48:21.461 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-10T10:48:21.462 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-10T10:48:21.462 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-10T10:48:21.462 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-10T10:48:21.462 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-10T10:48:21.462 DEBUG:tasks.check_counter:Getting stats from g 2024-05-10T10:48:21.462 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-10T10:48:21.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:21 smithi148 ceph-mon[33568]: pgmap v673: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 414 KiB/s wr, 317 op/s 2024-05-10T10:48:21.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:21 smithi195 ceph-mon[33773]: pgmap v673: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 414 KiB/s wr, 317 op/s 2024-05-10T10:48:21.713 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:23.422 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:23 smithi112 ceph-mon[30100]: pgmap v674: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 414 KiB/s wr, 228 op/s 2024-05-10T10:48:23.422 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:48:23.422 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:23.422 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 682, 2024-05-10T10:48:23.422 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 684, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 229097, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 216877, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.067647695, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.027395972, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.015257277, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.018514092, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 682, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.027232342, 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000039930 2024-05-10T10:48:23.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1201, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1359, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 995408, 2024-05-10T10:48:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 718825, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.078546401, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.028409729, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.018764419, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.025307898, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1357, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.035948861, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000026491 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 337168 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 247, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 222, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 668868, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 131301, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-10T10:48:23.426 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.030754570, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.012048376, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.016598326, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.003873973, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 222, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.009747407, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000043907 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 667520, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 130336 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3249, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.074867686, 2024-05-10T10:48:23.428 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000023043 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-05-10T10:48:23.429 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 151, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-05-10T10:48:23.430 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 151, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 150, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 3, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-05-10T10:48:23.431 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:23.432 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-05-10T10:48:23.433 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-05-10T10:48:23.434 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-05-10T10:48:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 374, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 2, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-05-10T10:48:23.436 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 4883392, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 4882254, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 374 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 9, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 169, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 160, 2024-05-10T10:48:23.437 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 7, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 31, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 24, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 151, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 337, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 186, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 51840, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215416 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-05-10T10:48:23.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:23.439 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.440 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-05-10T10:48:23.442 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.443 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.443 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.443 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-05-10T10:48:23.443 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.443 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.447 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.450 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 1, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 0, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1679, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:23.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:23.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:23.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 197480, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 55, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:23.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 170, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 135765, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 702, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-10T10:48:23.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 1885, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 21, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 3, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 1885, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1885, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.281965754, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000680087 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-05-10T10:48:23.456 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1885, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1886 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 1885, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 1885, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 1178, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 705, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-05-10T10:48:23.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-05-10T10:48:23.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-05-10T10:48:23.459 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 90, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 2, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 6, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 6, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-05-10T10:48:23.460 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:23.461 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2126, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1686848, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2126, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2126, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1686848, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-10T10:48:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 1885, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 689093, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1885, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 689093, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 1885, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1885, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1885, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1885, 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:23.465 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:23.466 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x5605b53d06e0": { 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-10T10:48:23.467 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:23.468 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:48:23.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:23 smithi148 ceph-mon[33568]: pgmap v674: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 414 KiB/s wr, 228 op/s 2024-05-10T10:48:23.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:23 smithi195 ceph-mon[33773]: pgmap v674: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 414 KiB/s wr, 228 op/s 2024-05-10T10:48:24.265 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-10T10:48:24.265 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-10T10:48:24.265 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-10T10:48:24.265 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-10T10:48:24.265 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-05-10T10:48:24.266 DEBUG:tasks.check_counter:Getting stats from j 2024-05-10T10:48:24.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-10T10:48:24.507 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:25.558 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:25 smithi148 ceph-mon[33568]: pgmap v675: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 KiB/s wr, 96 op/s 2024-05-10T10:48:25.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:25 smithi195 ceph-mon[33773]: pgmap v675: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 KiB/s wr, 96 op/s 2024-05-10T10:48:25.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:25 smithi112 ceph-mon[30100]: pgmap v675: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 KiB/s wr, 96 op/s 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-10T10:48:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001011918, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000157941, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 219, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 216, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 513664, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 132544, 2024-05-10T10:48:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.028912945, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.012353103, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.012971296, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.003976531, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 215, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.007436483, 2024-05-10T10:48:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000034588 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 513664, 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 132544 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5, 2024-05-10T10:48:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 166, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3104, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 107824, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.011483782, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.008678198, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001561257, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 165, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.007711209, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000046734 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 107824 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148984, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-10T10:48:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 220, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 482714, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 220, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 220, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 482714, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-10T10:48:26.240 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-10T10:48:26.972 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-10T10:48:26.973 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-10T10:48:26.973 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-10T10:48:26.973 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-10T10:48:26.973 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-10T10:48:26.973 DEBUG:tasks.check_counter:Getting stats from b 2024-05-10T10:48:26.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-10T10:48:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:48:27.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:27 smithi148 ceph-mon[33568]: pgmap v676: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 24 KiB/s wr, 45 op/s 2024-05-10T10:48:27.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:27 smithi195 ceph-mon[33773]: pgmap v676: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 24 KiB/s wr, 45 op/s 2024-05-10T10:48:27.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:27 smithi112 ceph-mon[30100]: pgmap v676: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 24 KiB/s wr, 45 op/s 2024-05-10T10:48:29.069 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-05-10T10:48:29.069 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:29.069 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 18725, 2024-05-10T10:48:29.069 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 17091, 2024-05-10T10:48:29.069 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 6422128, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 9668824, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 25, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.891337720, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.351492873, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.633231002, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.050987958, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 17088, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.306361555, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000017928 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:29.070 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 29120, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003584871, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000123 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 4528, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 1888 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2824, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2693, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1936258, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1167744, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 9, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 8, 2024-05-10T10:48:29.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.157896734, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.053827465, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.068082699, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.035148826, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2690, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.084222147, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000031309 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1264, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000214623, 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000169 2024-05-10T10:48:29.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 690336, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 140560 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 6814, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 7167, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 4214616, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 3291328, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 18, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 7, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.382872181, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.158948960, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.268489113, 2024-05-10T10:48:29.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.009019635, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7169, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.175523196, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000024483 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 12953, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002017522, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000155 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 338960 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.074 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3306, 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.198652653, 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000362568 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:29.075 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1102, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.090000064, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000081669 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 10693, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 10568, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10568, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.441527833, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000609531 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.076 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 125, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 18, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 2, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 5, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 11, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 14, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 1122, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 1, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 1, 2024-05-10T10:48:29.077 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 5, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 21838, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 2953, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 117, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 85, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 1, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 429, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 41, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 936, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 41, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 1049, 2024-05-10T10:48:29.078 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 3270, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 156, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 112533223, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 3270, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 43, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 43, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 3227, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 105, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 806, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 2, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 5190, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 175, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 18, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 1364 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 215, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 439, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 247, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 279, 2024-05-10T10:48:29.080 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 344, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 557, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 557, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:29.081 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 43, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 2855, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 2855, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-05-10T10:48:29.082 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 4, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 4, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 8311314, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 321, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.604747059, 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001883947 2024-05-10T10:48:29.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 26, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 1338, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 1312, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 56, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 294, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 238, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 185, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 2304, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 2119, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 1, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 1257, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 1256, 2024-05-10T10:48:29.084 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 68364, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223608 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 11822, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 44, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 10693, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 34, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 885, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 197, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.379484968, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001926319 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-05-10T10:48:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 51, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.020716959, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000406214 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.086 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 9655, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.108132360, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000218346 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-05-10T10:48:29.087 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.088 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-05-10T10:48:29.089 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.089 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.089 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.008354209, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001670841 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.090 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.009363093, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001560515 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 81, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.090970569, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001123093 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 15, 2024-05-10T10:48:29.091 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.727384316, 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.181825621 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.092 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5, 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.035989963, 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.007197992 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.093 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-05-10T10:48:29.094 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:29.095 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001240639, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001240639 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 552, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.059890757, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001920091 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:29.097 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 2, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 1, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 1292, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 1292, 2024-05-10T10:48:29.098 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 499, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:29.099 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:29.100 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 292124, 2024-05-10T10:48:29.101 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 120, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 170, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-05-10T10:48:29.102 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 283765, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 2603, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 4855, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 4235382, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 4855, 2024-05-10T10:48:29.103 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4855, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 8.752801468, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001802842 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4855, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4951 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 4855, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 3291, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 1564, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-05-10T10:48:29.104 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 21, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 342, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 24, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 1121, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 3270, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-05-10T10:48:29.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 2, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-05-10T10:48:29.106 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-05-10T10:48:29.107 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 107, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 34 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 10, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 5, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 1109, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 557, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:29.109 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 28359, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 10003140, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 28359, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 28359, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 10003140, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4855, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 4296055, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4855, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4296055, 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:29.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4855, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 4855, 2024-05-10T10:48:29.112 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4855, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4855, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 557, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 60673, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:29.113 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 557, 2024-05-10T10:48:29.114 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:29.114 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:29.114 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 21, 2024-05-10T10:48:29.114 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 60673, 2024-05-10T10:48:29.114 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x5604e6960500": { 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:29.115 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 2855, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 4117010, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 2855, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 321, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4117010, 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:29.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:29.117 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:29.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:29.117 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.117 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:29.117 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-05-10T10:48:29.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:29 smithi148 ceph-mon[33568]: pgmap v677: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-05-10T10:48:29.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:29 smithi195 ceph-mon[33773]: pgmap v677: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-05-10T10:48:29.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:29 smithi112 ceph-mon[30100]: pgmap v677: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-05-10T10:48:29.890 INFO:tasks.check_counter:Daemon mds.b mds.exported=41 2024-05-10T10:48:29.891 INFO:tasks.check_counter:Daemon mds.b mds.imported=41 2024-05-10T10:48:29.891 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=215 2024-05-10T10:48:29.891 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=439 2024-05-10T10:48:29.891 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-05-10T10:48:29.891 DEBUG:tasks.check_counter:Getting stats from e 2024-05-10T10:48:29.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-10T10:48:30.144 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:48:31.412 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:31 smithi148 ceph-mon[33568]: pgmap v678: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-10T10:48:31.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:31 smithi195 ceph-mon[33773]: pgmap v678: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-10T10:48:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:31 smithi112 ceph-mon[30100]: pgmap v678: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 934, 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 911, 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 897440, 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 352190, 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.101945537, 2024-05-10T10:48:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.038854589, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.032398326, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.024295783, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 910, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.037245323, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000040928 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 665936, 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 133088 2024-05-10T10:48:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 44, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 204, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 4207940, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 352400, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.015683595, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.007237915, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.005643573, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000982579, 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 203, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.011544408, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000056869 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 339936 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1211, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1213, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 12974415, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 385105, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.077646153, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.017406306, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.027872828, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.027453512, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1211, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.027387594, 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000022615 2024-05-10T10:48:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3341, 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.078267555, 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000023426 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 5134, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 5, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 5129, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 5126, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 1, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-05-10T10:48:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-05-10T10:48:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-05-10T10:48:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-05-10T10:48:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 7692, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 11, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-05-10T10:48:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 21053256, 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 21052118, 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 7692 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 19, 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 3764, 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 3745, 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 49, 2024-05-10T10:48:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 222, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 173, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 5134, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 7788, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 2654, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 61268, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215416 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-05-10T10:48:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-05-10T10:48:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-05-10T10:48:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-05-10T10:48:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-05-10T10:48:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-05-10T10:48:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1676, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 189383, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 50, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 170, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 3714884, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 20302, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 1939, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 21, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 3, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 1939, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1939, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.095952040, 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001080944 2024-05-10T10:48:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1939, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1940 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 1939, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 1939, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 1203, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 734, 2024-05-10T10:48:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 1, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-05-10T10:48:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-05-10T10:48:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 5, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 5, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-05-10T10:48:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 2185, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 17867700, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 2185, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2185, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 17867700, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1939, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 16859383, 2024-05-10T10:48:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1939, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 16859383, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1939, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 1939, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1939, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1939, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x564562de66e0": { 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:32.029 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-05-10T10:48:32.841 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-10T10:48:32.841 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-10T10:48:32.841 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-10T10:48:32.841 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-10T10:48:32.841 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-10T10:48:32.841 DEBUG:tasks.check_counter:Getting stats from h 2024-05-10T10:48:32.842 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-10T10:48:33.090 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:48:33.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:33 smithi148 ceph-mon[33568]: pgmap v679: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-10T10:48:33.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:33 smithi195 ceph-mon[33773]: pgmap v679: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-10T10:48:33.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:33 smithi112 ceph-mon[30100]: pgmap v679: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2553, 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2958, 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1474117, 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1726908, 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 15, 2024-05-10T10:48:34.919 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.171895188, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.065172561, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.119194241, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.009058649, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2958, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.063305838, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000021401 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:34.920 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4395, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000732780, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000166 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 682352, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 142656 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3191, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2297, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1196086, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1158671, 2024-05-10T10:48:34.921 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 21, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 7, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.184402767, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.071198773, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.121494388, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.013475427, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2293, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.079305884, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000034586 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4208, 2024-05-10T10:48:34.922 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000853845, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000202 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 341216 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2922, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2253, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1989246, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 873644, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 16, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 8, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.144162814, 2024-05-10T10:48:34.923 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.056180444, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.104003688, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.007000464, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2253, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.067860073, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000030119 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4412, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000703506, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000159 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:34.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 577, 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.219668628, 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000380708 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:34.925 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 299, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.020674177, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000069144 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 1850, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 374, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 374, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.399353798, 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001067790 2024-05-10T10:48:34.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 1476, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 27, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 7, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 9, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 11, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 15, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 139, 2024-05-10T10:48:34.927 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 7, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 2653, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 807, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 1476, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 97, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 5, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 14, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 6, 2024-05-10T10:48:34.928 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 167, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 588, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 156, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 112533223, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 588, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 28, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 28, 2024-05-10T10:48:34.929 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 560, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 28, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 158, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 76, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 27, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 6, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 294 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 15, 2024-05-10T10:48:34.930 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 724, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 396, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 534, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 23, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 145, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 145, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-05-10T10:48:34.931 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 5, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 374, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-05-10T10:48:34.932 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 374, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 2, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4883392, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-05-10T10:48:34.933 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 48, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.222758931, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004640811 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 34, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 249, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 215, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 32, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 401, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 369, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 27, 2024-05-10T10:48:34.934 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 481, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 454, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 152, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 152, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 57200, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223608 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 2118, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 13, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 1850, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 8, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 138, 2024-05-10T10:48:34.935 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.010274413, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002054882 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 13, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002685359, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000206566 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 203, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.070403593, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000346815 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000520468, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000520468 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-05-10T10:48:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.005025327, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000717903 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.075781165, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.012630194 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 139, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.234663473, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001688226 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 249, 2024-05-10T10:48:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 249, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 506, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 316890, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 129, 2024-05-10T10:48:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 170, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-05-10T10:48:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 136786, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 555, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 1000, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 725424, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 1000, 2024-05-10T10:48:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1000, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.103952272, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002103952 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1000, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1044 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 1000, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 594, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 406, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-05-10T10:48:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 6, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 54, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 16, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 296, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 588, 2024-05-10T10:48:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-05-10T10:48:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-05-10T10:48:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 40, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 14 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-05-10T10:48:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 10, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 6, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 284, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 145, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 8662, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 3862668, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 8662, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8662, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 3862668, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1000, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 741181, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1000, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 741181, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1000, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 1000, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1000, 2024-05-10T10:48:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1000, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 145, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 15757, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 145, 2024-05-10T10:48:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 6, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 15757, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x555a221d0500": { 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 374, 2024-05-10T10:48:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 689088, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 374, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 48, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 689088, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:34.964 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-05-10T10:48:35.544 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:35 smithi148 ceph-mon[33568]: pgmap v680: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-10T10:48:35.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:35 smithi195 ceph-mon[33773]: pgmap v680: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-10T10:48:35.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:35 smithi112 ceph-mon[30100]: pgmap v680: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-10T10:48:35.700 INFO:tasks.check_counter:Daemon mds.h mds.exported=5 2024-05-10T10:48:35.700 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2024-05-10T10:48:35.700 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=15 2024-05-10T10:48:35.700 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=724 2024-05-10T10:48:35.700 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-10T10:48:35.700 DEBUG:tasks.check_counter:Getting stats from k 2024-05-10T10:48:35.701 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-10T10:48:35.946 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T10:48:37.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:37 smithi148 ceph-mon[33568]: pgmap v681: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-10T10:48:37.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:37 smithi195 ceph-mon[33773]: pgmap v681: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-10T10:48:37.580 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:37 smithi112 ceph-mon[30100]: pgmap v681: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-10T10:48:37.634 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 255, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 230, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 671074, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 137060, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.035419571, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.012565076, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.018454971, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.004877640, 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:37.635 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 230, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.011352603, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000049359 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 670240, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 136736 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 606, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 772, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 149012, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 535819, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.056449276, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.020317278, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.012022683, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.018124911, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 770, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.027001826, 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000035067 2024-05-10T10:48:37.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 343424 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1442, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1444, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 4545511, 2024-05-10T10:48:37.638 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 458614, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.125672666, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.043305497, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.032901217, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.038810476, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1442, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.047448472, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000032904 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:37.639 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-05-10T10:48:37.640 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3429, 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.088917754, 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000025931 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-05-10T10:48:37.642 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-05-10T10:48:37.643 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 560, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 560, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 560, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 1, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-05-10T10:48:37.644 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-05-10T10:48:37.645 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:37.646 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:37.647 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-05-10T10:48:37.648 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 2855, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-05-10T10:48:37.649 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 4, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 8311314, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 8310176, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.650 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 2855 2024-05-10T10:48:37.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 6, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 1275, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 1269, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 4, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 91, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 87, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 560, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 1861, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 1301, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-05-10T10:48:37.654 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 52252, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215416 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.655 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.658 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.659 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.660 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-05-10T10:48:37.660 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.663 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.664 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.665 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.666 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.667 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-05-10T10:48:37.668 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.669 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 2, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 1, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1683, 2024-05-10T10:48:37.670 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:37.671 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:37.672 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 190185, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 53, 2024-05-10T10:48:37.673 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 170, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 413304, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 2291, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 2048, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 21, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 3, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 2048, 2024-05-10T10:48:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2048, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.355969040, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000662094 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2048, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2049 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 2048, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 2048, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-05-10T10:48:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 1298, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 748, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 1, 2024-05-10T10:48:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-05-10T10:48:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-05-10T10:48:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 5, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 5, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-05-10T10:48:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 2299, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 5142982, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 2299, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2299, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 5142982, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 2048, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 4117015, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2048, 2024-05-10T10:48:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4117015, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 2048, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 2048, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2048, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 2048, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x5582820646e0": { 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-05-10T10:48:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-05-10T10:48:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-05-10T10:48:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-05-10T10:48:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-05-10T10:48:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-05-10T10:48:37.688 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-05-10T10:48:38.421 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:38 smithi148 ceph-mon[33568]: pgmap v682: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-10T10:48:38.447 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-10T10:48:38.448 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-10T10:48:38.448 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-10T10:48:38.448 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-10T10:48:38.448 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-10T10:48:38.448 DEBUG:tasks.check_counter:Getting stats from c 2024-05-10T10:48:38.448 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-10T10:48:38.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:38 smithi195 ceph-mon[33773]: pgmap v682: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-10T10:48:38.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:38 smithi112 ceph-mon[30100]: pgmap v682: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-10T10:48:38.703 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:48:40.670 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 13816, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 16681, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 9277421, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 19670647, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 19, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.966407180, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.417825939, 2024-05-10T10:48:40.671 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.405657120, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.158782035, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 16680, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.844336932, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000050619 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 14598, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.002811632, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000192 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.672 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 30632, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 29748, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 11019839, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 16986015, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 15, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 1.562125155, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.657888239, 2024-05-10T10:48:40.673 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.977683353, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.106862233, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 29746, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.178312396, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000039612 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 44098, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.005799389, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000131 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.674 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 344896 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 14260, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 14976, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 8186710, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 9142023, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 18, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 8, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.883043563, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.347736739, 2024-05-10T10:48:40.675 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.248806210, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.226137824, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 14975, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.757164816, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000050561 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2993, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000479115, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000160 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.676 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 695584, 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 143728 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 13815, 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 7.818822711, 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000565966 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 6937, 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.133915912, 2024-05-10T10:48:40.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000163459 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 10492, 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 8025, 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 8025, 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 9.843428855, 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001226595 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-10T10:48:40.679 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 2467, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 13, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 2, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 48, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 19, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 12, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 17, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 20, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 40, 2024-05-10T10:48:40.680 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 2, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 2, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 4, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 14630, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 12057, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 2409, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 51, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-10T10:48:40.681 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 1, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 7, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 59, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 4012, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 18227, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 156, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 112533223, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 0, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 18227, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:40.682 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 199, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 199, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 18028, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 8, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 155, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 3854, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:40.683 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 13159, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 15, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 16, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 7503 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 652, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 114, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 69, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 198, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 956, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 7, 2024-05-10T10:48:40.684 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 3743, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 3736, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:40.685 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 1, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 2, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 1, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 7692, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-05-10T10:48:40.686 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 7692, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 11, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 11, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 21053256, 2024-05-10T10:48:40.687 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 163, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 3.062805615, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.018790218 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 30, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 3786, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 3756, 2024-05-10T10:48:40.688 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 59, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 282, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 223, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 48, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 7817, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 7769, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 2, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 3750, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 3748, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 83232, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 223608 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.689 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 10946, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 834, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 10492, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 8, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 1790, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 15, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.025050417, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001670027 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.690 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 300, 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.190653403, 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000635511 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.691 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3633, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2.394167480, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000659005 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.692 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:40.693 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.695 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 334, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.384289105, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001150566 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.697 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.698 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 46, 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.897589976, 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.019512825 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:40.699 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-10T10:48:40.701 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3697, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 5.951678474, 2024-05-10T10:48:40.702 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001609867 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 1, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 1, 2024-05-10T10:48:40.703 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 0, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 1, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 6925, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 6925, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 512, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:40.704 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:40.705 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:40.706 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 317542, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 134, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:40.707 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 170, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 188384, 2024-05-10T10:48:40.708 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 1882, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 26177, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 17403711, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 28, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 26177, 2024-05-10T10:48:40.709 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 26177, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 54.478959853, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002081176 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 26177, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 26256 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 26177, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 18279, 2024-05-10T10:48:40.710 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 7898, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 0, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 10, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 52, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 215, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 37, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 7580, 2024-05-10T10:48:40.711 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 18227, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 20, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-10T10:48:40.712 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-10T10:48:40.713 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 90, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 2, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 12, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 90, 2024-05-10T10:48:40.714 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 0, 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 25 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-10T10:48:40.715 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 36, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 9, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 7500, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 3736, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:40.716 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 58704, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 23183053, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 58704, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 58704, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 23183053, 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:40.717 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:40.718 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 26177, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 17810671, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 26177, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 17810671, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.719 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 26177, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 26177, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 26177, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 26177, 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:40.720 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 3736, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 406960, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 3736, 2024-05-10T10:48:40.721 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 52, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 406960, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x5572a77a0500": { 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:40.722 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 7692, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 16858952, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 7692, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 163, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 16858952, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:40.723 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:40.724 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:40.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:40.724 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-10T10:48:40.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:40 smithi148 ceph-mon[33568]: pgmap v683: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:48:40.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:40 smithi195 ceph-mon[33773]: pgmap v683: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:48:40.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:40 smithi112 ceph-mon[30100]: pgmap v683: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:48:41.423 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-05-10T10:48:41.424 INFO:tasks.check_counter:Daemon mds.c mds.imported=59 2024-05-10T10:48:41.424 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=652 2024-05-10T10:48:41.424 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=114 2024-05-10T10:48:41.424 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-10T10:48:41.424 DEBUG:tasks.check_counter:Getting stats from f 2024-05-10T10:48:41.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-10T10:48:41.688 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:48:42.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:42 smithi195 ceph-mon[33773]: pgmap v684: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-10T10:48:42.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:42 smithi112 ceph-mon[30100]: pgmap v684: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-10T10:48:43.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:42 smithi148 ceph-mon[33568]: pgmap v684: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 3192, 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 2166, 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1235760, 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 1525673, 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 19, 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-05-10T10:48:43.480 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.204983325, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.087817512, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.124599937, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.015269591, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2166, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.105848975, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000048868 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4189, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000806415, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000192 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.481 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1397, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1485, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1279639, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 649082, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 10, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 8, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.096846113, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.033750163, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.051316023, 2024-05-10T10:48:43.482 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.015480633, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1483, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.080394045, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000054210 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1272, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000244087, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000191 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 689392, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 145424 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.483 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 4663, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 4537, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 2399664, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 1799436, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 23, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 8, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.303931413, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.119845821, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.227519227, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.004261807, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4535, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.132660619, 2024-05-10T10:48:43.484 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000029252 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 8427, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001312123, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000155 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 345136 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.485 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 647, 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.267306231, 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000413147 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 325, 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.021835049, 2024-05-10T10:48:43.486 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000067184 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 2228, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 358, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 358, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.713187357, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001992143 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 1870, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:43.487 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 11, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 19, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 4, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 5, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 10, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 13, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 223, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 5, 2024-05-10T10:48:43.488 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 2986, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 818, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 1870, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 98, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 5, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 80, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 5, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 219, 2024-05-10T10:48:43.489 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 727, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 156, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 112533223, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 0, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 727, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 4, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 4, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 723, 2024-05-10T10:48:43.490 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 22, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 177, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 201, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 1, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 9, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 342 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 14, 2024-05-10T10:48:43.491 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 622, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 348, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 452, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 18, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 171, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 171, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:43.492 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 5, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 383, 2024-05-10T10:48:43.493 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 383, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 2, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 2, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-05-10T10:48:43.494 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 4901675, 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 36, 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.090668669, 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002518574 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.495 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 26, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 284, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 258, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 24, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 340, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 316, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 19, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 552, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 533, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 180, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 180, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 57084, 2024-05-10T10:48:43.496 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 223608 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 2540, 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 13, 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 2228, 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 8, 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 192, 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.497 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001291166, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000430388 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-10T10:48:43.498 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 175, 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.245744494, 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001404254 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:43.499 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.501 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.502 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.502 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-10T10:48:43.502 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.503 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.008585966, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000953996 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.061183398, 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.061183398 2024-05-10T10:48:43.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.506 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-10T10:48:43.507 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.508 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 170, 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.396382333, 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002331660 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.509 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 1, 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 1, 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 0, 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 1, 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 313, 2024-05-10T10:48:43.510 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 313, 2024-05-10T10:48:43.511 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 515, 2024-05-10T10:48:43.511 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:43.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.511 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-10T10:48:43.511 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:43.511 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:43.512 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:43.513 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:43.514 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 284379, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 118, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-10T10:48:43.515 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 170, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 103317, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 415, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-10T10:48:43.516 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 1180, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 744810, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 28, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 1180, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1180, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2.955943522, 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002505036 2024-05-10T10:48:43.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1180, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1224 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 1180, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 732, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 448, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 0, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 10, 2024-05-10T10:48:43.518 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 5, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 41, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 15, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 352, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 727, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 20, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:43.519 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-10T10:48:43.520 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 90, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 2, 2024-05-10T10:48:43.521 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 12, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 42, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 0, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 12 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:43.522 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 10, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 5, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 340, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 171, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.523 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 9248, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 4065394, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 9248, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 9248, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 4065394, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.524 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 1180, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 763433, 2024-05-10T10:48:43.525 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 1180, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 763433, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:43.526 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 1180, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 1180, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 1180, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1180, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.527 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 171, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 18623, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 171, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 5, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 18623, 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:43.528 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55f79ba38500": { 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 383, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 707371, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 383, 2024-05-10T10:48:43.529 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 36, 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 707371, 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.530 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:43.531 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-10T10:48:44.195 INFO:tasks.check_counter:Daemon mds.f mds.exported=5 2024-05-10T10:48:44.195 INFO:tasks.check_counter:Daemon mds.f mds.imported=5 2024-05-10T10:48:44.195 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=14 2024-05-10T10:48:44.196 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=622 2024-05-10T10:48:44.196 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-10T10:48:44.196 DEBUG:tasks.check_counter:Getting stats from i 2024-05-10T10:48:44.196 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-10T10:48:44.439 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:48:44.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:44 smithi195 ceph-mon[33773]: pgmap v685: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T10:48:44.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:44 smithi112 ceph-mon[30100]: pgmap v685: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T10:48:45.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:44 smithi148 ceph-mon[33568]: pgmap v685: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 37846, 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 36043, 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 11009302, 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 48872327, 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 10, 2024-05-10T10:48:46.234 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 2.302304608, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.773646885, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.763877308, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.704276593, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 36047, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 5.234419824, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000145210 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 13942, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.002163833, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000155 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 29302, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 30491, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 7855768, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 35445877, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 9, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 7, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 1.875433826, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.577738016, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.515837491, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.660258202, 2024-05-10T10:48:46.236 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 30489, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 5.611933888, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000184064 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4165, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000824546, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000197 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 370544 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 63734, 2024-05-10T10:48:46.237 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 110750, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 20503023, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 64914866, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 154, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 11, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 4.291196002, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 1.636038804, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 2.770361346, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.512380866, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 110744, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 5.910553442, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000053371 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.238 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 124282, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.022829520, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000183 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 769856, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 182640 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:46.239 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 91956, 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 23.739452863, 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000258160 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1093, 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.239026485, 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000218688 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.240 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 23073, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 17474, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 17474, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 55.484172652, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.003175241 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 5599, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 6, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 1, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 726, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 22, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-05-10T10:48:46.241 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 29, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 9, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 9, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 11, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 16, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 4683, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 2, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 2, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 7, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 26933, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 18891, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 5574, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 59, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:46.242 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 3, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 10, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 64, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 4563, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 5, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 153, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 2905, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 156, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 112533223, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 0, 2024-05-10T10:48:46.243 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 2905, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 136, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 9, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 127, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 2769, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 9555, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 14052, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 7982, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 15943, 2024-05-10T10:48:46.244 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 13065, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 31, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 15288 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 52, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 275, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 166, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 212, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 336, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 553, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 550, 2024-05-10T10:48:46.245 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 3, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 9, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 64, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 22, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-10T10:48:46.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 37505, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 37458, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 37458, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 47, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 100, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 72, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 71, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 71, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 1, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 1, 2024-05-10T10:48:46.247 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 1, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 80067669, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 80164348, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 25311, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 53.538774419, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002115237 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 35, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 5182, 2024-05-10T10:48:46.248 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 5147, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 31, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 400, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 369, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 29, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 5453, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 5424, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 2, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 5097, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 5095, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 97872, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 231800 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 41660, 2024-05-10T10:48:46.249 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 271, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 23073, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 85, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 512, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4880, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 14.272045372, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002924599 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1292, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.811921861, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000628422 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.250 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.005549660, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001849886 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1335, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.117010653, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000836712 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.251 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.252 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.253 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-10T10:48:46.254 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 68, 2024-05-10T10:48:46.255 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.074977404, 2024-05-10T10:48:46.255 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001102608 2024-05-10T10:48:46.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.255 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-10T10:48:46.255 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.255 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 161, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.149272028, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000927155 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.258 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 80, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.602571607, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.007532145 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 77, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.174961511, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002272227 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.082401828, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.009155758 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.260 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9063, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 36.795249855, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.004059941 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 8, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.122037103, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.015254637 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.262 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 8, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.024799406, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.003099925 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.004230759, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000846151 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 485, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.247143605, 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002571430 2024-05-10T10:48:46.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 1, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 3, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 2, 2024-05-10T10:48:46.264 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 1, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 1971, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 1971, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 518, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:46.265 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:46.266 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:46.267 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 307525, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 125, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 170, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:46.268 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 143485, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 743, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 77391, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 84793543, 2024-05-10T10:48:46.269 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 32, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 77391, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 77391, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 3172.944463118, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.040998881 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 77391, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 91646 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 77391, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 2977, 2024-05-10T10:48:46.270 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 74414, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 476, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 8578, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 56, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 25358, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 155, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 6, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 1677, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-10T10:48:46.271 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 2921, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 12909, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 37740, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-10T10:48:46.272 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-10T10:48:46.273 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 90, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 11, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 12, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 1668, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 14, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 88 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.274 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 31, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 6, 2024-05-10T10:48:46.275 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 1113, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 550, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 130878, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 27564180, 2024-05-10T10:48:46.276 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 130878, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 130878, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 27564180, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:46.277 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 77391, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 84853281, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 77391, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 84853281, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 77391, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 77391, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 77391, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 77391, 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:46.279 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 550, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 59838, 2024-05-10T10:48:46.280 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 550, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 47, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 59838, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55b6bfb0a500": { 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:46.281 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 37505, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 75970044, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 37505, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 25311, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 75970044, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.282 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:46.283 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-10T10:48:46.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:46 smithi112 ceph-mon[30100]: pgmap v686: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-10T10:48:47.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:46 smithi195 ceph-mon[33773]: pgmap v686: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-10T10:48:47.041 INFO:tasks.check_counter:Daemon mds.i mds.exported=64 2024-05-10T10:48:47.042 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-05-10T10:48:47.042 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=52 2024-05-10T10:48:47.042 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=275 2024-05-10T10:48:47.042 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=22 2024-05-10T10:48:47.042 DEBUG:tasks.check_counter:Getting stats from l 2024-05-10T10:48:47.042 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-10T10:48:47.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:46 smithi148 ceph-mon[33568]: pgmap v686: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-10T10:48:47.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:47 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:48:47.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:47 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:48:47.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:47 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T10:48:47.923 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T10:48:48.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:48 smithi112 ceph-mon[30100]: pgmap v687: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:48:48.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:48.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:48.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:48.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:48 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:48 smithi148 ceph-mon[33568]: pgmap v687: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:48:49.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:48 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:48 smithi195 ceph-mon[33773]: pgmap v687: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-10T10:48:49.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:48 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:49.657 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-10T10:48:49.657 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T10:48:49.657 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1464, 2024-05-10T10:48:49.657 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1466, 2024-05-10T10:48:49.657 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 33987399, 2024-05-10T10:48:49.657 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 465562, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 7, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.133198115, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.035498437, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.048605050, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.038227434, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1464, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.045006305, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000030742 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:49.658 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 556, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 531, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 21719168, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 233968, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 5, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-05-10T10:48:49.659 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.066113243, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.018889777, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.034777279, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.012099718, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 531, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.021974779, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000041383 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.660 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 671488, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 139456 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 750, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 919, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 21632049, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 585367, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.076999220, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.026540269, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.024455247, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.019564382, 2024-05-10T10:48:49.661 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 917, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.031862240, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000034746 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 347184 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-10T10:48:49.662 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4251, 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.133934927, 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000031506 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.664 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 571, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 382, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 189, 2024-05-10T10:48:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 24, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 3, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-10T10:48:49.666 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 156, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 112533223, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-10T10:48:49.667 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-10T10:48:49.669 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-05-10T10:48:49.670 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 37505, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 2, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 37, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 80067669, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 80164348, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 80162436, 2024-05-10T10:48:49.671 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 37505 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 19, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 5096, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 5077, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 15, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 270, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 255, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 571, 2024-05-10T10:48:49.672 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 5192, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 4621, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 67404, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215416 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-10T10:48:49.673 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.674 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.675 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.676 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.678 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.679 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.680 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.681 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.682 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.683 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-10T10:48:49.684 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 1, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 3, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 2, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 1, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 1701, 2024-05-10T10:48:49.685 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T10:48:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-10T10:48:49.687 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 223249, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 70, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-10T10:48:49.688 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 8888, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 170, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 463901, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 2505, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-10T10:48:49.689 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 2510, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 26, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 4, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 2510, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2510, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.522970626, 2024-05-10T10:48:49.690 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000606761 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2510, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2511 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 2510, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 2510, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 1417, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 1090, 2024-05-10T10:48:49.691 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-10T10:48:49.692 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 2, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-10T10:48:49.693 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-10T10:48:49.694 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 90, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 2, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 9, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 9, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-10T10:48:49.695 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-10T10:48:49.696 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 2766, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 77073717, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 2766, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:49.697 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2766, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 77073717, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:49.698 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2510, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 75971594, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2510, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 75971594, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-10T10:48:49.699 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2510, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 2510, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2510, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2510, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.700 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-05-10T10:48:49.701 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55667cb9e6e0": { 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-10T10:48:49.702 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-10T10:48:49.703 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-10T10:48:50.478 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-10T10:48:50.478 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-10T10:48:50.478 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-10T10:48:50.479 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-10T10:48:50.479 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-05-10T10:48:50.479 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-10T10:48:50.489 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-10T10:48:50.490 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:48:50.490 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:48:50.511 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:48:50.511 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:48:50.573 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T10:48:50.816 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:50 smithi195 ceph-mon[33773]: pgmap v688: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:48:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:50 smithi112 ceph-mon[30100]: pgmap v688: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:48:50.879 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:51.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:50 smithi148 ceph-mon[33568]: pgmap v688: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-10T10:48:52.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:48:52.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:48:52.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:51 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:48:52.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:48:52.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:51 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T10:48:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T10:48:52.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:51 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' 2024-05-10T10:48:52.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/404473646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102390 msgr2=0x7fdc54102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/404473646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102390 0x7fdc54102810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009990 tx=0x7fdc44009ca0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/404473646 shutdown_connections 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/404473646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc54102d50 0x7fdc5410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/404473646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc54108400 0x7fdc541087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/404473646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102390 0x7fdc54102810 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/404473646 >> 172.21.15.112:0/404473646 conn(0x7fdc54079310 msgr2=0x7fdc54079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/404473646 shutdown_connections 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.647+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/404473646 wait complete. 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 Processor -- start 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 -- start start 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc54102390 0x7fdc5419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102d50 0x7fdc5419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc54108400 0x7fdc54199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54113520 con 0x7fdc54102d50 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541133a0 con 0x7fdc54102390 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5c7ec700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541136a0 con 0x7fdc54108400 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102d50 0x7fdc5419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102d50 0x7fdc5419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51342/0 (socket says 172.21.15.112:51342) 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 -- 172.21.15.112:0/2900938369 learned_addr learned my addr 172.21.15.112:0/2900938369 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5a588700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc54102390 0x7fdc5419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc5ad89700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc54108400 0x7fdc54199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 -- 172.21.15.112:0/2900938369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc54108400 msgr2=0x7fdc54199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc54108400 0x7fdc54199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 -- 172.21.15.112:0/2900938369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc54102390 msgr2=0x7fdc5419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc54102390 0x7fdc5419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.648+0000 7fdc59d87700 1 -- 172.21.15.112:0/2900938369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5419a0c0 con 0x7fdc54102d50 2024-05-10T10:48:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.649+0000 7fdc59d87700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102d50 0x7fdc5419fce0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdc44006010 tx=0x7fdc440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:52.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.649+0000 7fdc4b7fe700 1 -- 172.21.15.112:0/2900938369 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44027070 con 0x7fdc54102d50 2024-05-10T10:48:52.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.649+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5419a350 con 0x7fdc54102d50 2024-05-10T10:48:52.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.649+0000 7fdc4b7fe700 1 -- 172.21.15.112:0/2900938369 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc44022470 con 0x7fdc54102d50 2024-05-10T10:48:52.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.649+0000 7fdc4b7fe700 1 -- 172.21.15.112:0/2900938369 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44017650 con 0x7fdc54102d50 2024-05-10T10:48:52.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.649+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc541a6670 con 0x7fdc54102d50 2024-05-10T10:48:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.650+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5406a190 con 0x7fdc54102d50 2024-05-10T10:48:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.652+0000 7fdc4b7fe700 1 -- 172.21.15.112:0/2900938369 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdc44009d10 con 0x7fdc54102d50 2024-05-10T10:48:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.653+0000 7fdc4b7fe700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdc400b2880 0x7fdc400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.653+0000 7fdc5a588700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdc400b2880 0x7fdc400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.653+0000 7fdc4b7fe700 1 -- 172.21.15.112:0/2900938369 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fdc440e3370 con 0x7fdc54102d50 2024-05-10T10:48:52.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.653+0000 7fdc5a588700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdc400b2880 0x7fdc400b4d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdc5419aaa0 tx=0x7fdc50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:52.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.656+0000 7fdc4b7fe700 1 -- 172.21.15.112:0/2900938369 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdc44013070 con 0x7fdc54102d50 2024-05-10T10:48:52.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.877+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc5419ae90 con 0x7fdc54102d50 2024-05-10T10:48:52.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.877+0000 7fdc4b7fe700 1 -- 172.21.15.112:0/2900938369 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+602 (secure 0 0 0) 0x7fdc440aae30 con 0x7fdc54102d50 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3420175550 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2112593785 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1230086877 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3235282798 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2711978475 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:52.879 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3592023612 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdc400b2880 msgr2=0x7fdc400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdc400b2880 0x7fdc400b4d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdc5419aaa0 tx=0x7fdc50009260 comp rx=0 tx=0).stop 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102d50 msgr2=0x7fdc5419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102d50 0x7fdc5419fce0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdc44006010 tx=0x7fdc440047b0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 shutdown_connections 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fdc400b2880 0x7fdc400b4d40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc54108400 0x7fdc54199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc54102390 0x7fdc5419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 --2- 172.21.15.112:0/2900938369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc54102d50 0x7fdc5419fce0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 >> 172.21.15.112:0/2900938369 conn(0x7fdc54079310 msgr2=0x7fdc541011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 shutdown_connections 2024-05-10T10:48:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:52.880+0000 7fdc5c7ec700 1 -- 172.21.15.112:0/2900938369 wait complete. 2024-05-10T10:48:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-05-10T10:48:53.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:52 smithi148 ceph-mon[33568]: pgmap v689: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-10T10:48:53.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:52 smithi195 ceph-mon[33773]: pgmap v689: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-10T10:48:53.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:52 smithi112 ceph-mon[30100]: pgmap v689: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-10T10:48:53.588 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-10T10:48:53.588 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:48:53.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T10:48:53.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:53 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2900938369' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:48:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:48:54.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:53 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2900938369' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:48:54.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:53 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2900938369' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:48:55.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:54 smithi148 ceph-mon[33568]: pgmap v690: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:48:55.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:54 smithi195 ceph-mon[33773]: pgmap v690: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:48:55.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:54 smithi112 ceph-mon[30100]: pgmap v690: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 -- 172.21.15.112:0/2351971079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ee0 msgr2=0x7f0990110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 --2- 172.21.15.112:0/2351971079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ee0 0x7f0990110370 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f098800b0a0 tx=0x7f098800b3b0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 -- 172.21.15.112:0/2351971079 shutdown_connections 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 --2- 172.21.15.112:0/2351971079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ee0 0x7f0990110370 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 --2- 172.21.15.112:0/2351971079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0990109530 0x7f0990109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 --2- 172.21.15.112:0/2351971079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0990103520 0x7f09901039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 -- 172.21.15.112:0/2351971079 >> 172.21.15.112:0/2351971079 conn(0x7f099007a650 msgr2=0x7f099007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 -- 172.21.15.112:0/2351971079 shutdown_connections 2024-05-10T10:48:55.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.691+0000 7f0997810700 1 -- 172.21.15.112:0/2351971079 wait complete. 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 Processor -- start 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 -- start start 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103520 0x7f099019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0990103ee0 0x7f099019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0990109530 0x7f0990199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0990113510 con 0x7f0990103520 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0990113390 con 0x7f0990109530 2024-05-10T10:48:55.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0997810700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0990113690 con 0x7f0990103ee0 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f09955ac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103520 0x7f099019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f09955ac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103520 0x7f099019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38662/0 (socket says 172.21.15.112:38662) 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f09955ac700 1 -- 172.21.15.112:0/3070867915 learned_addr learned my addr 172.21.15.112:0/3070867915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.692+0000 7f0995dad700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0990109530 0x7f0990199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f0994dab700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0990103ee0 0x7f099019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09955ac700 1 -- 172.21.15.112:0/3070867915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0990103ee0 msgr2=0x7f099019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09955ac700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0990103ee0 0x7f099019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09955ac700 1 -- 172.21.15.112:0/3070867915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0990109530 msgr2=0x7f0990199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09955ac700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0990109530 0x7f0990199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09955ac700 1 -- 172.21.15.112:0/3070867915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099019a030 con 0x7f0990103520 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f0994dab700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0990103ee0 0x7f099019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09955ac700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103520 0x7f099019f680 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c9b0 tx=0x7f098c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09867fc700 1 -- 172.21.15.112:0/3070867915 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c007b00 con 0x7f0990103520 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09867fc700 1 -- 172.21.15.112:0/3070867915 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f098c007ca0 con 0x7f0990103520 2024-05-10T10:48:55.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099019a2c0 con 0x7f0990103520 2024-05-10T10:48:55.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f09867fc700 1 -- 172.21.15.112:0/3070867915 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c018660 con 0x7f0990103520 2024-05-10T10:48:55.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.693+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09901a6600 con 0x7f0990103520 2024-05-10T10:48:55.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.694+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0990104c10 con 0x7f0990103520 2024-05-10T10:48:55.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.696+0000 7f09867fc700 1 -- 172.21.15.112:0/3070867915 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f098c00f450 con 0x7f0990103520 2024-05-10T10:48:55.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.697+0000 7f09867fc700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f097c0b2880 0x7f097c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:48:55.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.697+0000 7f0994dab700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f097c0b2880 0x7f097c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:48:55.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.697+0000 7f09867fc700 1 -- 172.21.15.112:0/3070867915 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f098c0e2790 con 0x7f0990103520 2024-05-10T10:48:55.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.697+0000 7f0994dab700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f097c0b2880 0x7f097c0b4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0980005fd0 tx=0x7f0980005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:48:55.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.700+0000 7f09867fc700 1 -- 172.21.15.112:0/3070867915 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f098c014030 con 0x7f0990103520 2024-05-10T10:48:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.919+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f099019ac80 con 0x7f0990103520 2024-05-10T10:48:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.919+0000 7f09867fc700 1 -- 172.21.15.112:0/3070867915 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+602 (secure 0 0 0) 0x7f098c0aa1b0 con 0x7f0990103520 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3420175550 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2112593785 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2836147288 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1230086877 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3235282798 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2711978475 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1796554484 2024-05-11T10:28:06.482690+0000 2024-05-10T10:48:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3592023612 2024-05-11T10:28:33.178665+0000 2024-05-10T10:48:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f097c0b2880 msgr2=0x7f097c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f097c0b2880 0x7f097c0b4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0980005fd0 tx=0x7f0980005e30 comp rx=0 tx=0).stop 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103520 msgr2=0x7f099019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103520 0x7f099019f680 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c9b0 tx=0x7f098c00ccc0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 shutdown_connections 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0990109530 0x7f0990199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103520 0x7f099019f680 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f097c0b2880 0x7f097c0b4d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 --2- 172.21.15.112:0/3070867915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0990103ee0 0x7f099019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 >> 172.21.15.112:0/3070867915 conn(0x7f099007a650 msgr2=0x7f0990102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.921+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 shutdown_connections 2024-05-10T10:48:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:48:55.922+0000 7f0997810700 1 -- 172.21.15.112:0/3070867915 wait complete. 2024-05-10T10:48:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-05-10T10:48:56.715 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-05-10T10:48:56.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:48:56.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-10T10:48:56.782 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[87278]: fuse finished with error 0 and tester_r 0 2024-05-10T10:48:56.783 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:48:57.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:56 smithi148 ceph-mon[33568]: pgmap v691: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-10T10:48:57.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:56 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3070867915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:48:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:56 smithi195 ceph-mon[33773]: pgmap v691: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-10T10:48:57.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:56 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3070867915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:48:57.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:56 smithi112 ceph-mon[30100]: pgmap v691: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-10T10:48:57.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:56 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3070867915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T10:48:59.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:48:58 smithi148 ceph-mon[33568]: pgmap v692: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-10T10:48:59.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:48:58 smithi195 ceph-mon[33773]: pgmap v692: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-10T10:48:59.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:48:58 smithi112 ceph-mon[30100]: pgmap v692: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-10T10:49:01.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:00 smithi148 ceph-mon[33568]: pgmap v693: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:49:01.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:00 smithi195 ceph-mon[33773]: pgmap v693: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:49:01.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:00 smithi112 ceph-mon[30100]: pgmap v693: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T10:49:02.886 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:49:02.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:49:02.887 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:49:02.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:49:02.909 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-05-10T10:49:02.969 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:49:02.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:49:02.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:49:03.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:02 smithi112 ceph-mon[30100]: pgmap v694: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-05-10T10:49:03.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:49:03.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-05-10T10:49:03.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:02 smithi148 ceph-mon[33568]: pgmap v694: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-05-10T10:49:03.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:02 smithi195 ceph-mon[33773]: pgmap v694: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-05-10T10:49:03.137 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-10T10:49:03.680 INFO:tasks.cephadm:Teardown begin 2024-05-10T10:49:03.680 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:49:03.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:49:03.938 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:05.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:04 smithi148 ceph-mon[33568]: pgmap v695: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:49:05.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:04 smithi195 ceph-mon[33773]: pgmap v695: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:49:05.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:04 smithi112 ceph-mon[30100]: pgmap v695: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:49:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.780+0000 7f3426dfa700 1 -- 172.21.15.112:0/2071883091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420103750 msgr2=0x7f3420103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.780+0000 7f3426dfa700 1 --2- 172.21.15.112:0/2071883091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420103750 0x7f3420103b30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f341c0099c0 tx=0x7f341c009cd0 comp rx=0 tx=0).stop 2024-05-10T10:49:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.780+0000 7f3426dfa700 1 -- 172.21.15.112:0/2071883091 shutdown_connections 2024-05-10T10:49:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.780+0000 7f3426dfa700 1 --2- 172.21.15.112:0/2071883091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3420108850 0x7f34201103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.780+0000 7f3426dfa700 1 --2- 172.21.15.112:0/2071883091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420103750 0x7f3420103b30 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.780+0000 7f3426dfa700 1 --2- 172.21.15.112:0/2071883091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3420104100 0x7f3420108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.780+0000 7f3426dfa700 1 -- 172.21.15.112:0/2071883091 >> 172.21.15.112:0/2071883091 conn(0x7f342007a600 msgr2=0x7f342007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 -- 172.21.15.112:0/2071883091 shutdown_connections 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 -- 172.21.15.112:0/2071883091 wait complete. 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 Processor -- start 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 -- start start 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3420103750 0x7f342019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3420104100 0x7f342019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420108850 0x7f34201998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34201134d0 con 0x7f3420108850 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113350 con 0x7f3420103750 2024-05-10T10:49:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3426dfa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113650 con 0x7f3420104100 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.781+0000 7f3425397700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420108850 0x7f34201998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420108850 0x7f34201998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58930/0 (socket says 172.21.15.112:58930) 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 -- 172.21.15.112:0/655806364 learned_addr learned my addr 172.21.15.112:0/655806364 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3424b96700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3420103750 0x7f342019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3417fff700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3420104100 0x7f342019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 -- 172.21.15.112:0/655806364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3420104100 msgr2=0x7f342019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3420104100 0x7f342019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 -- 172.21.15.112:0/655806364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3420103750 msgr2=0x7f342019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3420103750 0x7f342019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 -- 172.21.15.112:0/655806364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342019a030 con 0x7f3420108850 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3424b96700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3420103750 0x7f342019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3425397700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420108850 0x7f34201998d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3418007ee0 tx=0x7f341800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:05.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3415ffb700 1 -- 172.21.15.112:0/655806364 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800dbd0 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342019a320 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3415ffb700 1 -- 172.21.15.112:0/655806364 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f341800dd70 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3415ffb700 1 -- 172.21.15.112:0/655806364 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180058d0 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.782+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201a6590 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.783+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342010ba90 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.785+0000 7f3415ffb700 1 -- 172.21.15.112:0/655806364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f341801a020 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.785+0000 7f3415ffb700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f34080b2880 0x7f34080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.785+0000 7f3424b96700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f34080b2880 0x7f34080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.785+0000 7f3415ffb700 1 -- 172.21.15.112:0/655806364 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f34180e24a0 con 0x7f3420108850 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.786+0000 7f3424b96700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f34080b2880 0x7f34080b4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f341c009990 tx=0x7f341c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:05.789+0000 7f3415ffb700 1 -- 172.21.15.112:0/655806364 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3418010070 con 0x7f3420108850 2024-05-10T10:49:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.037+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f342019ade0 con 0x7f3420108850 2024-05-10T10:49:06.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.038+0000 7f3415ffb700 1 -- 172.21.15.112:0/655806364 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11975 (secure 0 0 0) 0x7f342019ade0 con 0x7f3420108850 2024-05-10T10:49:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:49:06.040 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:48:13.609075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24451,24469,24446,24445,24428]},"id":1}]} 2024-05-10T10:49:06.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f34080b2880 msgr2=0x7f34080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:06.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f34080b2880 0x7f34080b4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f341c009990 tx=0x7f341c01a040 comp rx=0 tx=0).stop 2024-05-10T10:49:06.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420108850 msgr2=0x7f34201998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:06.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420108850 0x7f34201998d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3418007ee0 tx=0x7f341800d3b0 comp rx=0 tx=0).stop 2024-05-10T10:49:06.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 shutdown_connections 2024-05-10T10:49:06.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420108850 0x7f34201998d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:06.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3420103750 0x7f342019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:06.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f34080b2880 0x7f34080b4d40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:06.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 --2- 172.21.15.112:0/655806364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3420104100 0x7f342019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:06.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 >> 172.21.15.112:0/655806364 conn(0x7f342007a600 msgr2=0x7f34201019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:06.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 shutdown_connections 2024-05-10T10:49:06.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:06.040+0000 7f3426dfa700 1 -- 172.21.15.112:0/655806364 wait complete. 2024-05-10T10:49:06.042 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 48 2024-05-10T10:49:06.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T10:49:06.995 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:06 smithi148 ceph-mon[33568]: pgmap v696: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-10T10:49:07.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:06 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/655806364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:49:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:06 smithi195 ceph-mon[33773]: pgmap v696: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-10T10:49:07.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:06 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/655806364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:49:07.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:06 smithi112 ceph-mon[30100]: pgmap v696: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-10T10:49:07.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:06 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/655806364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.668+0000 7fd05ac09700 1 -- 172.21.15.112:0/684403621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 msgr2=0x7fd05410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.668+0000 7fd05ac09700 1 --2- 172.21.15.112:0/684403621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05410f250 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b0a0 tx=0x7fd04c00b3b0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 -- 172.21.15.112:0/684403621 shutdown_connections 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 --2- 172.21.15.112:0/684403621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05410f250 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 --2- 172.21.15.112:0/684403621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd054108450 0x7fd054108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 --2- 172.21.15.112:0/684403621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0541023d0 0x7fd054102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 -- 172.21.15.112:0/684403621 >> 172.21.15.112:0/684403621 conn(0x7fd054078410 msgr2=0x7fd054078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 -- 172.21.15.112:0/684403621 shutdown_connections 2024-05-10T10:49:08.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 -- 172.21.15.112:0/684403621 wait complete. 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 Processor -- start 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 -- start start 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0541023d0 0x7fd05419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.669+0000 7fd05ac09700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd054108450 0x7fd0541a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd05ac09700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541123d0 con 0x7fd054102d90 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd05ac09700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd054112250 con 0x7fd0541023d0 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd05ac09700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd054112550 con 0x7fd054108450 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58940/0 (socket says 172.21.15.112:58940) 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 -- 172.21.15.112:0/2197993498 learned_addr learned my addr 172.21.15.112:0/2197993498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd0589a5700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0541023d0 0x7fd05419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd0591a6700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd054108450 0x7fd0541a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 -- 172.21.15.112:0/2197993498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd054108450 msgr2=0x7fd0541a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd054108450 0x7fd0541a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 -- 172.21.15.112:0/2197993498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0541023d0 msgr2=0x7fd05419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0541023d0 0x7fd05419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 -- 172.21.15.112:0/2197993498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0541a5e10 con 0x7fd054102d90 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd0589a5700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0541023d0 0x7fd05419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd053fff700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05419ad90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd04000d950 tx=0x7fd04000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd051ffb700 1 -- 172.21.15.112:0/2197993498 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040016070 con 0x7fd054102d90 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd051ffb700 1 -- 172.21.15.112:0/2197993498 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd040010460 con 0x7fd054102d90 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541a5fe0 con 0x7fd054102d90 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.670+0000 7fd051ffb700 1 -- 172.21.15.112:0/2197993498 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04000f620 con 0x7fd054102d90 2024-05-10T10:49:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.671+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0541a6540 con 0x7fd054102d90 2024-05-10T10:49:08.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.672+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd054103ac0 con 0x7fd054102d90 2024-05-10T10:49:08.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.673+0000 7fd051ffb700 1 -- 172.21.15.112:0/2197993498 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd040019070 con 0x7fd054102d90 2024-05-10T10:49:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.674+0000 7fd051ffb700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd0440b2900 0x7fd0440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.674+0000 7fd0589a5700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd0440b2900 0x7fd0440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.674+0000 7fd051ffb700 1 -- 172.21.15.112:0/2197993498 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd0400e20e0 con 0x7fd054102d90 2024-05-10T10:49:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.674+0000 7fd0589a5700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd0440b2900 0x7fd0440b4dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd04800bb90 tx=0x7fd048005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:08.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.677+0000 7fd051ffb700 1 -- 172.21.15.112:0/2197993498 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd0400a9a70 con 0x7fd054102d90 2024-05-10T10:49:08.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.930+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd05402e630 con 0x7fd054102d90 2024-05-10T10:49:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.931+0000 7fd051ffb700 1 -- 172.21.15.112:0/2197993498 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11975 (secure 0 0 0) 0x7fd0400a9370 con 0x7fd054102d90 2024-05-10T10:49:08.932 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:49:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/441557684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":441557684},{"type":"v1","addr":"172.21.15.112:6835","nonce":441557684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2497530637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2497530637},{"type":"v1","addr":"172.21.15.112:6837","nonce":2497530637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T10:35:52.903233+0000","modified":"2024-05-10T10:48:13.609075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24451,"mds_2":24446,"mds_3":24445,"mds_4":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2952588053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2952588053},{"type":"v1","addr":"172.21.15.112:6839","nonce":2952588053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/1213307484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1213307484},{"type":"v1","addr":"172.21.15.112:6841","nonce":1213307484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3726517324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3726517324},{"type":"v1","addr":"172.21.15.148:6833","nonce":3726517324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/2643629917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2643629917},{"type":"v1","addr":"172.21.15.148:6835","nonce":2643629917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/38979975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38979975},{"type":"v1","addr":"172.21.15.195:6833","nonce":38979975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.148:6837/1474359623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1474359623},{"type":"v1","addr":"172.21.15.148:6837","nonce":1474359623}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.195:6835/3491127541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3491127541},{"type":"v1","addr":"172.21.15.195:6835","nonce":3491127541}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.148:6839/1008412635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":1008412635},{"type":"v1","addr":"172.21.15.148:6839","nonce":1008412635}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6837/3223375652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3223375652},{"type":"v1","addr":"172.21.15.195:6837","nonce":3223375652}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.195:6839/2238321568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2238321568},{"type":"v1","addr":"172.21.15.195:6839","nonce":2238321568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24451,24469,24446,24445,24428]},"id":1}]} 2024-05-10T10:49:08.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd0440b2900 msgr2=0x7fd0440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd0440b2900 0x7fd0440b4dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd04800bb90 tx=0x7fd048005d80 comp rx=0 tx=0).stop 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 msgr2=0x7fd05419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05419ad90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd04000d950 tx=0x7fd04000dd10 comp rx=0 tx=0).stop 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 shutdown_connections 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fd0440b2900 0x7fd0440b4dc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd054108450 0x7fd0541a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0541023d0 0x7fd05419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 --2- 172.21.15.112:0/2197993498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd054102d90 0x7fd05419ad90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 >> 172.21.15.112:0/2197993498 conn(0x7fd054078410 msgr2=0x7fd0541013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:08.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 shutdown_connections 2024-05-10T10:49:08.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:08.933+0000 7fd05ac09700 1 -- 172.21.15.112:0/2197993498 wait complete. 2024-05-10T10:49:08.936 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 48 2024-05-10T10:49:09.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:08 smithi195 ceph-mon[33773]: pgmap v697: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:49:09.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:08 smithi112 ceph-mon[30100]: pgmap v697: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:49:09.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:08 smithi148 ceph-mon[33568]: pgmap v697: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-10T10:49:09.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T10:49:09.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:09 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2197993498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:49:09.831 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:10.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:09 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2197993498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:49:10.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:09 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2197993498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T10:49:11.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:10 smithi195 ceph-mon[33773]: pgmap v698: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T10:49:11.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:10 smithi112 ceph-mon[30100]: pgmap v698: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T10:49:11.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:10 smithi148 ceph-mon[33568]: pgmap v698: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.663+0000 7f20c37d4700 1 -- 172.21.15.112:0/1804006617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc1095d0 msgr2=0x7f20bc1099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.663+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1804006617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc1095d0 0x7f20bc1099b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f20b80099c0 tx=0x7f20b8009cd0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.663+0000 7f20c37d4700 1 -- 172.21.15.112:0/1804006617 shutdown_connections 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.663+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1804006617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20bc103ef0 0x7f20bc1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.663+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1804006617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc1095d0 0x7f20bc1099b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.663+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1804006617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20bc103530 0x7f20bc1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.663+0000 7f20c37d4700 1 -- 172.21.15.112:0/1804006617 >> 172.21.15.112:0/1804006617 conn(0x7f20bc07a690 msgr2=0x7f20bc07aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.664+0000 7f20c37d4700 1 -- 172.21.15.112:0/1804006617 shutdown_connections 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.664+0000 7f20c37d4700 1 -- 172.21.15.112:0/1804006617 wait complete. 2024-05-10T10:49:11.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.664+0000 7f20c37d4700 1 Processor -- start 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.664+0000 7f20c37d4700 1 -- start start 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.664+0000 7f20c37d4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20bc103530 0x7f20bc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c37d4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc103ef0 0x7f20bc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c37d4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20bc1095d0 0x7f20bc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c37d4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc113600 con 0x7f20bc103ef0 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c37d4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc113480 con 0x7f20bc103530 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c37d4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc113780 con 0x7f20bc1095d0 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc103ef0 0x7f20bc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc103ef0 0x7f20bc19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58962/0 (socket says 172.21.15.112:58962) 2024-05-10T10:49:11.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 -- 172.21.15.112:0/1173868193 learned_addr learned my addr 172.21.15.112:0/1173868193 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c1570700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20bc103530 0x7f20bc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c1d71700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20bc1095d0 0x7f20bc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 -- 172.21.15.112:0/1173868193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20bc1095d0 msgr2=0x7f20bc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20bc1095d0 0x7f20bc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 -- 172.21.15.112:0/1173868193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20bc103530 msgr2=0x7f20bc19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20bc103530 0x7f20bc19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 -- 172.21.15.112:0/1173868193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20bc19a080 con 0x7f20bc103ef0 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c0d6f700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc103ef0 0x7f20bc19fca0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00b530 tx=0x7f20ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20b27fc700 1 -- 172.21.15.112:0/1173868193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac015020 con 0x7f20bc103ef0 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20b27fc700 1 -- 172.21.15.112:0/1173868193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20ac011cc0 con 0x7f20bc103ef0 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20b27fc700 1 -- 172.21.15.112:0/1173868193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac010dc0 con 0x7f20bc103ef0 2024-05-10T10:49:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20bc19a370 con 0x7f20bc103ef0 2024-05-10T10:49:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.665+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20bc1a6630 con 0x7f20bc103ef0 2024-05-10T10:49:11.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.666+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20bc10ba60 con 0x7f20bc103ef0 2024-05-10T10:49:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.669+0000 7f20b27fc700 1 -- 172.21.15.112:0/1173868193 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f20ac018020 con 0x7f20bc103ef0 2024-05-10T10:49:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.669+0000 7f20b27fc700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20a80b2900 0x7f20a80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.669+0000 7f20c1570700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20a80b2900 0x7f20a80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.669+0000 7f20b27fc700 1 -- 172.21.15.112:0/1173868193 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f20ac0e2380 con 0x7f20bc103ef0 2024-05-10T10:49:11.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.670+0000 7f20c1570700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20a80b2900 0x7f20a80b4dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009990 tx=0x7f20b801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:11.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.673+0000 7f20b27fc700 1 -- 172.21.15.112:0/1173868193 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f20ac01f020 con 0x7f20bc103ef0 2024-05-10T10:49:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.881+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20bc19ae50 con 0x7f20bc103ef0 2024-05-10T10:49:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.882+0000 7f20b27fc700 1 -- 172.21.15.112:0/1173868193 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24730 (secure 0 0 0) 0x7f20ac0a9d10 con 0x7f20bc103ef0 2024-05-10T10:49:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-10T10:49:11.884 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"ccd347ac-0eb7-11ef-bc98-c7b262605968","created":"2024-05-10T10:27:34.800106+0000","modified":"2024-05-10T10:37:35.554118+0000","last_up_change":"2024-05-10T10:34:45.037279+0000","last_in_change":"2024-05-10T10:34:24.947136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:30:55.269138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:46.170078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:49.831849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:35:59.956882+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a67e2c4c-f481-4fd7-b502-3d561eb6ffc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6803","nonce":221387326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6805","nonce":221387326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6809","nonce":221387326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":221387326},{"type":"v1","addr":"172.21.15.112:6807","nonce":221387326}]},"public_addr":"172.21.15.112:6803/221387326","cluster_addr":"172.21.15.112:6805/221387326","heartbeat_back_addr":"172.21.15.112:6809/221387326","heartbeat_front_addr":"172.21.15.112:6807/221387326","state":["exists","up"]},{"osd":1,"uuid":"de68637a-80f9-450c-9208-784f54dc3df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6801","nonce":3233597314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6803","nonce":3233597314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6807","nonce":3233597314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3233597314},{"type":"v1","addr":"172.21.15.148:6805","nonce":3233597314}]},"public_addr":"172.21.15.148:6801/3233597314","cluster_addr":"172.21.15.148:6803/3233597314","heartbeat_back_addr":"172.21.15.148:6807/3233597314","heartbeat_front_addr":"172.21.15.148:6805/3233597314","state":["exists","up"]},{"osd":2,"uuid":"951b8444-0377-457c-b0c4-916113883584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6801","nonce":693285528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6803","nonce":693285528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6807","nonce":693285528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":693285528},{"type":"v1","addr":"172.21.15.195:6805","nonce":693285528}]},"public_addr":"172.21.15.195:6801/693285528","cluster_addr":"172.21.15.195:6803/693285528","heartbeat_back_addr":"172.21.15.195:6807/693285528","heartbeat_front_addr":"172.21.15.195:6805/693285528","state":["exists","up"]},{"osd":3,"uuid":"6c27d781-3ab0-4739-836d-c4016bb1e0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6811","nonce":1175679850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6813","nonce":1175679850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6817","nonce":1175679850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1175679850},{"type":"v1","addr":"172.21.15.112:6815","nonce":1175679850}]},"public_addr":"172.21.15.112:6811/1175679850","cluster_addr":"172.21.15.112:6813/1175679850","heartbeat_back_addr":"172.21.15.112:6817/1175679850","heartbeat_front_addr":"172.21.15.112:6815/1175679850","state":["exists","up"]},{"osd":4,"uuid":"4a8781c4-9d04-449c-a8df-45568364e555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6809","nonce":455926414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6811","nonce":455926414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6815","nonce":455926414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":455926414},{"type":"v1","addr":"172.21.15.148:6813","nonce":455926414}]},"public_addr":"172.21.15.148:6809/455926414","cluster_addr":"172.21.15.148:6811/455926414","heartbeat_back_addr":"172.21.15.148:6815/455926414","heartbeat_front_addr":"172.21.15.148:6813/455926414","state":["exists","up"]},{"osd":5,"uuid":"c65aefb7-ce11-4194-b1f4-ab1e323102cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6809","nonce":465167884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6811","nonce":465167884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6815","nonce":465167884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":465167884},{"type":"v1","addr":"172.21.15.195:6813","nonce":465167884}]},"public_addr":"172.21.15.195:6809/465167884","cluster_addr":"172.21.15.195:6811/465167884","heartbeat_back_addr":"172.21.15.195:6815/465167884","heartbeat_front_addr":"172.21.15.195:6813/465167884","state":["exists","up"]},{"osd":6,"uuid":"333835c8-bba4-4777-b7f9-ed8e6bb54ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6819","nonce":979146261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6821","nonce":979146261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6825","nonce":979146261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":979146261},{"type":"v1","addr":"172.21.15.112:6823","nonce":979146261}]},"public_addr":"172.21.15.112:6819/979146261","cluster_addr":"172.21.15.112:6821/979146261","heartbeat_back_addr":"172.21.15.112:6825/979146261","heartbeat_front_addr":"172.21.15.112:6823/979146261","state":["exists","up"]},{"osd":7,"uuid":"57dc7935-dd5e-47f4-87e0-8aae25736340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6817","nonce":3546374134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6819","nonce":3546374134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6823","nonce":3546374134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3546374134},{"type":"v1","addr":"172.21.15.148:6821","nonce":3546374134}]},"public_addr":"172.21.15.148:6817/3546374134","cluster_addr":"172.21.15.148:6819/3546374134","heartbeat_back_addr":"172.21.15.148:6823/3546374134","heartbeat_front_addr":"172.21.15.148:6821/3546374134","state":["exists","up"]},{"osd":8,"uuid":"e756afdf-f8df-4bda-ad15-a943ba0527d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6817","nonce":3432850695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6819","nonce":3432850695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6823","nonce":3432850695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3432850695},{"type":"v1","addr":"172.21.15.195:6821","nonce":3432850695}]},"public_addr":"172.21.15.195:6817/3432850695","cluster_addr":"172.21.15.195:6819/3432850695","heartbeat_back_addr":"172.21.15.195:6823/3432850695","heartbeat_front_addr":"172.21.15.195:6821/3432850695","state":["exists","up"]},{"osd":9,"uuid":"1111b3c0-269c-451e-8e48-bc2ae180960c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6827","nonce":2726539509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6829","nonce":2726539509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6833","nonce":2726539509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2726539509},{"type":"v1","addr":"172.21.15.112:6831","nonce":2726539509}]},"public_addr":"172.21.15.112:6827/2726539509","cluster_addr":"172.21.15.112:6829/2726539509","heartbeat_back_addr":"172.21.15.112:6833/2726539509","heartbeat_front_addr":"172.21.15.112:6831/2726539509","state":["exists","up"]},{"osd":10,"uuid":"841869b8-c760-4899-9e09-33c930cb81da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6825","nonce":2277563117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6827","nonce":2277563117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6831","nonce":2277563117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2277563117},{"type":"v1","addr":"172.21.15.148:6829","nonce":2277563117}]},"public_addr":"172.21.15.148:6825/2277563117","cluster_addr":"172.21.15.148:6827/2277563117","heartbeat_back_addr":"172.21.15.148:6831/2277563117","heartbeat_front_addr":"172.21.15.148:6829/2277563117","state":["exists","up"]},{"osd":11,"uuid":"e59ef38d-f7e9-4721-9779-75eb13dfddcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6825","nonce":1653173109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6827","nonce":1653173109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6831","nonce":1653173109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1653173109},{"type":"v1","addr":"172.21.15.195:6829","nonce":1653173109}]},"public_addr":"172.21.15.195:6825/1653173109","cluster_addr":"172.21.15.195:6827/1653173109","heartbeat_back_addr":"172.21.15.195:6831/1653173109","heartbeat_front_addr":"172.21.15.195:6829/1653173109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:04.692345+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:29.165594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:30:52.817578+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:17.514480+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:31:42.556701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:07.338386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:32.491209+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:32:58.862801+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.521276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:50.481440+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:17.074747+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:34:43.296559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3420175550":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6801/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/2112593785":"2024-05-11T10:28:33.178665+0000","172.21.15.112:6800/2836147288":"2024-05-11T10:28:33.178665+0000","172.21.15.112:0/1230086877":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6801/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3235282798":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/2711978475":"2024-05-11T10:28:06.482690+0000","172.21.15.112:6800/1796554484":"2024-05-11T10:28:06.482690+0000","172.21.15.112:0/3592023612":"2024-05-11T10:28:33.178665+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:49:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20a80b2900 msgr2=0x7f20a80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20a80b2900 0x7f20a80b4dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009990 tx=0x7f20b801a040 comp rx=0 tx=0).stop 2024-05-10T10:49:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc103ef0 msgr2=0x7f20bc19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc103ef0 0x7f20bc19fca0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00b530 tx=0x7f20ac00b840 comp rx=0 tx=0).stop 2024-05-10T10:49:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 shutdown_connections 2024-05-10T10:49:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f20a80b2900 0x7f20a80b4dc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20bc1095d0 0x7f20bc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20bc103530 0x7f20bc19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 --2- 172.21.15.112:0/1173868193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc103ef0 0x7f20bc19fca0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 >> 172.21.15.112:0/1173868193 conn(0x7f20bc07a690 msgr2=0x7f20bc102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 shutdown_connections 2024-05-10T10:49:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:11.885+0000 7f20c37d4700 1 -- 172.21.15.112:0/1173868193 wait complete. 2024-05-10T10:49:12.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-10T10:49:12.798 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:13.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:12 smithi112 ceph-mon[30100]: pgmap v699: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-10T10:49:13.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:12 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1173868193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:49:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:12 smithi195 ceph-mon[33773]: pgmap v699: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-10T10:49:13.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:12 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1173868193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:49:13.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:12 smithi148 ceph-mon[33568]: pgmap v699: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-10T10:49:13.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:12 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1173868193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 -- 172.21.15.112:0/3848009695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d8102fe0 msgr2=0x7fc5d8106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/3848009695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d8102fe0 0x7fc5d8106fe0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0099c0 tx=0x7fc5cc009cd0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 -- 172.21.15.112:0/3848009695 shutdown_connections 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/3848009695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d81076c0 0x7fc5d810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/3848009695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5d8102630 0x7fc5d8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/3848009695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d8102fe0 0x7fc5d8106fe0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 -- 172.21.15.112:0/3848009695 >> 172.21.15.112:0/3848009695 conn(0x7fc5d8079370 msgr2=0x7fc5d8079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:14.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 -- 172.21.15.112:0/3848009695 shutdown_connections 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.532+0000 7fc5dd98f700 1 -- 172.21.15.112:0/3848009695 wait complete. 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 Processor -- start 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 -- start start 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d8102630 0x7fc5d819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5d8102fe0 0x7fc5d819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81076c0 0x7fc5d81a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8112400 con 0x7fc5d81076c0 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8112280 con 0x7fc5d8102fe0 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.533+0000 7fc5dd98f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8112580 con 0x7fc5d8102630 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81076c0 0x7fc5d81a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81076c0 0x7fc5d81a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58978/0 (socket says 172.21.15.112:58978) 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 -- 172.21.15.112:0/2289231984 learned_addr learned my addr 172.21.15.112:0/2289231984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d67fc700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5d8102fe0 0x7fc5d819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:14.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d6ffd700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d8102630 0x7fc5d819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 -- 172.21.15.112:0/2289231984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d8102630 msgr2=0x7fc5d819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d8102630 0x7fc5d819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 -- 172.21.15.112:0/2289231984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5d8102fe0 msgr2=0x7fc5d819af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5d8102fe0 0x7fc5d819af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 -- 172.21.15.112:0/2289231984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d81a5fd0 con 0x7fc5d81076c0 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5d77fe700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81076c0 0x7fc5d81a5a90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000ada0 tx=0x7fc5d000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5dc98d700 1 -- 172.21.15.112:0/2289231984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0017020 con 0x7fc5d81076c0 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d81a61a0 con 0x7fc5d81076c0 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5dc98d700 1 -- 172.21.15.112:0/2289231984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d000ce10 con 0x7fc5d81076c0 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5dc98d700 1 -- 172.21.15.112:0/2289231984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0005280 con 0x7fc5d81076c0 2024-05-10T10:49:14.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.534+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d81a6700 con 0x7fc5d81076c0 2024-05-10T10:49:14.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.535+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d806a190 con 0x7fc5d81076c0 2024-05-10T10:49:14.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.537+0000 7fc5dc98d700 1 -- 172.21.15.112:0/2289231984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc5d000c980 con 0x7fc5d81076c0 2024-05-10T10:49:14.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.537+0000 7fc5dc98d700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5c00b2900 0x7fc5c00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.537+0000 7fc5d6ffd700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5c00b2900 0x7fc5c00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.537+0000 7fc5dc98d700 1 -- 172.21.15.112:0/2289231984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc5d00e1bb0 con 0x7fc5d81076c0 2024-05-10T10:49:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.538+0000 7fc5d6ffd700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5c00b2900 0x7fc5c00b4dc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8005fd0 tx=0x7fc5c800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.542+0000 7fc5dc98d700 1 -- 172.21.15.112:0/2289231984 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc5d0021020 con 0x7fc5d81076c0 2024-05-10T10:49:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.792+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc5d802e510 con 0x7fc5d81076c0 2024-05-10T10:49:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.794+0000 7fc5dc98d700 1 -- 172.21.15.112:0/2289231984 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7fc5d000a1f0 con 0x7fc5d81076c0 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5c00b2900 msgr2=0x7fc5c00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5c00b2900 0x7fc5c00b4dc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8005fd0 tx=0x7fc5c800a560 comp rx=0 tx=0).stop 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81076c0 msgr2=0x7fc5d81a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81076c0 0x7fc5d81a5a90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000ada0 tx=0x7fc5d000c330 comp rx=0 tx=0).stop 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 shutdown_connections 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81076c0 0x7fc5d81a5a90 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fc5c00b2900 0x7fc5c00b4dc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d8102630 0x7fc5d819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 --2- 172.21.15.112:0/2289231984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5d8102fe0 0x7fc5d819af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.797+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 >> 172.21.15.112:0/2289231984 conn(0x7fc5d8079370 msgr2=0x7fc5d8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.798+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 shutdown_connections 2024-05-10T10:49:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:14.798+0000 7fc5dd98f700 1 -- 172.21.15.112:0/2289231984 wait complete. 2024-05-10T10:49:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:49:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mon[33773]: pgmap v700: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:49:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2289231984' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T10:49:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mon[33773]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T10:49:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mon[33773]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:49:15.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2289231984' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T10:49:15.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mon[33773]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T10:49:15.067 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mon[33773]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T10:49:15.067 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l[63179]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.067 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l[63179]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.067 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l[63179]: starting mds.l at 2024-05-10T10:49:15.067 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mds[63815]: -- 172.21.15.195:0/1181581704 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652b44c7a40 con 0x5652b449a800 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: starting mds.c at 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f[63500]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f[63500]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mds[63504]: -- 172.21.15.195:0/2805521336 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605fb8a5a40 con 0x5605fb878c00 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f[63500]: starting mds.f at 2024-05-10T10:49:15.068 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.069 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.069 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:49:14 smithi195 ceph-mds[64123]: -- 172.21.15.195:0/3908876395 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f22e3b5a40 con 0x55f22e389000 2024-05-10T10:49:15.069 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:49:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: starting mds.i at 2024-05-10T10:49:15.080 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a[69621]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.080 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a[69621]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.080 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mds[69625]: -- 172.21.15.112:0/356980149 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556afbe47a40 con 0x556afbe1b000 2024-05-10T10:49:15.080 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a[69621]: starting mds.a at 2024-05-10T10:49:15.080 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:49:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g[69187]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.080 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:49:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g[69187]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.080 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mds[69191]: -- 172.21.15.112:0/349269893 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d404bba40 con 0x562d4048e800 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:49:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g[69187]: starting mds.g at 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a[30096]: 2024-05-10T10:49:14.792+0000 7ff612f31700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mon[30100]: pgmap v700: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2289231984' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mon[30100]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mon[30100]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2289231984' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mon[30100]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T10:49:15.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:14 smithi112 ceph-mon[30100]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T10:49:15.294 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e[63264]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.295 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e[63264]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.295 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mds[63269]: -- 172.21.15.148:0/3215757637 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bcf8e53a40 con 0x55bcf8e27000 2024-05-10T10:49:15.295 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e[63264]: starting mds.e at 2024-05-10T10:49:15.295 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e[63264]: 2024-05-10T10:49:14.846+0000 7f980f025700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-10T10:49:15.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mon[33568]: pgmap v700: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T10:49:15.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2289231984' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T10:49:15.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mon[33568]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mon[33568]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2289231984' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mon[33568]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mon[33568]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b[63901]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b[63901]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mds[63905]: -- 172.21.15.148:0/55805089 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642d7ad5a40 con 0x5642d7aa8c00 2024-05-10T10:49:15.296 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b[63901]: starting mds.b at 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h[63588]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h[63588]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mds[63592]: -- 172.21.15.148:0/3919673558 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fba27d1a40 con 0x55fba27a4c00 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h[63588]: starting mds.h at 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k[62942]: ignoring --setuser ceph since I am not root 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k[62942]: ignoring --setgroup ceph since I am not root 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:49:14 smithi148 ceph-mds[62946]: -- 172.21.15.148:0/2049170721 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c0d285a40 con 0x562c0d259000 2024-05-10T10:49:15.297 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:49:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k[62942]: starting mds.k at 2024-05-10T10:49:15.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:49:15.863 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: pgmap v702: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.112:6838/2128511331,v1:172.21.15.112:6839/2128511331] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6832/2995892299,v1:172.21.15.195:6834/2995892299] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6833/3479822216,v1:172.21.15.195:6835/3479822216] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6832/1389635986,v1:172.21.15.148:6834/1389635986] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6833/1568380206,v1:172.21.15.148:6835/1568380206] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.112:6840/732330055,v1:172.21.15.112:6841/732330055] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6836/682428051,v1:172.21.15.148:6837/682428051] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6836/4227273237,v1:172.21.15.195:6838/4227273237] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.148:6838/2221648634,v1:172.21.15.148:6839/2221648634] up:boot 2024-05-10T10:49:16.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: mds.? [v2:172.21.15.195:6837/2166486556,v1:172.21.15.195:6839/2166486556] up:boot 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T10:49:16.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:16 smithi112 ceph-mon[30100]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T10:49:17.147 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: pgmap v702: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T10:49:17.147 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.112:6838/2128511331,v1:172.21.15.112:6839/2128511331] up:boot 2024-05-10T10:49:17.147 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6832/2995892299,v1:172.21.15.195:6834/2995892299] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6833/3479822216,v1:172.21.15.195:6835/3479822216] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6832/1389635986,v1:172.21.15.148:6834/1389635986] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6833/1568380206,v1:172.21.15.148:6835/1568380206] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.112:6840/732330055,v1:172.21.15.112:6841/732330055] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6836/682428051,v1:172.21.15.148:6837/682428051] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6836/4227273237,v1:172.21.15.195:6838/4227273237] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.148:6838/2221648634,v1:172.21.15.148:6839/2221648634] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: mds.? [v2:172.21.15.195:6837/2166486556,v1:172.21.15.195:6839/2166486556] up:boot 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T10:49:17.148 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T10:49:17.149 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T10:49:17.149 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T10:49:17.149 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T10:49:17.149 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T10:49:17.149 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T10:49:17.149 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:16 smithi195 ceph-mon[33773]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T10:49:17.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: pgmap v702: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.112:6838/2128511331,v1:172.21.15.112:6839/2128511331] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6832/2995892299,v1:172.21.15.195:6834/2995892299] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6833/3479822216,v1:172.21.15.195:6835/3479822216] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6832/1389635986,v1:172.21.15.148:6834/1389635986] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6833/1568380206,v1:172.21.15.148:6835/1568380206] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.112:6840/732330055,v1:172.21.15.112:6841/732330055] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6836/682428051,v1:172.21.15.148:6837/682428051] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6836/4227273237,v1:172.21.15.195:6838/4227273237] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.148:6838/2221648634,v1:172.21.15.148:6839/2221648634] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: mds.? [v2:172.21.15.195:6837/2166486556,v1:172.21.15.195:6839/2166486556] up:boot 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T10:49:17.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T10:49:17.296 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:16 smithi148 ceph-mon[33568]: from='mgr.14150 172.21.15.112:0/2466927213' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T10:49:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 -- 172.21.15.112:0/3089604808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600109510 msgr2=0x7f06001098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 --2- 172.21.15.112:0/3089604808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600109510 0x7f06001098f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009990 tx=0x7f05f4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 -- 172.21.15.112:0/3089604808 shutdown_connections 2024-05-10T10:49:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 --2- 172.21.15.112:0/3089604808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f060006c5b0 0x7f0600110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 --2- 172.21.15.112:0/3089604808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600109510 0x7f06001098f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 --2- 172.21.15.112:0/3089604808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060006bbf0 0x7f060006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 -- 172.21.15.112:0/3089604808 >> 172.21.15.112:0/3089604808 conn(0x7f0600079370 msgr2=0x7f0600079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 -- 172.21.15.112:0/3089604808 shutdown_connections 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.529+0000 7f0606e23700 1 -- 172.21.15.112:0/3089604808 wait complete. 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 Processor -- start 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 -- start start 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060006bbf0 0x7f060010f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060006c5b0 0x7f060010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0600109510 0x7f06001a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06001134d0 con 0x7f060006c5b0 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0600113350 con 0x7f0600109510 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0606e23700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0600113650 con 0x7f060006bbf0 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060006c5b0 0x7f060010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060006c5b0 0x7f060010fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32882/0 (socket says 172.21.15.112:32882) 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 -- 172.21.15.112:0/1821303541 learned_addr learned my addr 172.21.15.112:0/1821303541 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f0604bbf700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060006bbf0 0x7f060010f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f06053c0700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0600109510 0x7f06001a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 -- 172.21.15.112:0/1821303541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060006bbf0 msgr2=0x7f060010f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060006bbf0 0x7f060010f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 -- 172.21.15.112:0/1821303541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0600109510 msgr2=0x7f06001a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0600109510 0x7f06001a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.530+0000 7f05fffff700 1 -- 172.21.15.112:0/1821303541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06001a5060 con 0x7f060006c5b0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.531+0000 7f05fffff700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060006c5b0 0x7f060010fc20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00d980 tx=0x7f05ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.531+0000 7f05fdffb700 1 -- 172.21.15.112:0/1821303541 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec016070 con 0x7f060006c5b0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.531+0000 7f05fdffb700 1 -- 172.21.15.112:0/1821303541 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05ec004830 con 0x7f060006c5b0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.531+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06001a5350 con 0x7f060006c5b0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.531+0000 7f05fdffb700 1 -- 172.21.15.112:0/1821303541 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec005440 con 0x7f060006c5b0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.531+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06001acbd0 con 0x7f060006c5b0 2024-05-10T10:49:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.532+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060006a190 con 0x7f060006c5b0 2024-05-10T10:49:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.533+0000 7f05fdffb700 1 -- 172.21.15.112:0/1821303541 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f05ec0055e0 con 0x7f060006c5b0 2024-05-10T10:49:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.534+0000 7f05fdffb700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f05f00b2880 0x7f05f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.534+0000 7f0604bbf700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f05f00b2880 0x7f05f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.534+0000 7f05fdffb700 1 -- 172.21.15.112:0/1821303541 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9580+0+0 (secure 0 0 0) 0x7f05ec0e2540 con 0x7f060006c5b0 2024-05-10T10:49:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.535+0000 7f0604bbf700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f05f00b2880 0x7f05f00b4d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f05f40098e0 tx=0x7f05f4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.539+0000 7f05fdffb700 1 -- 172.21.15.112:0/1821303541 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f05ec01f020 con 0x7f060006c5b0 2024-05-10T10:49:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.788+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f060002e510 con 0x7f060006c5b0 2024-05-10T10:49:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.820+0000 7f05fdffb700 1 -- 172.21.15.112:0/1821303541 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7f05ec0a9d20 con 0x7f060006c5b0 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.822+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f05f00b2880 msgr2=0x7f05f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.822+0000 7f0606e23700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f05f00b2880 0x7f05f00b4d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f05f40098e0 tx=0x7f05f4005960 comp rx=0 tx=0).stop 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.822+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060006c5b0 msgr2=0x7f060010fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.822+0000 7f0606e23700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060006c5b0 0x7f060010fc20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00d980 tx=0x7f05ec00dc90 comp rx=0 tx=0).stop 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 shutdown_connections 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0600109510 0x7f06001a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f05f00b2880 0x7f05f00b4d40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060006bbf0 0x7f060010f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 --2- 172.21.15.112:0/1821303541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060006c5b0 0x7f060010fc20 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 >> 172.21.15.112:0/1821303541 conn(0x7f0600079370 msgr2=0x7f0600101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 shutdown_connections 2024-05-10T10:49:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:17.823+0000 7f0606e23700 1 -- 172.21.15.112:0/1821303541 wait complete. 2024-05-10T10:49:18.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:17 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1821303541' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:49:18.081 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:17 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1821303541' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:49:18.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:17 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1821303541' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T10:49:18.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:49:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:19.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:18 smithi112 ceph-mon[30100]: pgmap v703: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T10:49:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:18 smithi112 ceph-mon[30100]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T10:49:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:18 smithi112 ceph-mon[30100]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T10:49:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:18 smithi112 ceph-mon[30100]: Cluster is now healthy 2024-05-10T10:49:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:18 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/1821303541' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T10:49:19.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:18 smithi112 ceph-mon[30100]: fsmap 12 up:standby 2024-05-10T10:49:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:18 smithi148 ceph-mon[33568]: pgmap v703: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T10:49:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:18 smithi148 ceph-mon[33568]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T10:49:19.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:18 smithi148 ceph-mon[33568]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T10:49:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:18 smithi148 ceph-mon[33568]: Cluster is now healthy 2024-05-10T10:49:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:18 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/1821303541' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T10:49:19.295 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:18 smithi148 ceph-mon[33568]: fsmap 12 up:standby 2024-05-10T10:49:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:18 smithi195 ceph-mon[33773]: pgmap v703: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T10:49:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:18 smithi195 ceph-mon[33773]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T10:49:19.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:18 smithi195 ceph-mon[33773]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T10:49:19.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:18 smithi195 ceph-mon[33773]: Cluster is now healthy 2024-05-10T10:49:19.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:18 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/1821303541' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T10:49:19.317 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:18 smithi195 ceph-mon[33773]: fsmap 12 up:standby 2024-05-10T10:49:20.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 -- 172.21.15.112:0/4130524213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98104100 msgr2=0x7f9c98108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:20.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/4130524213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98104100 0x7f9c98108120 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9c880099c0 tx=0x7f9c88009cd0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 -- 172.21.15.112:0/4130524213 shutdown_connections 2024-05-10T10:49:20.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/4130524213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c98108850 0x7f9c981103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/4130524213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98103750 0x7f9c98103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/4130524213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98104100 0x7f9c98108120 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 -- 172.21.15.112:0/4130524213 >> 172.21.15.112:0/4130524213 conn(0x7f9c9807a600 msgr2=0x7f9c9807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 -- 172.21.15.112:0/4130524213 shutdown_connections 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.438+0000 7f9c9fab3700 1 -- 172.21.15.112:0/4130524213 wait complete. 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.439+0000 7f9c9fab3700 1 Processor -- start 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.439+0000 7f9c9fab3700 1 -- start start 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.439+0000 7f9c9fab3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98103750 0x7f9c9819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.439+0000 7f9c9fab3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c98104100 0x7f9c9819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.439+0000 7f9c9fab3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98108850 0x7f9c981997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9fab3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981134c0 con 0x7f9c98108850 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9fab3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98113340 con 0x7f9c98103750 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9fab3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98113640 con 0x7f9c98104100 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98108850 0x7f9c981997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98108850 0x7f9c981997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32902/0 (socket says 172.21.15.112:32902) 2024-05-10T10:49:20.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 -- 172.21.15.112:0/3669117341 learned_addr learned my addr 172.21.15.112:0/3669117341 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9d84f700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98103750 0x7f9c9819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9d04e700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c98104100 0x7f9c9819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 -- 172.21.15.112:0/3669117341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c98104100 msgr2=0x7f9c9819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c98104100 0x7f9c9819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 -- 172.21.15.112:0/3669117341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98103750 msgr2=0x7f9c9819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98103750 0x7f9c9819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 -- 172.21.15.112:0/3669117341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c98199fa0 con 0x7f9c98108850 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9d04e700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c98104100 0x7f9c9819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9e050700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98108850 0x7f9c981997c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9c90007d70 tx=0x7f9c9000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c8effd700 1 -- 172.21.15.112:0/3669117341 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90016020 con 0x7f9c98108850 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9819a290 con 0x7f9c98108850 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c8effd700 1 -- 172.21.15.112:0/3669117341 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c9000ad40 con 0x7f9c98108850 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c8effd700 1 -- 172.21.15.112:0/3669117341 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c900115e0 con 0x7f9c98108850 2024-05-10T10:49:20.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.440+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c980756d0 con 0x7f9c98108850 2024-05-10T10:49:20.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.441+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c98104be0 con 0x7f9c98108850 2024-05-10T10:49:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.443+0000 7f9c8effd700 1 -- 172.21.15.112:0/3669117341 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9c90019020 con 0x7f9c98108850 2024-05-10T10:49:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.443+0000 7f9c8effd700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9c840b2880 0x7f9c840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.443+0000 7f9c9d84f700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9c840b2880 0x7f9c840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.443+0000 7f9c8effd700 1 -- 172.21.15.112:0/3669117341 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9580+0+0 (secure 0 0 0) 0x7f9c900e26f0 con 0x7f9c98108850 2024-05-10T10:49:20.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.444+0000 7f9c9d84f700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9c840b2880 0x7f9c840b4d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9c94005950 tx=0x7f9c9400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:20.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.448+0000 7f9c8effd700 1 -- 172.21.15.112:0/3669117341 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9c90020020 con 0x7f9c98108850 2024-05-10T10:49:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.653+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9c9802e510 con 0x7f9c98108850 2024-05-10T10:49:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.820+0000 7f9c8effd700 1 -- 172.21.15.112:0/3669117341 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7f9c900a9ed0 con 0x7f9c98108850 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.823+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9c840b2880 msgr2=0x7f9c840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.823+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9c840b2880 0x7f9c840b4d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9c94005950 tx=0x7f9c9400a300 comp rx=0 tx=0).stop 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.823+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98108850 msgr2=0x7f9c981997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.823+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98108850 0x7f9c981997c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9c90007d70 tx=0x7f9c9000a340 comp rx=0 tx=0).stop 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 shutdown_connections 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c98108850 0x7f9c981997c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98103750 0x7f9c9819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f9c840b2880 0x7f9c840b4d40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 --2- 172.21.15.112:0/3669117341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c98104100 0x7f9c9819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:20.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 >> 172.21.15.112:0/3669117341 conn(0x7f9c9807a600 msgr2=0x7f9c98101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 shutdown_connections 2024-05-10T10:49:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:20.824+0000 7f9c9fab3700 1 -- 172.21.15.112:0/3669117341 wait complete. 2024-05-10T10:49:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:49:21.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:20 smithi112 ceph-mon[30100]: pgmap v704: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-05-10T10:49:21.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:20 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3669117341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:21.096 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:20 smithi148 ceph-mon[33568]: pgmap v704: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-05-10T10:49:21.096 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:20 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3669117341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:20 smithi195 ceph-mon[33773]: pgmap v704: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-05-10T10:49:21.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:20 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3669117341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:21.535 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:49:21.777 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:22.159 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:21 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/3669117341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:22.159 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:21 smithi112 ceph-mon[30100]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T10:49:22.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:21 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/3669117341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:22.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:21 smithi148 ceph-mon[33568]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T10:49:22.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:21 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/3669117341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:22.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:21 smithi195 ceph-mon[33773]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T10:49:22.914 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:22 smithi112 ceph-mon[30100]: pgmap v706: 97 pgs: 97 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:23.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:22 smithi148 ceph-mon[33568]: pgmap v706: 97 pgs: 97 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:23.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:22 smithi195 ceph-mon[33773]: pgmap v706: 97 pgs: 97 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:23.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 -- 172.21.15.112:0/965847422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 msgr2=0x7f65dc101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:23.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 --2- 172.21.15.112:0/965847422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc101f10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009990 tx=0x7f65c4009ca0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 -- 172.21.15.112:0/965847422 shutdown_connections 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 --2- 172.21.15.112:0/965847422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65dc102450 0x7f65dc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 --2- 172.21.15.112:0/965847422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65dc109520 0x7f65dc109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 --2- 172.21.15.112:0/965847422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc101f10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 -- 172.21.15.112:0/965847422 >> 172.21.15.112:0/965847422 conn(0x7f65dc078a10 msgr2=0x7f65dc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 -- 172.21.15.112:0/965847422 shutdown_connections 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.490+0000 7f65e115b700 1 -- 172.21.15.112:0/965847422 wait complete. 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 Processor -- start 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 -- start start 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65dc102450 0x7f65dc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65dc109520 0x7f65dc199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:23.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc1134e0 con 0x7f65dc101a90 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc113360 con 0x7f65dc102450 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65e115b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc113660 con 0x7f65dc109520 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65dad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65dad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32910/0 (socket says 172.21.15.112:32910) 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65dad9d700 1 -- 172.21.15.112:0/591372307 learned_addr learned my addr 172.21.15.112:0/591372307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.491+0000 7f65d259c700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65dc102450 0x7f65dc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65db59e700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65dc109520 0x7f65dc199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65dad9d700 1 -- 172.21.15.112:0/591372307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65dc109520 msgr2=0x7f65dc199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65dad9d700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65dc109520 0x7f65dc199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65dad9d700 1 -- 172.21.15.112:0/591372307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65dc102450 msgr2=0x7f65dc19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65dad9d700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65dc102450 0x7f65dc19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65dad9d700 1 -- 172.21.15.112:0/591372307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc19a0c0 con 0x7f65dc101a90 2024-05-10T10:49:23.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65dad9d700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc19f750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00d980 tx=0x7f65cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65d8d99700 1 -- 172.21.15.112:0/591372307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc016070 con 0x7f65dc101a90 2024-05-10T10:49:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc19a3b0 con 0x7f65dc101a90 2024-05-10T10:49:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65d8d99700 1 -- 172.21.15.112:0/591372307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65cc004830 con 0x7f65dc101a90 2024-05-10T10:49:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65d8d99700 1 -- 172.21.15.112:0/591372307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc005440 con 0x7f65dc101a90 2024-05-10T10:49:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.492+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a6620 con 0x7f65dc101a90 2024-05-10T10:49:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.493+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc104db0 con 0x7f65dc101a90 2024-05-10T10:49:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.494+0000 7f65d8d99700 1 -- 172.21.15.112:0/591372307 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f65cc0055e0 con 0x7f65dc101a90 2024-05-10T10:49:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.495+0000 7f65d8d99700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65c80b2880 0x7f65c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.495+0000 7f65d259c700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65c80b2880 0x7f65c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.495+0000 7f65d8d99700 1 -- 172.21.15.112:0/591372307 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9055+0+0 (secure 0 0 0) 0x7f65cc0e22e0 con 0x7f65dc101a90 2024-05-10T10:49:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.496+0000 7f65d259c700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65c80b2880 0x7f65c80b4d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f65c400b5c0 tx=0x7f65c4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.500+0000 7f65d8d99700 1 -- 172.21.15.112:0/591372307 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f65cc01f020 con 0x7f65dc101a90 2024-05-10T10:49:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.701+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f65dc19ae40 con 0x7f65dc101a90 2024-05-10T10:49:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.842+0000 7f65d8d99700 1 -- 172.21.15.112:0/591372307 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7f65cc0a9c70 con 0x7f65dc101a90 2024-05-10T10:49:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.845+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65c80b2880 msgr2=0x7f65c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.845+0000 7f65e115b700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65c80b2880 0x7f65c80b4d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f65c400b5c0 tx=0x7f65c4005960 comp rx=0 tx=0).stop 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.845+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 msgr2=0x7f65dc19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.845+0000 7f65e115b700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc19f750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00d980 tx=0x7f65cc00dc90 comp rx=0 tx=0).stop 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.846+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 shutdown_connections 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.846+0000 7f65e115b700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7f65c80b2880 0x7f65c80b4d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.846+0000 7f65e115b700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65dc109520 0x7f65dc199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.846+0000 7f65e115b700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65dc101a90 0x7f65dc19f750 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.846+0000 7f65e115b700 1 --2- 172.21.15.112:0/591372307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65dc102450 0x7f65dc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.846+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 >> 172.21.15.112:0/591372307 conn(0x7f65dc078a10 msgr2=0x7f65dc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.846+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 shutdown_connections 2024-05-10T10:49:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:23.847+0000 7f65e115b700 1 -- 172.21.15.112:0/591372307 wait complete. 2024-05-10T10:49:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-05-10T10:49:24.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:23 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/591372307' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:24.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:23 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/591372307' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:24.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:23 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/591372307' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:24.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-10T10:49:24.814 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T10:49:25.268 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:24 smithi112 ceph-mon[30100]: pgmap v707: 97 pgs: 97 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:25.268 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:24 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/591372307' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:25.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:24 smithi112 ceph-mon[30100]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T10:49:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:24 smithi148 ceph-mon[33568]: pgmap v707: 97 pgs: 97 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:24 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/591372307' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:25.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:24 smithi148 ceph-mon[33568]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T10:49:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:24 smithi195 ceph-mon[33773]: pgmap v707: 97 pgs: 97 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:24 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/591372307' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:25.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:24 smithi195 ceph-mon[33773]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T10:49:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2518528706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 msgr2=0x7fde68103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2518528706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde68103b30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fde580099c0 tx=0x7fde58009cd0 comp rx=0 tx=0).stop 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2518528706 shutdown_connections 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2518528706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde68108850 0x7fde681103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2518528706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde68103b30 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2518528706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde68104100 0x7fde68108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2518528706 >> 172.21.15.112:0/2518528706 conn(0x7fde6807a600 msgr2=0x7fde6807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2518528706 shutdown_connections 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.676+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2518528706 wait complete. 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 Processor -- start 2024-05-10T10:49:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 -- start start 2024-05-10T10:49:26.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde6819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:26.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde68104100 0x7fde6819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde68108850 0x7fde68199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde681135c0 con 0x7fde68103750 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde68113440 con 0x7fde68104100 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6e5d2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fde68113740 con 0x7fde68108850 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde67fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde6819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde67fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde6819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59458/0 (socket says 172.21.15.112:59458) 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde67fff700 1 -- 172.21.15.112:0/2239635554 learned_addr learned my addr 172.21.15.112:0/2239635554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.677+0000 7fde6cb6f700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde68108850 0x7fde68199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde677fe700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde68104100 0x7fde6819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde67fff700 1 -- 172.21.15.112:0/2239635554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde68108850 msgr2=0x7fde68199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde67fff700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde68108850 0x7fde68199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde67fff700 1 -- 172.21.15.112:0/2239635554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde68104100 msgr2=0x7fde6819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde67fff700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde68104100 0x7fde6819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde67fff700 1 -- 172.21.15.112:0/2239635554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6819a080 con 0x7fde68103750 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde6cb6f700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde68108850 0x7fde68199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde677fe700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde68104100 0x7fde6819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde67fff700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde6819f760 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fde58000c00 tx=0x7fde5800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde657fa700 1 -- 172.21.15.112:0/2239635554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58026070 con 0x7fde68103750 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6819a310 con 0x7fde68103750 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde657fa700 1 -- 172.21.15.112:0/2239635554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde580054c0 con 0x7fde68103750 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde657fa700 1 -- 172.21.15.112:0/2239635554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58004c20 con 0x7fde68103750 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.678+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681a6630 con 0x7fde68103750 2024-05-10T10:49:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.679+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6806a190 con 0x7fde68103750 2024-05-10T10:49:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.680+0000 7fde657fa700 1 -- 172.21.15.112:0/2239635554 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fde58029070 con 0x7fde68103750 2024-05-10T10:49:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.681+0000 7fde657fa700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fde500b2880 0x7fde500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:49:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.681+0000 7fde677fe700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fde500b2880 0x7fde500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:49:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.681+0000 7fde657fa700 1 -- 172.21.15.112:0/2239635554 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8496+0+0 (secure 0 0 0) 0x7fde580e32e0 con 0x7fde68103750 2024-05-10T10:49:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.682+0000 7fde677fe700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fde500b2880 0x7fde500b4d40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde68103130 tx=0x7fde5c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:49:26.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.685+0000 7fde657fa700 1 -- 172.21.15.112:0/2239635554 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fde58030020 con 0x7fde68103750 2024-05-10T10:49:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:26.888+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fde6802e510 con 0x7fde68103750 2024-05-10T10:49:27.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:26 smithi148 ceph-mon[33568]: pgmap v709: 33 pgs: 33 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:27.316 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:26 smithi195 ceph-mon[33773]: pgmap v709: 33 pgs: 33 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:27.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:26 smithi112 ceph-mon[30100]: pgmap v709: 33 pgs: 33 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:27.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.852+0000 7fde657fa700 1 -- 172.21.15.112:0/2239635554 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v94) ==== 178+0+0 (secure 0 0 0) 0x7fde58013070 con 0x7fde68103750 2024-05-10T10:49:27.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.855+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fde500b2880 msgr2=0x7fde500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:27.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.855+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fde500b2880 0x7fde500b4d40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde68103130 tx=0x7fde5c008040 comp rx=0 tx=0).stop 2024-05-10T10:49:27.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.855+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 msgr2=0x7fde6819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:49:27.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.855+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde6819f760 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fde58000c00 tx=0x7fde5800f740 comp rx=0 tx=0).stop 2024-05-10T10:49:27.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 shutdown_connections 2024-05-10T10:49:27.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:6800/1898750131,v1:172.21.15.112:6801/1898750131] conn(0x7fde500b2880 0x7fde500b4d40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:27.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde68108850 0x7fde68199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:27.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde68103750 0x7fde6819f760 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:27.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 --2- 172.21.15.112:0/2239635554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde68104100 0x7fde6819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:49:27.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 >> 172.21.15.112:0/2239635554 conn(0x7fde6807a600 msgr2=0x7fde681099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:49:27.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 shutdown_connections 2024-05-10T10:49:27.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-10T10:49:27.856+0000 7fde6e5d2700 1 -- 172.21.15.112:0/2239635554 wait complete. 2024-05-10T10:49:27.860 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data_ec' removed 2024-05-10T10:49:28.154 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:27 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2239635554' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:28.240 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:27 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2239635554' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:28.294 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:27 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2239635554' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T10:49:28.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:49:28.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:49:28.643 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:49:28.673 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-10T10:49:28.673 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T10:49:28.689 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T10:49:28.708 DEBUG:teuthology.orchestra.run.smithi195:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T10:49:28.729 INFO:tasks.cephadm:Stopping all daemons... 2024-05-10T10:49:28.729 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-10T10:49:28.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a 2024-05-10T10:49:28.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:28 smithi112 systemd[1]: Stopping Ceph mon.a for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:29.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:28 smithi148 ceph-mon[33568]: pgmap v710: 33 pgs: 33 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:29.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:28 smithi148 ceph-mon[33568]: from='client.? 172.21.15.112:0/2239635554' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:29.045 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:28 smithi148 ceph-mon[33568]: osdmap e94: 12 total, 12 up, 12 in 2024-05-10T10:49:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:28 smithi195 ceph-mon[33773]: pgmap v710: 33 pgs: 33 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:28 smithi195 ceph-mon[33773]: from='client.? 172.21.15.112:0/2239635554' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:29.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:28 smithi195 ceph-mon[33773]: osdmap e94: 12 total, 12 up, 12 in 2024-05-10T10:49:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:28 smithi112 ceph-mon[30100]: pgmap v710: 33 pgs: 33 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T10:49:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:28 smithi112 ceph-mon[30100]: from='client.? 172.21.15.112:0/2239635554' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T10:49:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:28 smithi112 ceph-mon[30100]: osdmap e94: 12 total, 12 up, 12 in 2024-05-10T10:49:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a[30096]: 2024-05-10T10:49:29.035+0000 7ff61873c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:49:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a[30096]: 2024-05-10T10:49:29.035+0000 7ff61873c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-10T10:49:29.487 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:29 smithi112 podman[127147]: 2024-05-10 10:49:29.219722413 +0000 UTC m=+0.301170394 container died 41fabb039a5b7ad4eb696899bf0e1d3b195e4ecba2544fbdd001eac4f48a1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-10T10:49:29.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:29 smithi112 podman[127147]: 2024-05-10 10:49:29.586614991 +0000 UTC m=+0.668062982 container cleanup 41fabb039a5b7ad4eb696899bf0e1d3b195e4ecba2544fbdd001eac4f48a1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:49:29.830 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 10 10:49:29 smithi112 bash[127147]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-a 2024-05-10T10:49:30.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.a.service' 2024-05-10T10:49:30.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:30.063 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-10T10:49:30.064 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-10T10:49:30.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.b 2024-05-10T10:49:30.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:30 smithi148 systemd[1]: Stopping Ceph mon.b for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-b[33559]: 2024-05-10T10:49:30.375+0000 7fae521d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:49:30.794 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-b[33559]: 2024-05-10T10:49:30.375+0000 7fae521d3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-10T10:49:30.795 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:30 smithi148 podman[68292]: 2024-05-10 10:49:30.52059344 +0000 UTC m=+0.241444854 container died 0b5202427c3280eb3bef4b81effb7f233f9f141017c21203fc902b6441645267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-b, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T10:49:31.171 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:30 smithi148 podman[68292]: 2024-05-10 10:49:30.927326673 +0000 UTC m=+0.648178094 container cleanup 0b5202427c3280eb3bef4b81effb7f233f9f141017c21203fc902b6441645267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:49:31.171 INFO:journalctl@ceph.mon.b.smithi148.stdout:May 10 10:49:30 smithi148 bash[68292]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-b 2024-05-10T10:49:31.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.b.service' 2024-05-10T10:49:31.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:31.336 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-10T10:49:31.336 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-10T10:49:31.336 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.c 2024-05-10T10:49:31.671 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:31 smithi195 systemd[1]: Stopping Ceph mon.c for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:32.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c[33769]: 2024-05-10T10:49:31.669+0000 7faca56c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:49:32.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c[33769]: 2024-05-10T10:49:31.669+0000 7faca56c7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-10T10:49:32.066 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:31 smithi195 podman[68875]: 2024-05-10 10:49:31.803013133 +0000 UTC m=+0.264167660 container died 249aec4d15a61c0e6b7634e635f6a709d749c30c38fb85493dd0607122380cfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-10T10:49:32.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:32 smithi195 podman[68875]: 2024-05-10 10:49:32.170742624 +0000 UTC m=+0.631897157 container cleanup 249aec4d15a61c0e6b7634e635f6a709d749c30c38fb85493dd0607122380cfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:49:32.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 10 10:49:32 smithi195 bash[68875]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mon-c 2024-05-10T10:49:32.587 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mon.c.service' 2024-05-10T10:49:32.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:32.632 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-10T10:49:32.632 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-10T10:49:32.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.x 2024-05-10T10:49:32.950 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:49:32 smithi112 systemd[1]: Stopping Ceph mgr.x for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:33.285 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:49:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:49:32.949+0000 7f3e53a9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:49:33.285 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:49:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x[30331]: 2024-05-10T10:49:32.949+0000 7f3e53a9d700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-10T10:49:33.285 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:49:32 smithi112 podman[127269]: 2024-05-10 10:49:32.980832343 +0000 UTC m=+0.152908308 container died 5cb65258e8973b548758a4495c0f00fc0a9dfac6a6d44cb7f6ddf663894d5646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-10T10:49:33.579 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:49:33 smithi112 podman[127269]: 2024-05-10 10:49:33.51061407 +0000 UTC m=+0.682690044 container cleanup 5cb65258e8973b548758a4495c0f00fc0a9dfac6a6d44cb7f6ddf663894d5646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-10T10:49:33.580 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 10 10:49:33 smithi112 bash[127269]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-x 2024-05-10T10:49:33.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-10T10:49:34.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:34.012 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-10T10:49:34.012 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-10T10:49:34.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.y 2024-05-10T10:49:34.294 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:34 smithi148 systemd[1]: Stopping Ceph mgr.y for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:34.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:34 smithi148 podman[68414]: 2024-05-10 10:49:34.353427392 +0000 UTC m=+0.144474798 container died 6f205209571bb871ca6eee07c67c7d57026db2d9c88dde622410cbb1ce6622f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-10T10:49:35.010 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:34 smithi148 podman[68414]: 2024-05-10 10:49:34.766922757 +0000 UTC m=+0.557970160 container cleanup 6f205209571bb871ca6eee07c67c7d57026db2d9c88dde622410cbb1ce6622f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-10T10:49:35.010 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:34 smithi148 bash[68414]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y 2024-05-10T10:49:35.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-10T10:49:35.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:35 smithi148 podman[68428]: 2024-05-10 10:49:35.010108774 +0000 UTC m=+0.646575629 container remove 6f205209571bb871ca6eee07c67c7d57026db2d9c88dde622410cbb1ce6622f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-10T10:49:35.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:35 smithi148 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-10T10:49:35.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:35 smithi148 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-05-10T10:49:35.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:35 smithi148 systemd[1]: Stopped Ceph mgr.y for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:49:35.263 INFO:journalctl@ceph.mgr.y.smithi148.stdout:May 10 10:49:35 smithi148 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.y.service: Consumed 13.303s CPU time. 2024-05-10T10:49:35.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:35.276 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-10T10:49:35.276 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-10T10:49:35.276 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.z 2024-05-10T10:49:35.566 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:35 smithi195 systemd[1]: Stopping Ceph mgr.z for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:35.927 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:35 smithi195 podman[68996]: 2024-05-10 10:49:35.613320638 +0000 UTC m=+0.144755697 container stop c839f810009dd06697fd482171633689826292b8717c695db5161bc399c7496b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-10T10:49:35.928 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:35 smithi195 podman[68996]: 2024-05-10 10:49:35.639103049 +0000 UTC m=+0.170538108 container died c839f810009dd06697fd482171633689826292b8717c695db5161bc399c7496b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2) 2024-05-10T10:49:36.203 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:36 smithi195 podman[68996]: 2024-05-10 10:49:36.018401069 +0000 UTC m=+0.549836132 container cleanup c839f810009dd06697fd482171633689826292b8717c695db5161bc399c7496b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-10T10:49:36.204 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:36 smithi195 bash[68996]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z 2024-05-10T10:49:36.443 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-10T10:49:36.464 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:36 smithi195 podman[69010]: 2024-05-10 10:49:36.203374898 +0000 UTC m=+0.584239035 container remove c839f810009dd06697fd482171633689826292b8717c695db5161bc399c7496b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-10T10:49:36.464 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:36 smithi195 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-10T10:49:36.465 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:36 smithi195 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-05-10T10:49:36.465 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:36 smithi195 systemd[1]: Stopped Ceph mgr.z for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:49:36.465 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 10 10:49:36 smithi195 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mgr.z.service: Consumed 13.434s CPU time. 2024-05-10T10:49:36.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:36.477 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-10T10:49:36.477 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-10T10:49:36.477 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.0 2024-05-10T10:49:36.823 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:36 smithi112 systemd[1]: Stopping Ceph osd.0 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:37.079 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0[42393]: 2024-05-10T10:49:36.821+0000 7f31ed600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:49:37.080 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0[42393]: 2024-05-10T10:49:36.821+0000 7f31ed600700 -1 osd.0 94 *** Got signal Terminated *** 2024-05-10T10:49:37.080 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0[42393]: 2024-05-10T10:49:36.821+0000 7f31ed600700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:49:42.440 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:42 smithi112 podman[127392]: 2024-05-10 10:49:42.184961267 +0000 UTC m=+5.519018339 container died a0f16b7e93cc29185963bfa6f2a2915e9cf304ce85c76f3ea4a2d429fd1cb4e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:49:42.830 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:42 smithi112 podman[127392]: 2024-05-10 10:49:42.647920936 +0000 UTC m=+5.981978083 container cleanup a0f16b7e93cc29185963bfa6f2a2915e9cf304ce85c76f3ea4a2d429fd1cb4e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-10T10:49:42.830 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:42 smithi112 bash[127392]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0 2024-05-10T10:49:43.318 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:42 smithi112 podman[127411]: 2024-05-10 10:49:42.94957465 +0000 UTC m=+0.760887098 container remove a0f16b7e93cc29185963bfa6f2a2915e9cf304ce85c76f3ea4a2d429fd1cb4e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:49:43.584 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:43 smithi112 podman[127501]: 2024-05-10 10:49:43.219587462 +0000 UTC m=+0.017913630 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:49:43.584 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:43 smithi112 podman[127501]: 2024-05-10 10:49:43.336890134 +0000 UTC m=+0.135216334 container create ca5abbd4522f55856fbc97fc78320b55629c8cd36cea5795da1d45045784df8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-10T10:49:43.932 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:43 smithi112 podman[127501]: 2024-05-10 10:49:43.629297753 +0000 UTC m=+0.427623924 container init ca5abbd4522f55856fbc97fc78320b55629c8cd36cea5795da1d45045784df8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-10T10:49:43.932 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:43 smithi112 podman[127501]: 2024-05-10 10:49:43.633033594 +0000 UTC m=+0.431359753 container start ca5abbd4522f55856fbc97fc78320b55629c8cd36cea5795da1d45045784df8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-10T10:49:43.933 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:43 smithi112 podman[127501]: 2024-05-10 10:49:43.72082109 +0000 UTC m=+0.519147253 container attach ca5abbd4522f55856fbc97fc78320b55629c8cd36cea5795da1d45045784df8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-10T10:49:44.301 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 10 10:49:43 smithi112 podman[127501]: 2024-05-10 10:49:43.932559234 +0000 UTC m=+0.730885391 container died ca5abbd4522f55856fbc97fc78320b55629c8cd36cea5795da1d45045784df8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-10T10:49:44.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.0.service' 2024-05-10T10:49:44.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:44.775 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-10T10:49:44.775 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-10T10:49:44.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.1 2024-05-10T10:49:45.197 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:44 smithi148 systemd[1]: Stopping Ceph osd.1 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:45.544 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1[38152]: 2024-05-10T10:49:45.195+0000 7fc0955ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:49:45.544 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1[38152]: 2024-05-10T10:49:45.195+0000 7fc0955ee700 -1 osd.1 94 *** Got signal Terminated *** 2024-05-10T10:49:45.545 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1[38152]: 2024-05-10T10:49:45.195+0000 7fc0955ee700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:49:50.981 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:50 smithi148 podman[68540]: 2024-05-10 10:49:50.685282837 +0000 UTC m=+5.716575953 container died ceeb370a15fa6880da7d35a5a100e37144dbf512fa3f154af228902e76f66cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:49:51.290 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 podman[68540]: 2024-05-10 10:49:51.071571913 +0000 UTC m=+6.102865027 container cleanup ceeb370a15fa6880da7d35a5a100e37144dbf512fa3f154af228902e76f66cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:49:51.290 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 bash[68540]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1 2024-05-10T10:49:51.544 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 podman[68559]: 2024-05-10 10:49:51.289677223 +0000 UTC m=+0.604313926 container remove ceeb370a15fa6880da7d35a5a100e37144dbf512fa3f154af228902e76f66cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:49:51.994 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 podman[68652]: 2024-05-10 10:49:51.601836899 +0000 UTC m=+0.018238926 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:49:51.994 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 podman[68652]: 2024-05-10 10:49:51.698938311 +0000 UTC m=+0.115340329 container create 20ada0371aacc0f4fa6ce80c44f57ad23dd460e9070bcf35cf8e48c178231f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-10T10:49:51.994 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 podman[68652]: 2024-05-10 10:49:51.910311504 +0000 UTC m=+0.326713512 container init 20ada0371aacc0f4fa6ce80c44f57ad23dd460e9070bcf35cf8e48c178231f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507) 2024-05-10T10:49:51.994 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 podman[68652]: 2024-05-10 10:49:51.913618584 +0000 UTC m=+0.330020597 container start 20ada0371aacc0f4fa6ce80c44f57ad23dd460e9070bcf35cf8e48c178231f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-10T10:49:51.994 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:51 smithi148 podman[68652]: 2024-05-10 10:49:51.993470846 +0000 UTC m=+0.409872862 container attach 20ada0371aacc0f4fa6ce80c44f57ad23dd460e9070bcf35cf8e48c178231f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:49:52.246 INFO:journalctl@ceph.osd.1.smithi148.stdout:May 10 10:49:52 smithi148 podman[68652]: 2024-05-10 10:49:52.215420891 +0000 UTC m=+0.631822912 container died 20ada0371aacc0f4fa6ce80c44f57ad23dd460e9070bcf35cf8e48c178231f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-10T10:49:52.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.1.service' 2024-05-10T10:49:52.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:49:52.947 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-10T10:49:52.947 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-10T10:49:52.948 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.2 2024-05-10T10:49:53.273 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:53 smithi195 systemd[1]: Stopping Ceph osd.2 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:49:53.566 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2[38935]: 2024-05-10T10:49:53.271+0000 7fbbe6d0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:49:53.566 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2[38935]: 2024-05-10T10:49:53.271+0000 7fbbe6d0f700 -1 osd.2 94 *** Got signal Terminated *** 2024-05-10T10:49:53.566 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2[38935]: 2024-05-10T10:49:53.271+0000 7fbbe6d0f700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:49:59.067 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:58 smithi195 podman[69122]: 2024-05-10 10:49:58.808596397 +0000 UTC m=+5.663451814 container died f139b1b254508337d484c90bf5d716015c1e5084b3db6a32a5abe0c9d356ba2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-05-10T10:49:59.694 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:59 smithi195 podman[69122]: 2024-05-10 10:49:59.189581455 +0000 UTC m=+6.044436879 container cleanup f139b1b254508337d484c90bf5d716015c1e5084b3db6a32a5abe0c9d356ba2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-10T10:49:59.694 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:59 smithi195 bash[69122]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2 2024-05-10T10:49:59.809 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:59 smithi195 podman[69153]: 2024-05-10 10:49:59.374594953 +0000 UTC m=+0.563029874 container remove f139b1b254508337d484c90bf5d716015c1e5084b3db6a32a5abe0c9d356ba2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-10T10:50:00.067 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:59 smithi195 podman[69246]: 2024-05-10 10:49:59.711366227 +0000 UTC m=+0.018437302 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:00.067 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:49:59 smithi195 podman[69246]: 2024-05-10 10:49:59.83388084 +0000 UTC m=+0.140951917 container create 966e20c5cc56e31b1765637056c6dd9c2c91e54d2a4a7ffaab45ae02b8808d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-10T10:50:00.374 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:50:00 smithi195 podman[69246]: 2024-05-10 10:50:00.070052969 +0000 UTC m=+0.377124043 container init 966e20c5cc56e31b1765637056c6dd9c2c91e54d2a4a7ffaab45ae02b8808d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD) 2024-05-10T10:50:00.375 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:50:00 smithi195 podman[69246]: 2024-05-10 10:50:00.073425806 +0000 UTC m=+0.380496881 container start 966e20c5cc56e31b1765637056c6dd9c2c91e54d2a4a7ffaab45ae02b8808d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD) 2024-05-10T10:50:00.375 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:50:00 smithi195 podman[69246]: 2024-05-10 10:50:00.145125318 +0000 UTC m=+0.452196414 container attach 966e20c5cc56e31b1765637056c6dd9c2c91e54d2a4a7ffaab45ae02b8808d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-10T10:50:00.732 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 10 10:50:00 smithi195 podman[69246]: 2024-05-10 10:50:00.374230292 +0000 UTC m=+0.681301378 container died 966e20c5cc56e31b1765637056c6dd9c2c91e54d2a4a7ffaab45ae02b8808d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T10:50:01.094 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.2.service' 2024-05-10T10:50:01.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:01.136 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-10T10:50:01.137 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-10T10:50:01.137 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.3 2024-05-10T10:50:01.445 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:01 smithi112 systemd[1]: Stopping Ceph osd.3 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:01.829 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:01 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3[49336]: 2024-05-10T10:50:01.444+0000 7fddfeda4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:01.830 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:01 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3[49336]: 2024-05-10T10:50:01.444+0000 7fddfeda4700 -1 osd.3 94 *** Got signal Terminated *** 2024-05-10T10:50:01.830 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:01 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3[49336]: 2024-05-10T10:50:01.444+0000 7fddfeda4700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:04.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:03 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:03.870+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:05.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:04 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:04.950+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:05.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:04 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:04.898+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:05.811 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:05.501+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:06.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:05.810+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:06.257 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:05 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:05.901+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:06.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:05 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:05.916+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:06.579 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3[49336]: 2024-05-10T10:50:06.257+0000 7fddf7390700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.332545+0000 front 2024-05-10T10:49:40.332565+0000 (oldest deadline 2024-05-10T10:50:05.632674+0000) 2024-05-10T10:50:06.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:06.465+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:07.129 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:06 smithi112 podman[127608]: 2024-05-10 10:50:06.877175946 +0000 UTC m=+5.554614462 container died c43bc1a2a31aa38ea75a7740b1860e07caef96195069e2ffd8f0c94bcef07310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507) 2024-05-10T10:50:07.129 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:06 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:06.942+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:07.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:06 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:06.958+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:07.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:06.816+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:07.521 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:07 smithi112 podman[127608]: 2024-05-10 10:50:07.31989808 +0000 UTC m=+5.997336628 container cleanup c43bc1a2a31aa38ea75a7740b1860e07caef96195069e2ffd8f0c94bcef07310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-10T10:50:07.522 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:07 smithi112 bash[127608]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3 2024-05-10T10:50:07.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:07.438+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:07.829 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:07 smithi112 podman[127625]: 2024-05-10 10:50:07.521327875 +0000 UTC m=+0.644234015 container remove c43bc1a2a31aa38ea75a7740b1860e07caef96195069e2ffd8f0c94bcef07310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-10T10:50:08.191 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:07 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:07.912+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:08.191 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:07 smithi112 podman[127720]: 2024-05-10 10:50:07.842687608 +0000 UTC m=+0.017981816 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:08.192 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:07 smithi112 podman[127720]: 2024-05-10 10:50:07.947148765 +0000 UTC m=+0.122442955 container create 40fca00f216bc6f974378108eb42c41cedfc1ce776e933fc27d604cbf73788aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-10T10:50:08.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:07 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:07.910+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:08.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:07.821+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:08.491 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:08 smithi112 podman[127720]: 2024-05-10 10:50:08.19133806 +0000 UTC m=+0.366632260 container init 40fca00f216bc6f974378108eb42c41cedfc1ce776e933fc27d604cbf73788aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0) 2024-05-10T10:50:08.491 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:08 smithi112 podman[127720]: 2024-05-10 10:50:08.194698837 +0000 UTC m=+0.369993027 container start 40fca00f216bc6f974378108eb42c41cedfc1ce776e933fc27d604cbf73788aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:50:08.491 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:08 smithi112 podman[127720]: 2024-05-10 10:50:08.291467236 +0000 UTC m=+0.466761425 container attach 40fca00f216bc6f974378108eb42c41cedfc1ce776e933fc27d604cbf73788aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:50:08.774 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:08.465+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:08.803 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 10 10:50:08 smithi112 podman[127720]: 2024-05-10 10:50:08.490601469 +0000 UTC m=+0.665895663 container died 40fca00f216bc6f974378108eb42c41cedfc1ce776e933fc27d604cbf73788aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-3-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-10T10:50:09.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:08.773+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:09.078 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:08 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:08.875+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:09.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.3.service' 2024-05-10T10:50:09.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:09.142 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-10T10:50:09.142 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-10T10:50:09.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.4 2024-05-10T10:50:09.173 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:08 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:08.923+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:09.469 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:09 smithi148 systemd[1]: Stopping Ceph osd.4 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:09.743 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:09.442+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:09.794 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:09 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4[44919]: 2024-05-10T10:50:09.468+0000 7fe5adf1d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:09.795 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:09 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4[44919]: 2024-05-10T10:50:09.468+0000 7fe5adf1d700 -1 osd.4 94 *** Got signal Terminated *** 2024-05-10T10:50:09.795 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:09 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4[44919]: 2024-05-10T10:50:09.468+0000 7fe5adf1d700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:10.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:09.742+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:10.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:09 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:09.898+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:10.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:09 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:09.884+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:10.755 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:10.458+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:11.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:10.753+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:11.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:10 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:10.849+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:11.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:10 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:10.920+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:11.800 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:11 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:11.450+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:12.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:11 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:11.799+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:12.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:11 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:11.861+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:12.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:11 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:11.916+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:12.402 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:12 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:12.089+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:12.806 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:12 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:12.401+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:13.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:12 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:12.805+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:13.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:12 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:12.825+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:13.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:12 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:12.939+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:13.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:12 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:12.939+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:13.432 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:13 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:13.093+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:13.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:13 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:13.431+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:14.078 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:13 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:13.815+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:14.079 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:13 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:13.815+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:14.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:13 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:13.848+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:14.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:13 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:13.898+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:14.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:13 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:13.898+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:14.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:13 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:13.828+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:14.446 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:14.077+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:14.771 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:14.446+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:15.049 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:14.770+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:15.049 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:14 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:14.770+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:15.111 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:14 smithi148 podman[68773]: 2024-05-10 10:50:14.823229636 +0000 UTC m=+5.480522156 container died 3fe9ad244e0f03865412c60c38353d4340504ff52dde2f16a89f1243f5d7af90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:50:15.112 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:14.801+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:15.112 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:14.862+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:15.112 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:14 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:14.862+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:15.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:15 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:15.048+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:15.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:14.873+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:15.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:14.925+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:15.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:14 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:14.925+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:15.461 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:15 smithi148 podman[68773]: 2024-05-10 10:50:15.201320957 +0000 UTC m=+5.858613480 container cleanup 3fe9ad244e0f03865412c60c38353d4340504ff52dde2f16a89f1243f5d7af90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-10T10:50:15.461 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:15 smithi148 bash[68773]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4 2024-05-10T10:50:15.783 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:15 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:15.464+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:15.794 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:15 smithi148 podman[68790]: 2024-05-10 10:50:15.461309357 +0000 UTC m=+0.635831741 container remove 3fe9ad244e0f03865412c60c38353d4340504ff52dde2f16a89f1243f5d7af90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:50:16.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:15 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:15.781+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:16.066 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:15 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:15.781+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:16.098 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:15 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:15.834+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:16.099 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:15 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:15.861+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:16.099 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:15 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:15.861+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:16.099 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:15 smithi148 podman[68879]: 2024-05-10 10:50:15.719950961 +0000 UTC m=+0.018453535 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:16.099 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:15 smithi148 podman[68879]: 2024-05-10 10:50:15.832502579 +0000 UTC m=+0.131005181 container create ced34f00ddcea1ce4db6cf6749370af3bf170527ba9fbaf90bc3eeb663e6c41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=) 2024-05-10T10:50:16.099 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:16 smithi148 podman[68879]: 2024-05-10 10:50:16.098364294 +0000 UTC m=+0.396866859 container init ced34f00ddcea1ce4db6cf6749370af3bf170527ba9fbaf90bc3eeb663e6c41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T10:50:16.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:15 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:15.901+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:16.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:15 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:15.949+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:16.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:15 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:15.949+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:16.395 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:16 smithi148 podman[68879]: 2024-05-10 10:50:16.101878872 +0000 UTC m=+0.400381434 container start ced34f00ddcea1ce4db6cf6749370af3bf170527ba9fbaf90bc3eeb663e6c41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:50:16.395 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:16 smithi148 podman[68879]: 2024-05-10 10:50:16.20682375 +0000 UTC m=+0.505326320 container attach ced34f00ddcea1ce4db6cf6749370af3bf170527ba9fbaf90bc3eeb663e6c41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507) 2024-05-10T10:50:16.479 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:16 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:16.083+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:16.769 INFO:journalctl@ceph.osd.4.smithi148.stdout:May 10 10:50:16 smithi148 podman[68879]: 2024-05-10 10:50:16.395512883 +0000 UTC m=+0.694015450 container died ced34f00ddcea1ce4db6cf6749370af3bf170527ba9fbaf90bc3eeb663e6c41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-10T10:50:16.815 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:16 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:16.478+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:17.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:16 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:16.866+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:17.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:16 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:16.834+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:17.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:16 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:16.834+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:17.069 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.4.service' 2024-05-10T10:50:17.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:17.150 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-10T10:50:17.150 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-10T10:50:17.150 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.5 2024-05-10T10:50:17.178 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:17 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:17.047+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:17.178 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:16 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:16.830+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:17.178 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:16 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:16.830+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:17.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:16 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:16.874+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:17.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:16 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:16.956+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:17.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:16 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:16.956+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:17.444 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:17 smithi195 systemd[1]: Stopping Ceph osd.5 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:17.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:17 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:17.500+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:17.816 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:17 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:17.443+0000 7f86d0542700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:17.816 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:17 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:17.443+0000 7f86d0542700 -1 osd.5 94 *** Got signal Terminated *** 2024-05-10T10:50:17.816 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:17 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:17.443+0000 7f86d0542700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:18.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:17 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:17.873+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:18.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:17 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:17.785+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:18.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:17 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:17.785+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:18.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:18 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:18.030+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:18.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:17 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:17.872+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:18.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:17 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:17.872+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:18.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:17 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:17.904+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:18.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:17 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:17.956+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:18.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:17 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:17.956+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:18.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:18 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:18.521+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:19.295 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:18 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:18.828+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:19.295 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:18 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:18.799+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:19.295 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:18 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:18.799+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:19.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:18 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:18.982+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:19.317 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:18 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:18.873+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:19.317 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:18 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:18.873+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:19.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:18 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:18.946+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:19.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:18 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:18.989+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:19.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:18 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:18.989+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:19.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:19 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:19.538+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:19.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:19 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:19.538+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:20.294 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:19 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:19.794+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:20.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:19 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:19.815+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:20.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:19 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:19.815+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:20.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:19 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:19.997+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:20.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:19 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:19.840+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:20.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:19 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:19.840+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:20.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:20 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:20.016+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:20.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:20 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:20.016+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:20.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:19 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:19.928+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:20.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:19 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:19.928+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:20.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:20 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:20.508+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:20.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:20 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:20.508+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:21.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:20 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:20.747+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:21.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:20 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:20.811+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:21.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:20 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:20.811+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:21.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:20 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:20.856+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:21.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:20 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:20.856+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:21.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:20 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:20.969+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:21.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:20 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:20.969+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:21.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:21 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:21.013+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:21.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:21 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:21.013+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:21.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:20 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:20.908+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:21.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:20 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:20.908+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:21.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:21 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:21.502+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:21.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:21 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:21.502+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:22.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:21 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:21.819+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:22.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:21 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:21.819+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:22.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:21 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:21.722+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:22.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:21 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:21.722+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:22.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:21 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:21.998+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:22.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:21 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:21.998+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:22.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:21 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:21.906+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:41.648801+0000 front 2024-05-10T10:49:41.648961+0000 (oldest deadline 2024-05-10T10:50:05.748799+0000) 2024-05-10T10:50:22.316 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:21 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5[45829]: 2024-05-10T10:50:21.906+0000 7f86ccb36700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.750479+0000 front 2024-05-10T10:49:49.750402+0000 (oldest deadline 2024-05-10T10:50:13.250325+0000) 2024-05-10T10:50:22.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:21 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:21.949+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:22.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:21 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:21.949+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:22.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:22 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:22.002+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:22.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:22 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:22.002+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:22.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:22 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:22.002+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:22.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:22 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:22.477+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:22.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:22 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:22.477+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:23.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:22 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:22.747+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:23.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:22 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:22.747+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:23.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:22 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:22.834+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:23.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:22 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:22.834+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:23.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:22 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:22.834+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:23.262 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:22 smithi195 podman[69352]: 2024-05-10 10:50:22.982491531 +0000 UTC m=+5.639265429 container died 6659666bc9a9e5748dd71a11d68755bfef3df5e1b2b05d798fb3713586c1366b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-10T10:50:23.262 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:23 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:23.043+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:23.262 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:23 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:23.043+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:23.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:22 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:22.916+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:23.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:22 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:22.916+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:23.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:23 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:23.022+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:23.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:23 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:23.022+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:23.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:23 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:23.022+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:23.566 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:23 smithi195 podman[69352]: 2024-05-10 10:50:23.352627657 +0000 UTC m=+6.009401562 container cleanup 6659666bc9a9e5748dd71a11d68755bfef3df5e1b2b05d798fb3713586c1366b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-10T10:50:23.566 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:23 smithi195 bash[69352]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5 2024-05-10T10:50:23.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:23 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:23.476+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:23.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:23 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:23.476+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:23.966 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:23 smithi195 podman[69370]: 2024-05-10 10:50:23.620825406 +0000 UTC m=+0.632730881 container remove 6659666bc9a9e5748dd71a11d68755bfef3df5e1b2b05d798fb3713586c1366b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:50:24.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:23 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:23.777+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:24.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:23 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:23.777+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:24.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:23 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:23.833+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:24.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:23 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:23.833+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:24.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:23 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:23.833+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:24.224 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:24 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:24.075+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:24.225 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:24 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:24.075+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:24.225 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:23 smithi195 podman[69463]: 2024-05-10 10:50:23.869592075 +0000 UTC m=+0.018163943 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:24.225 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:23 smithi195 podman[69463]: 2024-05-10 10:50:23.988264293 +0000 UTC m=+0.136836150 container create 94cf6df12e981ebe131573691721155a7006fbb8b3580f1e5054999e91b78b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-10T10:50:24.225 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:24 smithi195 podman[69463]: 2024-05-10 10:50:24.224510441 +0000 UTC m=+0.373082306 container init 94cf6df12e981ebe131573691721155a7006fbb8b3580f1e5054999e91b78b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-10T10:50:24.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:23 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:23.909+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:24.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:23 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:23.909+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:24.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:24 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:24.004+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:24.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:24 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:24.004+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:24.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:24 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:24.004+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:24.519 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:24 smithi195 podman[69463]: 2024-05-10 10:50:24.22780536 +0000 UTC m=+0.376377220 container start 94cf6df12e981ebe131573691721155a7006fbb8b3580f1e5054999e91b78b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux ) 2024-05-10T10:50:24.519 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:24 smithi195 podman[69463]: 2024-05-10 10:50:24.307832832 +0000 UTC m=+0.456404713 container attach 94cf6df12e981ebe131573691721155a7006fbb8b3580f1e5054999e91b78b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-10T10:50:24.816 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 10 10:50:24 smithi195 podman[69463]: 2024-05-10 10:50:24.526552568 +0000 UTC m=+0.675124427 container died 94cf6df12e981ebe131573691721155a7006fbb8b3580f1e5054999e91b78b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-10T10:50:24.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:24 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:24.517+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:24.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:24 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:24.517+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:25.145 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.5.service' 2024-05-10T10:50:25.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:25.226 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-10T10:50:25.226 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-10T10:50:25.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.6 2024-05-10T10:50:25.252 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:24 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:24.868+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:25.252 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:24 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:24.868+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:25.253 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:25.002+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:25.253 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:25.002+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:25.253 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:25.002+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:25.294 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:24 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:24.817+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:25.294 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:24 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:24.817+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:25.295 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:24 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:24.821+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:25.295 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:24 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:24.821+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:25.295 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:24 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:24.821+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:25.401 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:25 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:25.111+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:25.401 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:25 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:25.111+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:25.540 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:25 smithi112 systemd[1]: Stopping Ceph osd.6 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:25.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:25 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:25.478+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:25.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:25 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:25.478+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:25.829 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:25.539+0000 7fccea053700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:25.829 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:25.539+0000 7fccea053700 -1 osd.6 94 *** Got signal Terminated *** 2024-05-10T10:50:25.829 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:25.539+0000 7fccea053700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:26.294 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:25 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:25.799+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:26.294 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:25 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:25.799+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:26.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:25 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:25.805+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:26.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:25 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:25.805+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:26.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:25 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:25.805+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:26.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:25.832+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:26.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:25.832+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:26.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:25.990+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:26.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:25.990+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:26.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:25 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:25.990+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:26.456 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:26 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:26.134+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:26.457 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:26 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:26.134+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:26.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:26 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:26.456+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:26.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:26 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:26.456+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:27.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:26 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:26.786+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:27.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:26 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:26.786+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:27.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:26 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:26.786+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:27.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:26 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:26.765+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:27.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:26 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:26.765+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:27.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:26.848+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:27.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:26.848+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:27.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:26.990+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:27.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:26.990+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:27.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:26.990+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:27.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:26 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:26.990+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:27.478 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:27 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:27.168+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:27.478 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:27 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:27.168+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:27.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:27 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:27.477+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:27.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:27 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:27.477+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:28.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:27 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:27.802+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:28.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:27 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:27.802+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:28.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:27 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:27.802+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:28.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:27 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:27.792+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:28.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:27 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:27.792+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:28.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:27 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:27.792+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:28.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:27 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:27.792+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:28.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:27 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:27.897+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:28.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:27 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:27.897+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:28.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.015+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:28.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.015+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:28.331 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.015+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:28.331 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.015+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:28.525 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:28 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:28.203+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:28.525 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:28 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:28.203+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:28.525 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:28 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:28.203+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:28.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:28 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:28.524+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:28.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:28 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:28.524+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:28.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:28 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:28.524+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:29.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:28 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:28.781+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:29.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:28 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:28.781+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:29.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:28 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:28.781+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:29.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:28 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:28.778+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:29.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:28 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:28.778+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:29.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:28 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:28.778+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:29.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:28 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:28.778+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:29.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:28.860+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:29.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:28.860+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:29.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.997+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:29.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.997+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:29.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.997+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:29.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:28.997+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:29.508 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:29 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:29.160+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:29.508 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:29 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:29.160+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:29.508 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:29 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:29.160+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:29.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:29 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:29.507+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:29.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:29 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:29.507+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:29.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:29 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:29.507+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:30.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:29 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:29.757+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:30.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:29 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:29.757+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:30.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:29 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:29.757+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:30.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:29 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:29.733+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:30.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:29 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:29.733+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:30.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:29 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:29.733+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:30.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:29 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:29.733+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:30.329 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:29.853+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.055261+0000 front 2024-05-10T10:49:49.055306+0000 (oldest deadline 2024-05-10T10:50:13.155424+0000) 2024-05-10T10:50:30.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:29.853+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.456339+0000 front 2024-05-10T10:49:55.456308+0000 (oldest deadline 2024-05-10T10:50:19.556477+0000) 2024-05-10T10:50:30.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6[56026]: 2024-05-10T10:50:29.853+0000 7fcce263f700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.357862+0000 front 2024-05-10T10:50:05.357891+0000 (oldest deadline 2024-05-10T10:50:29.458038+0000) 2024-05-10T10:50:30.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:29.983+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:30.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:29.983+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:30.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:29.983+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:30.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:29 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:29.983+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:30.462 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:30 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:30.162+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:30.462 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:30 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:30.163+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:30.462 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:30 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:30.163+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:30.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:30 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:30.460+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:30.817 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:30 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:30.460+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:30.817 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:30 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:30.460+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:31.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:30.735+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:31.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:30.735+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:31.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:30.735+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:31.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:30.771+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:31.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:30.771+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:31.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:30.771+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:31.046 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:30.771+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:31.330 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:31 smithi112 podman[127828]: 2024-05-10 10:50:31.076457636 +0000 UTC m=+5.659216999 container died 43170962fdb46a7049223436c665c2a94503823a3ff035b52c22fcf74d8d5e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:50:31.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:30.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:31.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:30.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:31.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:30.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:31.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:30 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:30.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:31.444 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:31.146+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:31.445 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:31.146+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:31.445 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:31.146+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:31.732 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:31 smithi112 podman[127828]: 2024-05-10 10:50:31.51440014 +0000 UTC m=+6.097159516 container cleanup 43170962fdb46a7049223436c665c2a94503823a3ff035b52c22fcf74d8d5e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-10T10:50:31.732 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:31 smithi112 bash[127828]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6 2024-05-10T10:50:31.817 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:31.444+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:31.817 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:31.444+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:31.817 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:31 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:31.444+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:32.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:31 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:31.723+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:32.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:31 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:31.723+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:32.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:31 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:31.723+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:32.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:31 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:31.748+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:32.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:31 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:31.748+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:32.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:31 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:31.748+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:32.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:31 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:31.748+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:32.079 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:31 smithi112 podman[127846]: 2024-05-10 10:50:31.732502329 +0000 UTC m=+0.654018681 container remove 43170962fdb46a7049223436c665c2a94503823a3ff035b52c22fcf74d8d5e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:50:32.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:31.938+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:32.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:31.938+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:32.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:31.938+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:32.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:31 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:31.938+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:32.395 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:32 smithi112 podman[127930]: 2024-05-10 10:50:32.037056774 +0000 UTC m=+0.017689777 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:32.395 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:32 smithi112 podman[127930]: 2024-05-10 10:50:32.14997841 +0000 UTC m=+0.130611396 container create 10b0700c3ddc771f655b5fd306d68d4ebc56b84565b4ba4bae17627bc3d79e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-10T10:50:32.479 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:32 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:32.156+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:32.480 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:32 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:32.156+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:32.480 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:32 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:32.156+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:32.693 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:32 smithi112 podman[127930]: 2024-05-10 10:50:32.394504277 +0000 UTC m=+0.375137272 container init 10b0700c3ddc771f655b5fd306d68d4ebc56b84565b4ba4bae17627bc3d79e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:50:32.693 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:32 smithi112 podman[127930]: 2024-05-10 10:50:32.398108712 +0000 UTC m=+0.378741707 container start 10b0700c3ddc771f655b5fd306d68d4ebc56b84565b4ba4bae17627bc3d79e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T10:50:32.693 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:32 smithi112 podman[127930]: 2024-05-10 10:50:32.502402135 +0000 UTC m=+0.483035132 container attach 10b0700c3ddc771f655b5fd306d68d4ebc56b84565b4ba4bae17627bc3d79e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-10T10:50:32.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:32 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:32.479+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:32.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:32 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:32.479+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:32.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:32 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:32.479+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:32.989 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 10 10:50:32 smithi112 podman[127930]: 2024-05-10 10:50:32.693573977 +0000 UTC m=+0.674206973 container died 10b0700c3ddc771f655b5fd306d68d4ebc56b84565b4ba4bae17627bc3d79e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:50:32.989 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:32.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:32.989 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:32.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:32.989 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:32.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:32.989 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:32 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:32.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:33.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:32 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:32.731+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:33.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:32 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:32.731+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:33.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:32 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:32.731+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:33.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:32 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:32.767+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:33.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:32 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:32.767+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:33.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:32 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:32.767+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:33.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:32 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:32.767+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:33.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.6.service' 2024-05-10T10:50:33.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:33.380 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-10T10:50:33.381 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-10T10:50:33.381 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.7 2024-05-10T10:50:33.433 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:33 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:33.112+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:33.433 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:33 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:33.112+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:33.433 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:33 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:33.112+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:33.698 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:33 smithi148 systemd[1]: Stopping Ceph osd.7 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:33.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:33 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:33.433+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:33.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:33 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:33.433+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:33.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:33 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:33.433+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:34.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:33.807+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:34.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:33.807+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:34.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:33.807+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:34.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:33.807+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:34.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:33.697+0000 7ff6029ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:34.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:33.697+0000 7ff6029ab700 -1 osd.7 94 *** Got signal Terminated *** 2024-05-10T10:50:34.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:33.697+0000 7ff6029ab700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:34.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:33.700+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:34.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:33.700+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:34.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:33 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:33.700+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:34.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:33 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:33.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:34.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:33 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:33.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:34.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:33 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:33.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:34.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:33 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:33.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:34.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:34 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:34.117+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:34.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:34 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:34.117+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:34.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:34 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:34.117+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:34.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:34 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:34.409+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:34.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:34 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:34.409+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:34.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:34 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:34.409+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:35.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:34 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:34.723+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:35.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:34 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:34.723+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:35.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:34 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:34.723+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:35.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:34 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:34.798+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:35.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:34 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:34.798+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:35.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:34 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:34.798+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:35.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:34 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:34.798+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:35.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:34 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:34.946+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:35.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:34 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:34.946+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:35.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:34 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:34.946+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:35.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:34 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:34.946+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:35.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:35 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:35.136+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:35.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:35 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:35.136+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:35.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:35 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:35.136+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:35.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:35 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:35.372+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:35.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:35 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:35.372+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:35.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:35 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:35.372+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:35.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:35 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:35.372+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:36.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:35.760+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:36.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:35.760+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:36.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:35.760+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:36.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:35.776+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:36.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:35.776+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:36.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:35.776+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:36.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:35.776+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:36.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:35 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:35.939+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:36.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:35 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:35.939+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:36.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:35 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:35.939+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:36.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:35 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:35.939+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:36.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:36.172+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:36.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:36.172+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:36.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:36.172+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:36.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:36.172+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:36.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:36.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:36.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:36.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:36.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:36.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:36.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:36 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:36.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:37.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:36.765+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:37.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:36.765+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:37.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:36.765+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:37.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:36.731+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:37.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:36.731+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:37.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:36.731+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:37.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:36.731+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:37.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:36 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:36.731+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:37.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:36.985+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:37.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:36.985+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:37.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:36.985+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:37.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:36.985+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:37.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:36 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:36.985+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:37.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:37.334+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:37.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:37.334+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:37.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:37.334+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:37.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:37.334+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:37.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:37.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:37.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:37.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:37.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:37.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:37.567 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:37 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:37.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:38.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:37.748+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:48.243752+0000 front 2024-05-10T10:49:48.243778+0000 (oldest deadline 2024-05-10T10:50:14.143961+0000) 2024-05-10T10:50:38.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:37.748+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.245053+0000 front 2024-05-10T10:49:55.245082+0000 (oldest deadline 2024-05-10T10:50:21.145252+0000) 2024-05-10T10:50:38.045 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7[51423]: 2024-05-10T10:50:37.748+0000 7ff5faf97700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.047656+0000 front 2024-05-10T10:50:05.047675+0000 (oldest deadline 2024-05-10T10:50:27.248160+0000) 2024-05-10T10:50:38.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:37.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:38.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:37.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:38.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:37.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:38.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:37.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:38.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:37 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:37.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:38.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:37 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:37.943+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:38.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:37 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:37.943+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:38.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:37 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:37.943+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:38.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:37 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:37.943+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:38.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:37 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:37.943+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:38.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:38.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:38.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:38.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:38.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:38.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:38.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:38.130+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:38.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:38.362+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:38.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:38.362+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:38.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:38.362+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:38.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:38 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:38.362+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:39.135 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:38 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:38.796+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:39.135 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:38 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:38.796+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:39.135 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:38 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:38.796+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:39.135 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:38 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:38.796+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:39.135 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:38 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:38.796+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:39.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:38 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:38.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:39.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:38 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:38.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:39.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:38 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:38.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:39.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:38 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:38.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:39.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:38 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:38.959+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:39.390 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:39 smithi148 podman[68996]: 2024-05-10 10:50:39.134274367 +0000 UTC m=+5.554730901 container died 933ad6e6d139231e6ef88aba9adc560ff966aa85fe1cdfdf48be6bdfa9ea2af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-10T10:50:39.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:39.136+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:39.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:39.136+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:39.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:39.136+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:39.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:39.136+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:39.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:39.328+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:39.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:39.328+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:39.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:39.328+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:39.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:39 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:39.328+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:39.766 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:39 smithi148 podman[68996]: 2024-05-10 10:50:39.490270473 +0000 UTC m=+5.910727023 container cleanup 933ad6e6d139231e6ef88aba9adc560ff966aa85fe1cdfdf48be6bdfa9ea2af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T10:50:39.766 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:39 smithi148 bash[68996]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7 2024-05-10T10:50:40.044 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:39 smithi148 podman[69014]: 2024-05-10 10:50:39.766951465 +0000 UTC m=+0.626922881 container remove 933ad6e6d139231e6ef88aba9adc560ff966aa85fe1cdfdf48be6bdfa9ea2af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, ceph=True) 2024-05-10T10:50:40.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:39 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:39.765+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:40.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:39 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:39.765+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:40.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:39 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:39.765+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:40.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:39 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:39.765+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:40.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:39 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:39.765+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:40.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:39 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:39.923+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:40.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:39 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:39.923+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:40.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:39 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:39.923+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:40.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:39 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:39.923+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:40.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:39 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:39.923+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:40.421 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:40 smithi148 podman[69108]: 2024-05-10 10:50:40.0075581 +0000 UTC m=+0.018028543 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:40.421 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:40 smithi148 podman[69108]: 2024-05-10 10:50:40.134939927 +0000 UTC m=+0.145410372 container create 18aae7c902f097f91ce88f46647b5886b2122c8109622583c7cc112be0af9d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5) 2024-05-10T10:50:40.421 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:40 smithi148 podman[69108]: 2024-05-10 10:50:40.329571728 +0000 UTC m=+0.340042171 container init 18aae7c902f097f91ce88f46647b5886b2122c8109622583c7cc112be0af9d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-05-10T10:50:40.421 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:40 smithi148 podman[69108]: 2024-05-10 10:50:40.332790566 +0000 UTC m=+0.343261009 container start 18aae7c902f097f91ce88f46647b5886b2122c8109622583c7cc112be0af9d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-10T10:50:40.421 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:40 smithi148 podman[69108]: 2024-05-10 10:50:40.421127676 +0000 UTC m=+0.431598140 container attach 18aae7c902f097f91ce88f46647b5886b2122c8109622583c7cc112be0af9d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-10T10:50:40.570 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:40.312+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:40.572 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:40.312+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:40.572 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:40.312+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:40.572 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:40.312+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:40.574 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:40.093+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:40.574 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:40.093+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:40.574 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:40.093+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:40.575 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:40 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:40.093+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:40.758 INFO:journalctl@ceph.osd.7.smithi148.stdout:May 10 10:50:40 smithi148 podman[69108]: 2024-05-10 10:50:40.631083567 +0000 UTC m=+0.641554010 container died 18aae7c902f097f91ce88f46647b5886b2122c8109622583c7cc112be0af9d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5) 2024-05-10T10:50:41.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:40 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:40.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:41.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:40 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:40.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:41.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:40 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:40.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:41.046 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:40 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:40.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:41.046 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:40 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:40.754+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:41.314 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.7.service' 2024-05-10T10:50:41.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:40 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:40.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:41.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:40 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:40.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:41.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:40 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:40.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:41.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:40 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:40.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:41.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:40 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:40.960+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:41.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:41.359 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-10T10:50:41.359 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-10T10:50:41.359 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.8 2024-05-10T10:50:41.388 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:41.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:41.388 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:41.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:41.388 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:41.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:41.388 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:41.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:41.389 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:41.135+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:41.389 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:41.135+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:41.389 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:41.135+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:41.389 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:41.135+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:41.673 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 systemd[1]: Stopping Ceph osd.8 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:41.673 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:41.671+0000 7f62cc5a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:41.673 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:41.671+0000 7f62cc5a3700 -1 osd.8 94 *** Got signal Terminated *** 2024-05-10T10:50:41.673 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:41 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:41.671+0000 7f62cc5a3700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:42.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:41 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:41.724+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:42.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:41 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:41.724+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:42.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:41 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:41.724+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:42.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:41 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:41.724+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:42.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:41 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:41.724+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:42.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:41.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:42.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:41.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:42.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:41.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:42.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:41.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:42.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:41 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:41.914+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:42.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:42.313+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:42.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:42.313+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:42.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:42.313+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:42.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:42.313+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:42.567 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:42.128+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:42.567 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:42.128+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:42.567 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:42.128+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:42.567 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:42 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:42.128+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:42.794 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:42 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:42.749+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:42.794 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:42 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:42.749+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:42.794 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:42 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:42.749+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:42.794 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:42 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:42.749+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:42.795 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:42 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:42.749+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:43.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:42 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:42.926+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:43.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:42 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:42.926+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:43.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:42 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:42.926+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:43.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:42 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:42.926+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:43.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:42 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:42.926+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:43.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:43.160+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:43.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:43.160+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:43.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:43.160+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:43.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:43.160+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:43.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:43.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:43.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:43.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:43.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:43.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:43.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:43 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:43.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:44.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:43.710+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:44.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:43.710+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:44.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:43.710+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:44.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:43.710+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:44.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:43 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:43.710+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:44.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:43.944+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:44.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:43.944+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:44.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:43.944+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:44.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:43.944+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:44.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:43 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:43.944+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:44.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:44.133+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:44.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:44.133+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:44.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:44.133+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:44.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:44.133+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:44.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:44.344+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:44.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:44.344+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:44.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:44.344+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:44.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:44 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:44.344+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:45.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:44.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:45.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:44.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:45.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:44.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:45.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:44.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:45.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:44.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:45.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:44.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:45.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:44.917+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:45.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:44.917+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:45.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:44.917+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:45.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:44.917+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:45.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:44 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:44.917+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:45.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:45.156+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:45.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:45.156+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:45.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:45.156+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:45.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:45.156+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:45.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:45.342+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:45.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:45.342+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:45.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:45.342+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:45.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:45 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:45.342+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:46.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:45.675+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:46.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:45.675+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:46.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:45.675+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:46.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:45.675+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:46.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:45.675+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:46.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:45 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:45.675+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:46.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:45.891+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:46.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:45.891+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:46.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:45.891+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:46.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:45.891+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:46.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:45.891+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:46.331 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:45 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:45.891+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:46.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:46.205+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:45.845693+0000 front 2024-05-10T10:49:45.845779+0000 (oldest deadline 2024-05-10T10:50:11.145689+0000) 2024-05-10T10:50:46.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:46.205+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:55.846479+0000 front 2024-05-10T10:49:55.846533+0000 (oldest deadline 2024-05-10T10:50:20.546630+0000) 2024-05-10T10:50:46.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:46.205+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:02.247733+0000 front 2024-05-10T10:50:02.247721+0000 (oldest deadline 2024-05-10T10:50:27.547630+0000) 2024-05-10T10:50:46.566 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8[52337]: 2024-05-10T10:50:46.205+0000 7f62c8b97700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.749040+0000 front 2024-05-10T10:50:12.749018+0000 (oldest deadline 2024-05-10T10:50:35.649067+0000) 2024-05-10T10:50:46.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:46.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:46.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:46.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:46.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:46.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:46.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:46 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:46.327+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:47.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:46.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:47.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:46.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:47.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:46.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:47.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:46.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:47.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:46.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:47.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:46 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:46.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:47.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:46.846+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:47.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:46.846+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:47.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:46.846+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:47.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:46.846+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:47.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:46.846+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:47.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:46 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:46.846+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:47.515 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:47 smithi195 podman[69580]: 2024-05-10 10:50:47.258942028 +0000 UTC m=+5.704007959 container died e50ad85efcca68f6b5426dbb31a32997f065e1d1d7b115f2bb1c92c8baedb141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:50:47.515 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:47.348+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:47.516 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:47.348+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:47.516 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:47.348+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:47.516 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:47.348+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:47.516 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:47 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:47.348+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:47.799 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:47 smithi195 podman[69580]: 2024-05-10 10:50:47.572329433 +0000 UTC m=+6.017395367 container cleanup e50ad85efcca68f6b5426dbb31a32997f065e1d1d7b115f2bb1c92c8baedb141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-10T10:50:47.799 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:47 smithi195 bash[69580]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8 2024-05-10T10:50:48.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:47.650+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:48.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:47.650+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:48.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:47.650+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:48.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:47.650+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:48.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:47.650+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:48.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:47 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:47.650+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:48.065 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:47 smithi195 podman[69597]: 2024-05-10 10:50:47.798412071 +0000 UTC m=+0.537036177 container remove e50ad85efcca68f6b5426dbb31a32997f065e1d1d7b115f2bb1c92c8baedb141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:50:48.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:47.865+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:48.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:47.865+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:48.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:47.865+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:48.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:47.865+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:48.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:47.865+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:48.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:47 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:47.865+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:48.477 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:48.379+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:48.477 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:48.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:48.477 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:48.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:48.477 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:48.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:48.477 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:48.380+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:48.477 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:48 smithi195 podman[69693]: 2024-05-10 10:50:48.190969858 +0000 UTC m=+0.106631816 container create 26d6227c975110e474c7674f4c85559b2c8017c42e133f26f59d25e1a4ce5c08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:50:48.478 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:48 smithi195 podman[69693]: 2024-05-10 10:50:48.102717312 +0000 UTC m=+0.018379271 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:48.478 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:48 smithi195 podman[69693]: 2024-05-10 10:50:48.418815477 +0000 UTC m=+0.334477444 container init 26d6227c975110e474c7674f4c85559b2c8017c42e133f26f59d25e1a4ce5c08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:50:48.478 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:48 smithi195 podman[69693]: 2024-05-10 10:50:48.421991774 +0000 UTC m=+0.337653742 container start 26d6227c975110e474c7674f4c85559b2c8017c42e133f26f59d25e1a4ce5c08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS) 2024-05-10T10:50:48.478 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:48 smithi195 podman[69693]: 2024-05-10 10:50:48.476879775 +0000 UTC m=+0.392541748 container attach 26d6227c975110e474c7674f4c85559b2c8017c42e133f26f59d25e1a4ce5c08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True) 2024-05-10T10:50:48.816 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 10 10:50:48 smithi195 podman[69693]: 2024-05-10 10:50:48.721132023 +0000 UTC m=+0.636793980 container died 26d6227c975110e474c7674f4c85559b2c8017c42e133f26f59d25e1a4ce5c08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-05-10T10:50:49.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:48.700+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:49.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:48.700+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:49.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:48.700+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:49.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:48.700+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:49.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:48.700+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:49.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:48 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:48.700+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:49.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:48.910+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:49.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:48.910+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:49.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:48.910+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:49.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:48.910+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:49.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:48.910+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:49.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:48 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:48.910+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:49.332 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.8.service' 2024-05-10T10:50:49.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:49.413 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-10T10:50:49.413 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-10T10:50:49.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.9 2024-05-10T10:50:49.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:49 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:49.357+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:49.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:49 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:49.357+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:49.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:49 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:49.357+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:49.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:49 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:49.357+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:49.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:49 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:49.357+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:49.735 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 systemd[1]: Stopping Ceph osd.9 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:49.735 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.734+0000 7f5217ead700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:49.736 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.734+0000 7f5217ead700 -1 osd.9 94 *** Got signal Terminated *** 2024-05-10T10:50:50.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:49.652+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:50.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:49.652+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:50.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:49.652+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:50.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:49.652+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:50.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:49.652+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:50.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:49 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:49.652+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:50.079 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.734+0000 7f5217ead700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:50.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.864+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:50.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.864+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:50.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.864+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:50.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.864+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:50.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.864+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:50.080 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:49 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:49.864+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:50.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:50 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:50.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:50.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:50 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:50.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:50.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:50 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:50.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:50.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:50 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:50.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:50.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:50 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:50.326+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:51.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:50.651+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:51.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:50.651+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:51.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:50.651+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:51.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:50.651+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:51.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:50.651+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:51.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:50 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:50.651+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:51.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:50.906+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:51.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:50.906+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:51.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:50.906+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:51.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:50.906+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:51.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:50.906+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:51.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:50 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:50.906+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:51.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:51 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:51.325+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:51.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:51 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:51.325+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:51.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:51 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:51.325+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:51.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:51 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:51.325+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:51.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:51 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:51.325+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:51.946 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:51 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:51.628+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:51.946 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:51 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:51.628+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:51.946 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:51 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:51.628+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:51.946 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:51 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:51.628+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:51.946 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:51 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:51.628+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:51.946 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:51 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:51.628+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:52.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:51 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:51.892+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:52.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:51 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:51.892+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:52.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:51 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:51.893+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:52.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:51 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:51.893+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:52.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:51 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:51.893+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:52.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:51 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:51.893+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:52.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:52 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:52.355+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:52.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:52 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:52.355+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:52.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:52 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:52.355+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:52.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:52 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:52.355+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:52.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:52 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:52.355+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:53.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:52 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:52.647+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:53.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:52 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:52.647+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:53.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:52 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:52.647+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:53.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:52 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:52.647+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:53.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:52 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:52.647+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:53.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:52 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:52.647+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:53.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:52 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:52.858+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:53.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:52 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:52.858+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:53.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:52 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:52.858+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:53.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:52 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:52.858+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:53.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:52 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:52.858+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:53.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:52 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:52.858+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:53.717 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:53.329+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:53.717 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:53.329+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:53.717 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:53.329+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:53.718 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:53.329+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:53.718 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:53 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:53.329+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:54.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:53 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:53.605+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:54.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:53 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:53.605+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:54.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:53 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:53.605+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:54.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:53 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:53.605+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:54.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:53 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:53.605+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:54.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:53 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:53.605+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:54.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:53 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:53.844+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:40.286433+0000 front 2024-05-10T10:49:40.286460+0000 (oldest deadline 2024-05-10T10:50:03.186564+0000) 2024-05-10T10:50:54.329 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:53 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:53.844+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:50.188291+0000 front 2024-05-10T10:49:50.188339+0000 (oldest deadline 2024-05-10T10:50:12.488375+0000) 2024-05-10T10:50:54.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:53 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:53.844+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.990763+0000 front 2024-05-10T10:49:56.990810+0000 (oldest deadline 2024-05-10T10:50:21.591619+0000) 2024-05-10T10:50:54.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:53 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:53.844+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:01.592182+0000 front 2024-05-10T10:50:01.592162+0000 (oldest deadline 2024-05-10T10:50:26.892163+0000) 2024-05-10T10:50:54.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:53 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:53.844+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:10.894025+0000 front 2024-05-10T10:50:10.893936+0000 (oldest deadline 2024-05-10T10:50:36.193776+0000) 2024-05-10T10:50:54.330 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:53 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9[62602]: 2024-05-10T10:50:53.844+0000 7f52144a1700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.694981+0000 front 2024-05-10T10:50:19.694943+0000 (oldest deadline 2024-05-10T10:50:44.994925+0000) 2024-05-10T10:50:54.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:54 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:54.285+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:54.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:54 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:54.285+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:54.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:54 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:54.285+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:54.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:54 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:54.285+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:54.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:54 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:54.285+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:55.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:54 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:54.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:55.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:54 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:54.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:55.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:54 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:54.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:55.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:54 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:54.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:55.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:54 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:54.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:55.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:54 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:54.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:55.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:54 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:54.634+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:50:55.502 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:55 smithi112 podman[128036]: 2024-05-10 10:50:55.243246295 +0000 UTC m=+5.641358311 container died ae78d0dde9d2c61427510068c07d96bcbcc287187fc874cc4ff5aee6d4d9ce39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-10T10:50:55.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:55.245+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:55.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:55.245+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:55.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:55.245+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:55.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:55.245+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:55.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:55.245+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:55.829 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:55 smithi112 podman[128036]: 2024-05-10 10:50:55.667485951 +0000 UTC m=+6.065598008 container cleanup ae78d0dde9d2c61427510068c07d96bcbcc287187fc874cc4ff5aee6d4d9ce39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:50:55.829 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:55 smithi112 bash[128036]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9 2024-05-10T10:50:56.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:55 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:55.645+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:56.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:55 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:55.645+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:56.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:55 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:55.645+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:56.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:55 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:55.645+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:56.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:55 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:55.645+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:56.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:55 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:55.645+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:56.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:55 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:55.645+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:50:56.289 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:55 smithi112 podman[128052]: 2024-05-10 10:50:55.862265076 +0000 UTC m=+0.616387836 container remove ae78d0dde9d2c61427510068c07d96bcbcc287187fc874cc4ff5aee6d4d9ce39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-10T10:50:56.542 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:56 smithi112 podman[128141]: 2024-05-10 10:50:56.191946534 +0000 UTC m=+0.018210425 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:50:56.542 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:56 smithi112 podman[128141]: 2024-05-10 10:50:56.31141746 +0000 UTC m=+0.137681339 container create 46924abfee69d8f5e7d90fd8c1cca4f4418c9deea8cc91d68a33d62a04814b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-10T10:50:56.542 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:56 smithi112 podman[128141]: 2024-05-10 10:50:56.538529974 +0000 UTC m=+0.364793859 container init 46924abfee69d8f5e7d90fd8c1cca4f4418c9deea8cc91d68a33d62a04814b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-10T10:50:56.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:56 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:56.249+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:56.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:56 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:56.249+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:56.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:56 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:56.249+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:56.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:56 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:56.249+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:56.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:56 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:56.249+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:56.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:56 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:56.249+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:50:56.829 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:56 smithi112 podman[128141]: 2024-05-10 10:50:56.541846479 +0000 UTC m=+0.368110362 container start 46924abfee69d8f5e7d90fd8c1cca4f4418c9deea8cc91d68a33d62a04814b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, ceph=True) 2024-05-10T10:50:56.829 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:56 smithi112 podman[128141]: 2024-05-10 10:50:56.630286355 +0000 UTC m=+0.456550248 container attach 46924abfee69d8f5e7d90fd8c1cca4f4418c9deea8cc91d68a33d62a04814b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-10T10:50:57.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:56 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:56.632+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:57.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:56 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:56.632+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:57.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:56 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:56.632+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:57.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:56 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:56.632+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:57.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:56 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:56.632+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:57.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:56 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:56.632+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:57.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:56 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:56.632+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:50:57.124 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 10 10:50:56 smithi112 podman[128141]: 2024-05-10 10:50:56.8358075 +0000 UTC m=+0.662071383 container died 46924abfee69d8f5e7d90fd8c1cca4f4418c9deea8cc91d68a33d62a04814b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5) 2024-05-10T10:50:57.423 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.9.service' 2024-05-10T10:50:57.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:50:57.463 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-10T10:50:57.463 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-10T10:50:57.463 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.10 2024-05-10T10:50:57.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:57 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:57.244+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:57.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:57 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:57.244+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:57.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:57 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:57.244+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:57.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:57 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:57.244+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:57.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:57 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:57.244+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:57.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:57 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:57.244+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 systemd[1]: Stopping Ceph osd.10 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.665+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.665+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.665+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.665+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.665+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.665+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.665+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.778+0000 7fcaf5e66700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.778+0000 7fcaf5e66700 -1 osd.10 94 *** Got signal Terminated *** 2024-05-10T10:50:57.781 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:57 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:57.778+0000 7fcaf5e66700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:50:58.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:58 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:58.221+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:58.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:58 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:58.221+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:58.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:58 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:58.221+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:58.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:58 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:58.221+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:58.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:58 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:58.221+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:58.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:58 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:58.221+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:50:59.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:58 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:58.653+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:50:59.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:58 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:58.653+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:50:59.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:58 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:58.653+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:50:59.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:58 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:58.653+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:50:59.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:58 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:58.653+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:50:59.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:58 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:58.653+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:50:59.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:58 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:58.653+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:50:59.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:59.268+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:50:59.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:59.268+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:50:59.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:59.268+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:50:59.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:59.268+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:50:59.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:59.268+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:50:59.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:50:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:50:59.268+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:00.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:59.676+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:51:00.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:59.676+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:51:00.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:59.676+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:51:00.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:59.676+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:51:00.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:59.676+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:51:00.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:59.676+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:51:00.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:50:59 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:50:59.676+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:51:00.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:00 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:00.270+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:00.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:00 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:00.270+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:00.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:00 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:00.270+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:00.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:00 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:00.270+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:00.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:00 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:00.270+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:00.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:00 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:00.270+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:01.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:00.685+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:51:01.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:00.685+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:51:01.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:00.685+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:51:01.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:00.685+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:51:01.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:00.685+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:51:01.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:00.685+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:51:01.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:00 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:00.685+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:51:01.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:01 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:01.241+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:01.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:01 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:01.241+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:01.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:01 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:01.241+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:01.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:01 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:01.241+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:01.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:01 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:01.241+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:01.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:01 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:01.241+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:02.008 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:01 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:01.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:51:02.008 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:01 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:01.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:51:02.009 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:01 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:01.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:51:02.009 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:01 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:01.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:51:02.009 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:01 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:01.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:51:02.009 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:01 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:01.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:51:02.009 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:01 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:01.706+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:51:02.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:02 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:02.193+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:02.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:02 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:02.193+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:02.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:02 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:02.193+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:02.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:02 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:02.193+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:02.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:02 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:02.193+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:02.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:02 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:02.193+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:02.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:02 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:02.193+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:03.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:02 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:02.752+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.328381+0000 front 2024-05-10T10:49:39.328460+0000 (oldest deadline 2024-05-10T10:50:04.628531+0000) 2024-05-10T10:51:03.044 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:02 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:02.752+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-05-10T10:49:49.929604+0000 front 2024-05-10T10:49:49.929643+0000 (oldest deadline 2024-05-10T10:50:14.629753+0000) 2024-05-10T10:51:03.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:02 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:02.752+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:56.930944+0000 front 2024-05-10T10:49:56.931060+0000 (oldest deadline 2024-05-10T10:50:22.830943+0000) 2024-05-10T10:51:03.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:02 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:02.752+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:03.932042+0000 front 2024-05-10T10:50:03.931974+0000 (oldest deadline 2024-05-10T10:50:27.431957+0000) 2024-05-10T10:51:03.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:02 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:02.752+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:11.433416+0000 front 2024-05-10T10:50:11.433448+0000 (oldest deadline 2024-05-10T10:50:36.133478+0000) 2024-05-10T10:51:03.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:02 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:02.752+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:19.034741+0000 front 2024-05-10T10:50:19.034756+0000 (oldest deadline 2024-05-10T10:50:43.734564+0000) 2024-05-10T10:51:03.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:02 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10[58420]: 2024-05-10T10:51:02.752+0000 7fcaf245a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:27.736419+0000 front 2024-05-10T10:50:27.736542+0000 (oldest deadline 2024-05-10T10:50:53.636187+0000) 2024-05-10T10:51:03.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:03 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:03.164+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:03.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:03 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:03.164+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:03.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:03 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:03.164+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:03.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:03 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:03.164+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:03.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:03 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:03.164+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:03.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:03 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:03.164+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:03.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:03 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:03.164+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:03.647 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:03 smithi148 podman[69227]: 2024-05-10 10:51:03.378720028 +0000 UTC m=+5.716829573 container died 97ccf899a2e5232081140a99ba40d86543ae2eddd227f95e670dc1905c8d7423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:51:04.006 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:03 smithi148 podman[69227]: 2024-05-10 10:51:03.745951316 +0000 UTC m=+6.084060896 container cleanup 97ccf899a2e5232081140a99ba40d86543ae2eddd227f95e670dc1905c8d7423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:51:04.006 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:03 smithi148 bash[69227]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10 2024-05-10T10:51:04.294 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:04 smithi148 podman[69244]: 2024-05-10 10:51:04.00566678 +0000 UTC m=+0.629503325 container remove 97ccf899a2e5232081140a99ba40d86543ae2eddd227f95e670dc1905c8d7423 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:51:04.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:04 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:04.192+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:04.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:04 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:04.192+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:04.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:04 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:04.192+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:04.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:04 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:04.192+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:04.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:04 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:04.192+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:04.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:04 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:04.192+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:04.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:04 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:04.192+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:04.618 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:04 smithi148 podman[69332]: 2024-05-10 10:51:04.213655613 +0000 UTC m=+0.018285531 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:51:04.618 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:04 smithi148 podman[69332]: 2024-05-10 10:51:04.315001533 +0000 UTC m=+0.119631441 container create 65d0e3268e9b4effa84294f8dc6a4f6515cc5efb6ed773eed478912d65d8e59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T10:51:04.618 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:04 smithi148 podman[69332]: 2024-05-10 10:51:04.533940313 +0000 UTC m=+0.338570221 container init 65d0e3268e9b4effa84294f8dc6a4f6515cc5efb6ed773eed478912d65d8e59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-10T10:51:04.618 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:04 smithi148 podman[69332]: 2024-05-10 10:51:04.537132235 +0000 UTC m=+0.341762147 container start 65d0e3268e9b4effa84294f8dc6a4f6515cc5efb6ed773eed478912d65d8e59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:51:04.618 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:04 smithi148 podman[69332]: 2024-05-10 10:51:04.617033239 +0000 UTC m=+0.421663145 container attach 65d0e3268e9b4effa84294f8dc6a4f6515cc5efb6ed773eed478912d65d8e59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:51:05.045 INFO:journalctl@ceph.osd.10.smithi148.stdout:May 10 10:51:04 smithi148 podman[69332]: 2024-05-10 10:51:04.832830989 +0000 UTC m=+0.637460897 container died 65d0e3268e9b4effa84294f8dc6a4f6515cc5efb6ed773eed478912d65d8e59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-10T10:51:05.455 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.10.service' 2024-05-10T10:51:05.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:05.495 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-10T10:51:05.496 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-10T10:51:05.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.11 2024-05-10T10:51:05.522 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:05.148+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:05.523 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:05.148+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:05.523 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:05.148+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:05.523 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:05.148+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:05.523 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:05.148+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:05.523 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:05.148+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:05.523 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:05.148+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:05.816 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:05 smithi195 systemd[1]: Stopping Ceph osd.11 for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:06.316 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.032+0000 7f849010b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:06.316 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.032+0000 7f849010b700 -1 osd.11 94 *** Got signal Terminated *** 2024-05-10T10:51:06.316 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.032+0000 7f849010b700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T10:51:06.316 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.113+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:06.316 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.113+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:06.316 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.113+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:06.316 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.113+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:06.317 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.113+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:06.317 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.113+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:06.317 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:06 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:06.113+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:07.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:07.085+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:07.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:07.085+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:07.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:07.085+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:07.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:07.085+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:07.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:07.085+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:07.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:07.085+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:07.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:07 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:07.085+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:08.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:08.118+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:08.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:08.118+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:08.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:08.118+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:08.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:08.118+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:08.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:08.118+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:08.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:08.118+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:08.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:08 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:08.118+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:09.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:09.146+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:09.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:09.146+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:09.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:09.146+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:09.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:09.146+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:09.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:09.146+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:09.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:09.146+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:09.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:09 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:09.146+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:10.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-10T10:49:39.313937+0000 front 2024-05-10T10:49:39.314081+0000 (oldest deadline 2024-05-10T10:50:05.214091+0000) 2024-05-10T10:51:10.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-10T10:49:53.817018+0000 front 2024-05-10T10:49:53.817039+0000 (oldest deadline 2024-05-10T10:50:19.117304+0000) 2024-05-10T10:51:10.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-10T10:50:05.919894+0000 front 2024-05-10T10:50:05.919919+0000 (oldest deadline 2024-05-10T10:50:27.619831+0000) 2024-05-10T10:51:10.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-05-10T10:50:12.722021+0000 front 2024-05-10T10:50:12.722005+0000 (oldest deadline 2024-05-10T10:50:35.021932+0000) 2024-05-10T10:51:10.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-05-10T10:50:22.325258+0000 front 2024-05-10T10:50:22.325285+0000 (oldest deadline 2024-05-10T10:50:47.025362+0000) 2024-05-10T10:51:10.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-10T10:50:30.526334+0000 front 2024-05-10T10:50:30.526353+0000 (oldest deadline 2024-05-10T10:50:55.826156+0000) 2024-05-10T10:51:10.566 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-05-10T10:50:38.127028+0000 front 2024-05-10T10:50:38.127121+0000 (oldest deadline 2024-05-10T10:51:01.626913+0000) 2024-05-10T10:51:10.567 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:10 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11[58939]: 2024-05-10T10:51:10.097+0000 7f848c6ff700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-05-10T10:50:44.527916+0000 front 2024-05-10T10:50:44.527997+0000 (oldest deadline 2024-05-10T10:51:09.828000+0000) 2024-05-10T10:51:11.800 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:11 smithi195 podman[69810]: 2024-05-10 10:51:11.533934263 +0000 UTC m=+5.842326951 container died de7cf6e855b9b49c6f3996e852864df159221d75751781f9cd21864ebb312ff9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:51:12.067 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:11 smithi195 podman[69810]: 2024-05-10 10:51:11.898949732 +0000 UTC m=+6.207342461 container cleanup de7cf6e855b9b49c6f3996e852864df159221d75751781f9cd21864ebb312ff9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD) 2024-05-10T10:51:12.067 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:11 smithi195 bash[69810]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11 2024-05-10T10:51:12.518 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:12 smithi195 podman[69830]: 2024-05-10 10:51:12.166904511 +0000 UTC m=+0.625219297 container remove de7cf6e855b9b49c6f3996e852864df159221d75751781f9cd21864ebb312ff9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-10T10:51:12.803 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:12 smithi195 podman[69923]: 2024-05-10 10:51:12.421411969 +0000 UTC m=+0.018329324 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T10:51:12.803 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:12 smithi195 podman[69923]: 2024-05-10 10:51:12.537416993 +0000 UTC m=+0.134334336 container create 9383e9873e324b66e36b4017bbce565ae5c9a71169906f7b7336e3e8937b5c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-10T10:51:12.804 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:12 smithi195 podman[69923]: 2024-05-10 10:51:12.737409452 +0000 UTC m=+0.334326801 container init 9383e9873e324b66e36b4017bbce565ae5c9a71169906f7b7336e3e8937b5c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-10T10:51:12.804 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:12 smithi195 podman[69923]: 2024-05-10 10:51:12.741189076 +0000 UTC m=+0.338106418 container start 9383e9873e324b66e36b4017bbce565ae5c9a71169906f7b7336e3e8937b5c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-10T10:51:12.804 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:12 smithi195 podman[69923]: 2024-05-10 10:51:12.803217111 +0000 UTC m=+0.400134472 container attach 9383e9873e324b66e36b4017bbce565ae5c9a71169906f7b7336e3e8937b5c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-10T10:51:13.066 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:13 smithi195 podman[69923]: 2024-05-10 10:51:13.034305626 +0000 UTC m=+0.631222969 container died 9383e9873e324b66e36b4017bbce565ae5c9a71169906f7b7336e3e8937b5c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-10T10:51:13.579 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 10 10:51:13 smithi195 podman[69949]: 2024-05-10 10:51:13.564717857 +0000 UTC m=+0.524588836 container remove 9383e9873e324b66e36b4017bbce565ae5c9a71169906f7b7336e3e8937b5c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:51:13.585 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@osd.11.service' 2024-05-10T10:51:13.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:13.665 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-10T10:51:13.665 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-10T10:51:13.665 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.a 2024-05-10T10:51:13.960 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:51:13 smithi112 systemd[1]: Stopping Ceph mds.a for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:14.225 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:51:13 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a[69621]: 2024-05-10T10:51:13.959+0000 7f7da8ae7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:14.226 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:51:13 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a[69621]: 2024-05-10T10:51:13.959+0000 7f7da8ae7700 -1 mds.a *** got signal Terminated *** 2024-05-10T10:51:18.285 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:51:17 smithi112 podman[128246]: 2024-05-10 10:51:17.968823023 +0000 UTC m=+4.109750398 container died a70829d91c71fe1b1873b616d5e4fb2872f87f1b08a36c988a2fb2afe1309305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-10T10:51:18.579 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:51:18 smithi112 podman[128246]: 2024-05-10 10:51:18.434563049 +0000 UTC m=+4.575490423 container cleanup a70829d91c71fe1b1873b616d5e4fb2872f87f1b08a36c988a2fb2afe1309305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507) 2024-05-10T10:51:18.579 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 10 10:51:18 smithi112 bash[128246]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-a 2024-05-10T10:51:18.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.a.service' 2024-05-10T10:51:18.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:18.828 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-10T10:51:18.829 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-10T10:51:18.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.d 2024-05-10T10:51:19.079 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:18 smithi112 systemd[1]: Stopping Ceph mds.d for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:19.579 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:19 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d[68346]: 2024-05-10T10:51:19.189+0000 7ff494442700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:19.579 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:19 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d[68346]: 2024-05-10T10:51:19.189+0000 7ff494442700 -1 mds.d *** got signal Terminated *** 2024-05-10T10:51:22.480 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:22 smithi112 podman[128381]: 2024-05-10 10:51:22.111237523 +0000 UTC m=+3.068987557 container died 1f93a651e3f46cef0712f063730ffa51c946cfea4b9b5700144eac59cf1ee7e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-10T10:51:22.730 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:22 smithi112 podman[128381]: 2024-05-10 10:51:22.554904327 +0000 UTC m=+3.512654360 container cleanup 1f93a651e3f46cef0712f063730ffa51c946cfea4b9b5700144eac59cf1ee7e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5) 2024-05-10T10:51:22.731 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:22 smithi112 bash[128381]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d 2024-05-10T10:51:22.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.d.service' 2024-05-10T10:51:22.998 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:22 smithi112 podman[128395]: 2024-05-10 10:51:22.730854825 +0000 UTC m=+0.612730611 container remove 1f93a651e3f46cef0712f063730ffa51c946cfea4b9b5700144eac59cf1ee7e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD) 2024-05-10T10:51:22.998 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:22 smithi112 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.d.service: Deactivated successfully. 2024-05-10T10:51:22.998 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:22 smithi112 systemd[1]: Stopped Ceph mds.d for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:51:22.998 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 10 10:51:22 smithi112 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.d.service: Consumed 1.044s CPU time. 2024-05-10T10:51:23.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:23.010 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-10T10:51:23.010 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-10T10:51:23.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.g 2024-05-10T10:51:23.329 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:23 smithi112 systemd[1]: Stopping Ceph mds.g for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:23.829 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:23 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g[69187]: 2024-05-10T10:51:23.333+0000 7f7da38cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:23.829 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:23 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g[69187]: 2024-05-10T10:51:23.333+0000 7f7da38cc700 -1 mds.g *** got signal Terminated *** 2024-05-10T10:51:27.234 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:26 smithi112 podman[128502]: 2024-05-10 10:51:26.862840909 +0000 UTC m=+3.635916268 container died 325f4ad1801784c0ceb255319f1b91da02175c89a286913220b220115495c155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-10T10:51:27.509 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:27 smithi112 podman[128502]: 2024-05-10 10:51:27.316757431 +0000 UTC m=+4.089832790 container cleanup 325f4ad1801784c0ceb255319f1b91da02175c89a286913220b220115495c155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-10T10:51:27.509 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:27 smithi112 bash[128502]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g 2024-05-10T10:51:27.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.g.service' 2024-05-10T10:51:27.760 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:27 smithi112 podman[128516]: 2024-05-10 10:51:27.509516084 +0000 UTC m=+0.636508660 container remove 325f4ad1801784c0ceb255319f1b91da02175c89a286913220b220115495c155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-10T10:51:27.761 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:27 smithi112 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-10T10:51:27.761 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:27 smithi112 systemd[1]: Stopped Ceph mds.g for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:51:27.761 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 10 10:51:27 smithi112 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.g.service: Consumed 1.960s CPU time. 2024-05-10T10:51:27.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:27.772 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-10T10:51:27.773 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-10T10:51:27.773 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.j 2024-05-10T10:51:28.079 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:51:27 smithi112 systemd[1]: Stopping Ceph mds.j for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:28.579 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:51:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j[67889]: 2024-05-10T10:51:28.085+0000 7f4e87758700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:28.579 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:51:28 smithi112 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j[67889]: 2024-05-10T10:51:28.085+0000 7f4e87758700 -1 mds.j *** got signal Terminated *** 2024-05-10T10:51:29.487 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:51:29 smithi112 podman[128621]: 2024-05-10 10:51:29.21222813 +0000 UTC m=+1.226072498 container died 40dc4b06be95cb7d5bf266f4f1a4feff12caedff47889332a94595fe9b9a20e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-05-10T10:51:29.829 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:51:29 smithi112 podman[128621]: 2024-05-10 10:51:29.561716931 +0000 UTC m=+1.575561307 container cleanup 40dc4b06be95cb7d5bf266f4f1a4feff12caedff47889332a94595fe9b9a20e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-10T10:51:29.829 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 10 10:51:29 smithi112 bash[128621]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-j 2024-05-10T10:51:29.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.j.service' 2024-05-10T10:51:30.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:30.016 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-10T10:51:30.017 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-10T10:51:30.017 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.b 2024-05-10T10:51:30.338 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:51:30 smithi148 systemd[1]: Stopping Ceph mds.b for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:30.794 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:51:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b[63901]: 2024-05-10T10:51:30.337+0000 7f3ffd973700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:30.794 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:51:30 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b[63901]: 2024-05-10T10:51:30.337+0000 7f3ffd973700 -1 mds.b *** got signal Terminated *** 2024-05-10T10:51:34.696 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:51:34 smithi148 podman[69438]: 2024-05-10 10:51:34.345763937 +0000 UTC m=+4.122398793 container died 564e4250fa82a6058aa42dae2dbb1e4c3ec5deda0db7e93781338b191d10a7f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-10T10:51:35.030 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:51:34 smithi148 podman[69438]: 2024-05-10 10:51:34.778632072 +0000 UTC m=+4.555266911 container cleanup 564e4250fa82a6058aa42dae2dbb1e4c3ec5deda0db7e93781338b191d10a7f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-10T10:51:35.030 INFO:journalctl@ceph.mds.b.smithi148.stdout:May 10 10:51:34 smithi148 bash[69438]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-b 2024-05-10T10:51:35.161 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.b.service' 2024-05-10T10:51:35.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:35.197 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-10T10:51:35.197 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-10T10:51:35.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.e 2024-05-10T10:51:35.294 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:51:35 smithi148 systemd[1]: Stopping Ceph mds.e for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:35.794 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:51:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e[63264]: 2024-05-10T10:51:35.532+0000 7f980d021700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:35.794 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:51:35 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e[63264]: 2024-05-10T10:51:35.532+0000 7f980d021700 -1 mds.e *** got signal Terminated *** 2024-05-10T10:51:39.142 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:51:38 smithi148 podman[69568]: 2024-05-10 10:51:38.86416035 +0000 UTC m=+3.450911384 container died 567a75ddddc06503480089e31fe45e60676a939cd138bdcfa7a2c1aa8cddf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-10T10:51:39.492 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:51:39 smithi148 podman[69568]: 2024-05-10 10:51:39.24047211 +0000 UTC m=+3.827223141 container cleanup 567a75ddddc06503480089e31fe45e60676a939cd138bdcfa7a2c1aa8cddf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:51:39.492 INFO:journalctl@ceph.mds.e.smithi148.stdout:May 10 10:51:39 smithi148 bash[69568]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-e 2024-05-10T10:51:39.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.e.service' 2024-05-10T10:51:39.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:39.645 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-10T10:51:39.645 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-10T10:51:39.645 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.h 2024-05-10T10:51:39.794 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:51:39 smithi148 systemd[1]: Stopping Ceph mds.h for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:40.294 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:51:39 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h[63588]: 2024-05-10T10:51:39.985+0000 7f738e744700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:40.294 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:51:39 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h[63588]: 2024-05-10T10:51:39.986+0000 7f738e744700 -1 mds.h *** got signal Terminated *** 2024-05-10T10:51:43.246 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:51:42 smithi148 podman[69689]: 2024-05-10 10:51:42.865155499 +0000 UTC m=+2.992994883 container died 3c8d2f32cc7ecdf5be9b6973774c93d062b968164798062426df33b278e923e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-10T10:51:43.544 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:51:43 smithi148 podman[69689]: 2024-05-10 10:51:43.327559783 +0000 UTC m=+3.455399166 container cleanup 3c8d2f32cc7ecdf5be9b6973774c93d062b968164798062426df33b278e923e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-10T10:51:43.544 INFO:journalctl@ceph.mds.h.smithi148.stdout:May 10 10:51:43 smithi148 bash[69689]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-h 2024-05-10T10:51:43.751 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.h.service' 2024-05-10T10:51:43.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:43.785 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-10T10:51:43.785 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-10T10:51:43.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.k 2024-05-10T10:51:44.114 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:51:43 smithi148 systemd[1]: Stopping Ceph mds.k for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:44.544 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:51:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k[62942]: 2024-05-10T10:51:44.113+0000 7fea1b8f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:44.544 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:51:44 smithi148 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k[62942]: 2024-05-10T10:51:44.113+0000 7fea1b8f8700 -1 mds.k *** got signal Terminated *** 2024-05-10T10:51:47.148 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:51:46 smithi148 podman[69811]: 2024-05-10 10:51:46.874415732 +0000 UTC m=+2.872349650 container died 3e3306709d8f20aed6ec03e13af570d01b11b962cf3db8de1d5b6b417beaca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-10T10:51:47.482 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:51:47 smithi148 podman[69811]: 2024-05-10 10:51:47.247626094 +0000 UTC m=+3.245560021 container cleanup 3e3306709d8f20aed6ec03e13af570d01b11b962cf3db8de1d5b6b417beaca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507) 2024-05-10T10:51:47.482 INFO:journalctl@ceph.mds.k.smithi148.stdout:May 10 10:51:47 smithi148 bash[69811]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-k 2024-05-10T10:51:47.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.k.service' 2024-05-10T10:51:47.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:47.632 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-10T10:51:47.632 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-10T10:51:47.632 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.c 2024-05-10T10:51:48.057 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:51:47 smithi195 systemd[1]: Stopping Ceph mds.c for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:48.316 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:51:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: 2024-05-10T10:51:48.055+0000 7f4b01f8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:48.316 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:51:48 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c[63811]: 2024-05-10T10:51:48.055+0000 7f4b01f8c700 -1 mds.c *** got signal Terminated *** 2024-05-10T10:51:51.231 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:51:50 smithi195 podman[70028]: 2024-05-10 10:51:50.864321373 +0000 UTC m=+3.043326499 container died edd25aea5ac33d12594dfde00b260f41f348e063fffa0484dfb8b1d2641471b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T10:51:51.566 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:51:51 smithi195 podman[70028]: 2024-05-10 10:51:51.322183253 +0000 UTC m=+3.501188359 container cleanup edd25aea5ac33d12594dfde00b260f41f348e063fffa0484dfb8b1d2641471b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:51:51.566 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 10 10:51:51 smithi195 bash[70028]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-c 2024-05-10T10:51:51.708 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.c.service' 2024-05-10T10:51:51.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:51.745 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-10T10:51:51.745 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-10T10:51:51.745 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.f 2024-05-10T10:51:52.065 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:51:51 smithi195 systemd[1]: Stopping Ceph mds.f for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:52.565 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:51:52 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f[63500]: 2024-05-10T10:51:52.067+0000 7fa46190d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:52.566 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:51:52 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f[63500]: 2024-05-10T10:51:52.067+0000 7fa46190d700 -1 mds.f *** got signal Terminated *** 2024-05-10T10:51:55.135 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:51:54 smithi195 podman[70160]: 2024-05-10 10:51:54.873720805 +0000 UTC m=+2.912758795 container died 3f98fe838b65941f25f7580a5aec9693e60b8ba9bc5ca46e53a7977de4ff3c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-10T10:51:55.393 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:51:55 smithi195 podman[70160]: 2024-05-10 10:51:55.225560587 +0000 UTC m=+3.264598573 container cleanup 3f98fe838b65941f25f7580a5aec9693e60b8ba9bc5ca46e53a7977de4ff3c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True) 2024-05-10T10:51:55.393 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 10 10:51:55 smithi195 bash[70160]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-f 2024-05-10T10:51:55.603 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.f.service' 2024-05-10T10:51:55.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:55.637 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-10T10:51:55.637 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-10T10:51:55.637 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.i 2024-05-10T10:51:55.955 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:51:55 smithi195 systemd[1]: Stopping Ceph mds.i for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:51:56.315 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:51:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: 2024-05-10T10:51:55.953+0000 7fcee2708700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:51:56.316 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:51:55 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i[64119]: 2024-05-10T10:51:55.953+0000 7fcee2708700 -1 mds.i *** got signal Terminated *** 2024-05-10T10:51:59.065 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:51:58 smithi195 podman[70281]: 2024-05-10 10:51:58.869071071 +0000 UTC m=+3.007830395 container died 8ede9de1eb86562c52fa3971e52e3ef5dde7024cfa7a23689b2c845277845cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T10:51:59.413 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:51:59 smithi195 podman[70281]: 2024-05-10 10:51:59.204170587 +0000 UTC m=+3.342929920 container cleanup 8ede9de1eb86562c52fa3971e52e3ef5dde7024cfa7a23689b2c845277845cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T10:51:59.414 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 10 10:51:59 smithi195 bash[70281]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-i 2024-05-10T10:51:59.621 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.i.service' 2024-05-10T10:51:59.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:51:59.653 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-10T10:51:59.653 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-10T10:51:59.653 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.l 2024-05-10T10:51:59.975 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:51:59 smithi195 systemd[1]: Stopping Ceph mds.l for ccd347ac-0eb7-11ef-bc98-c7b262605968... 2024-05-10T10:52:00.316 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:51:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l[63179]: 2024-05-10T10:51:59.973+0000 7fc7c01e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T10:52:00.316 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:51:59 smithi195 ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l[63179]: 2024-05-10T10:51:59.973+0000 7fc7c01e6700 -1 mds.l *** got signal Terminated *** 2024-05-10T10:52:03.133 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:52:02 smithi195 podman[70403]: 2024-05-10 10:52:02.872681958 +0000 UTC m=+3.005623935 container died 61f4e1feb90372b4fc1536ac1388085fd41e979b1241fb9c27e8069b05ea6e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T10:52:03.408 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:52:03 smithi195 podman[70403]: 2024-05-10 10:52:03.215968763 +0000 UTC m=+3.348910749 container cleanup 61f4e1feb90372b4fc1536ac1388085fd41e979b1241fb9c27e8069b05ea6e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-10T10:52:03.409 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:52:03 smithi195 bash[70403]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l 2024-05-10T10:52:03.651 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.l.service' 2024-05-10T10:52:03.672 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:52:03 smithi195 podman[70418]: 2024-05-10 10:52:03.4086573 +0000 UTC m=+0.529806757 container remove 61f4e1feb90372b4fc1536ac1388085fd41e979b1241fb9c27e8069b05ea6e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-10T10:52:03.672 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:52:03 smithi195 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.l.service: Deactivated successfully. 2024-05-10T10:52:03.672 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:52:03 smithi195 systemd[1]: Stopped Ceph mds.l for ccd347ac-0eb7-11ef-bc98-c7b262605968. 2024-05-10T10:52:03.672 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 10 10:52:03 smithi195 systemd[1]: ceph-ccd347ac-0eb7-11ef-bc98-c7b262605968@mds.l.service: Consumed 7.833s CPU time. 2024-05-10T10:52:03.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T10:52:03.682 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-10T10:52:03.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T10:52:03.875 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:52:06.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T10:52:06.549 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:52:09.048 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T10:52:09.238 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:52:11.609 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:52:11.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:52:11.676 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T10:52:11.703 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-10T10:52:11.704 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi112/crash 2024-05-10T10:52:11.705 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash -- . 2024-05-10T10:52:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T10:52:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-05-10T10:52:11.752 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi148/crash 2024-05-10T10:52:11.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash -- . 2024-05-10T10:52:11.793 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T10:52:11.793 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-05-10T10:52:11.795 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi195/crash 2024-05-10T10:52:11.796 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash -- . 2024-05-10T10:52:11.832 INFO:teuthology.orchestra.run.smithi195.stderr:tar: /var/lib/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T10:52:11.832 INFO:teuthology.orchestra.run.smithi195.stderr:tar: Error is not recoverable: exiting now 2024-05-10T10:52:11.833 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-10T10:52:11.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T10:52:11.869 INFO:teuthology.orchestra.run.smithi112.stdout:2024-05-10T10:43:55.975814+0000 mds.c (mds.1) 5 : cluster [WRN] Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:52:11.870 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-10T10:52:11.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[SEC\]' /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T10:52:11.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]' /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T10:52:12.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[WRN\]' /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T10:52:12.088 INFO:teuthology.orchestra.run.smithi112.stdout:2024-05-10T10:43:55.975814+0000 mds.c (mds.1) 5 : cluster [WRN] Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details 2024-05-10T10:52:12.088 INFO:tasks.cephadm:Compressing logs... 2024-05-10T10:52:12.088 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T10:52:12.130 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T10:52:12.132 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T10:52:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-10T10:52:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:: No such file or directory 2024-05-10T10:52:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T10:52:12.161 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.log: /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log: /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.0.log: 86.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.a.log 90.3% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-10T10:52:12.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.a.log: 94.7% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-10T10:52:12.172 INFO:teuthology.orchestra.run.smithi112.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T10:52:12.172 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T10:52:12.172 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-10T10:52:12.172 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T10:52:12.172 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T10:52:12.174 INFO:teuthology.orchestra.run.smithi195.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T10:52:12.174 INFO:teuthology.orchestra.run.smithi195.stderr:: No such file or directory 2024-05-10T10:52:12.174 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.f.log: /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr: 86.9%/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.z.log: -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr: 90.5% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr: 93.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-10T10:52:12.175 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T10:52:12.176 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-10T10:52:12.176 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-10T10:52:12.176 INFO:teuthology.orchestra.run.smithi195.stderr: 82.6% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T10:52:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T10:52:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-10T10:52:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip 90.4% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr: 86.8% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose --/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.h.log: /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.e.log: 92.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-10T10:52:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-10T10:52:12.183 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T10:52:12.183 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-10T10:52:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-10T10:52:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T10:52:12.193 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-10T10:52:12.201 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-10T10:52:12.203 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T10:52:12.208 INFO:teuthology.orchestra.run.smithi112.stderr: 94.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-10T10:52:12.218 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.9.log: -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-10T10:52:12.222 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T10:52:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-10T10:52:12.258 INFO:teuthology.orchestra.run.smithi112.stderr: 94.2% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-10T10:52:12.293 INFO:teuthology.orchestra.run.smithi112.stderr: 94.1% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-10T10:52:12.441 INFO:teuthology.orchestra.run.smithi112.stderr: 89.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-10T10:52:12.600 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.k.log: 94.3% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-10T10:52:12.627 INFO:teuthology.orchestra.run.smithi195.stderr: 91.8% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-10T10:52:12.722 INFO:teuthology.orchestra.run.smithi148.stderr: 91.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-10T10:52:13.014 INFO:teuthology.orchestra.run.smithi148.stderr: 95.1% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-10T10:52:13.022 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-10T10:52:13.087 INFO:teuthology.orchestra.run.smithi195.stderr: 93.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-10T10:52:13.412 INFO:teuthology.orchestra.run.smithi112.stderr: 90.5% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-10T10:52:14.701 INFO:teuthology.orchestra.run.smithi195.stderr: 96.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-10T10:52:15.641 INFO:teuthology.orchestra.run.smithi148.stderr: 94.8% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-10T10:52:22.353 INFO:teuthology.orchestra.run.smithi195.stderr: 94.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-10T10:52:24.658 INFO:teuthology.orchestra.run.smithi195.stderr: 92.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-10T10:52:26.113 INFO:teuthology.orchestra.run.smithi148.stderr: 92.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-10T10:52:28.125 INFO:teuthology.orchestra.run.smithi112.stderr: 91.9% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-10T10:52:29.475 INFO:teuthology.orchestra.run.smithi148.stderr: 91.8% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-10T10:52:30.219 INFO:teuthology.orchestra.run.smithi112.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-10T10:52:31.276 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-10T10:52:34.720 INFO:teuthology.orchestra.run.smithi148.stderr: 91.5% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-10T10:52:34.882 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-10T10:52:36.289 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-10T10:52:36.349 INFO:teuthology.orchestra.run.smithi195.stderr: 93.6% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-10T10:52:40.952 INFO:teuthology.orchestra.run.smithi148.stderr: 92.0% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-10T10:52:40.954 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-05-10T10:52:40.954 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m28.806s 2024-05-10T10:52:40.954 INFO:teuthology.orchestra.run.smithi148.stderr:user 1m24.214s 2024-05-10T10:52:40.954 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m2.670s 2024-05-10T10:52:43.510 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-10T10:52:44.142 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-10T10:52:44.144 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-05-10T10:52:44.144 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m32.002s 2024-05-10T10:52:44.145 INFO:teuthology.orchestra.run.smithi112.stderr:user 1m45.975s 2024-05-10T10:52:44.145 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m3.394s 2024-05-10T10:52:47.378 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/ccd347ac-0eb7-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-10T10:52:47.380 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-05-10T10:52:47.380 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m35.233s 2024-05-10T10:52:47.380 INFO:teuthology.orchestra.run.smithi195.stderr:user 2m16.540s 2024-05-10T10:52:47.380 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m4.161s 2024-05-10T10:52:47.381 INFO:tasks.cephadm:Archiving logs... 2024-05-10T10:52:47.381 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi112/log 2024-05-10T10:52:47.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T10:53:04.289 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi148/log 2024-05-10T10:53:04.290 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T10:53:18.487 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi195/log 2024-05-10T10:53:18.489 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T10:53:39.268 INFO:tasks.cephadm:Removing cluster... 2024-05-10T10:53:39.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 --force 2024-05-10T10:53:39.461 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:53:39.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 --force 2024-05-10T10:53:40.038 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:53:40.423 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccd347ac-0eb7-11ef-bc98-c7b262605968 --force 2024-05-10T10:53:40.616 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: ccd347ac-0eb7-11ef-bc98-c7b262605968 2024-05-10T10:53:41.056 INFO:tasks.cephadm:Removing cephadm ... 2024-05-10T10:53:41.057 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T10:53:41.076 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T10:53:41.094 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T10:53:41.110 INFO:tasks.cephadm:Teardown complete 2024-05-10T10:53:41.110 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T10:53:41.122 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T10:53:41.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T10:53:41.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T10:53:41.136 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T10:53:41.243 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T10:53:41.254 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T10:53:41.254 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:53:41.257 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:53:41.260 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:53:41.272 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:53:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:53:41.274 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:53:41.277 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:53:41.277 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-05-10T10:53:41.277 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1140us[+1148us] +/- 65ms 2024-05-10T10:53:41.277 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -1439us[-1430us] +/- 57ms 2024-05-10T10:53:41.277 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +713us[ +713us] +/- 82ms 2024-05-10T10:53:41.277 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +1201us[+1227us] +/- 65ms 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 -1416us[-1416us] +/- 57ms 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +751us[ +751us] +/- 82ms 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:53:41.280 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-05-10T10:53:41.281 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +1151us[+1179us] +/- 65ms 2024-05-10T10:53:41.281 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -1395us[-1367us] +/- 57ms 2024-05-10T10:53:41.281 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +709us[ +709us] +/- 82ms 2024-05-10T10:53:41.281 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:53:41.281 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T10:53:41.293 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T10:53:41.294 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T10:53:41.305 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T10:53:41.406 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T10:53:41.474 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T10:53:41.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:53:41.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:53:41.571 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:53:41.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:53:41.601 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:53:41.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:53:41.628 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T10:53:41.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T10:53:41.650 INFO:teuthology.task.internal:Duration was 1965.761827 seconds 2024-05-10T10:53:41.651 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T10:53:41.661 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T10:53:41.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T10:53:41.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T10:53:41.666 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T10:53:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:53:41.704 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:53:41.707 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:53:42.080 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T10:53:42.080 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-05-10T10:53:42.080 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T10:53:42.101 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-05-10T10:53:42.101 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T10:53:42.149 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-05-10T10:53:42.149 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T10:53:42.174 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T10:53:42.175 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T10:53:42.177 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T10:53:42.191 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T10:53:42.242 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T10:53:42.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T10:53:42.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T10:53:42.259 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T10:53:43.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T10:53:43.196 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T10:53:43.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T10:53:43.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T10:53:43.248 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T10:53:43.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T10:53:43.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:53:43.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:53:43.292 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:53:43.310 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-05-10T10:53:43.314 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-05-10T10:53:43.341 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-05-10T10:53:43.386 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:53:43.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:53:43.402 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:53:43.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:53:43.419 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:53:43.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:53:43.443 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T10:53:43.457 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T10:53:43.458 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi112 2024-05-10T10:53:43.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T10:53:43.517 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi148 2024-05-10T10:53:43.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T10:53:43.556 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904/remote/smithi195 2024-05-10T10:53:43.556 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T10:53:43.594 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T10:53:43.594 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T10:53:43.596 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T10:53:43.599 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T10:53:43.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T10:53:43.658 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T10:53:43.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T10:53:43.670 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T10:53:43.671 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T10:53:43.673 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T10:53:43.676 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T10:53:43.686 INFO:teuthology.orchestra.run.smithi112.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 10:53 /home/ubuntu/cephtest 2024-05-10T10:53:43.689 INFO:teuthology.orchestra.run.smithi148.stdout: 265566 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 10:53 /home/ubuntu/cephtest 2024-05-10T10:53:43.700 INFO:teuthology.orchestra.run.smithi195.stdout: 266027 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 10:53 /home/ubuntu/cephtest 2024-05-10T10:53:43.702 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T10:53:43.714 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T10:53:43.743 INFO:teuthology.nuke:Checking targets against current locks 2024-05-10T10:53:43.771 DEBUG:teuthology.nuke:shortname: smithi112 2024-05-10T10:53:43.771 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:53:43.800 DEBUG:teuthology.nuke:shortname: smithi148 2024-05-10T10:53:43.800 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:53:43.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:12:39.721000', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCcNlzza/Wth97WXmsA16V6N0RanwyKC1rwpVsJpKJlklIvO5iXt6PS9+aaMjV2oJt7RwoHC38k+XGAbNE+0rc='} 2024-05-10T10:53:43.825 DEBUG:teuthology.nuke:shortname: smithi195 2024-05-10T10:53:43.825 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:53:43.828 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:12:39.722898', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAKMvqwK2l+mQbHzOhMHA4sdBAWjsxoMVsjPP/cSzda6NBCOlbJnNsSuMj4o5dAuvQV7UhS94n80ApSnOQMit2I='} 2024-05-10T10:53:43.851 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:12:39.724592', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIvswHVDPfo+n1QAdxqgSrGktq16rQECYi/memjoqFAv2uM7Nj0kV01C9uJgq2GBwJymI7g2SA4qtqNOWRdDApc='} 2024-05-10T10:53:43.862 INFO:teuthology.orchestra.console.smithi112:Power off 2024-05-10T10:53:43.862 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T10:53:43.886 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-05-10T10:53:43.887 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:43.906 INFO:teuthology.orchestra.console.smithi148:Power off 2024-05-10T10:53:43.906 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T10:53:43.917 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-05-10T10:53:43.929 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2024-05-10T10:53:43.929 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:43.944 INFO:teuthology.orchestra.console.smithi195:Power off 2024-05-10T10:53:43.945 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T10:53:43.956 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-05-10T10:53:43.969 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2024-05-10T10:53:43.969 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:43.995 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-05-10T10:53:47.918 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:47.957 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:47.996 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:48.034 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-05-10T10:53:48.068 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-05-10T10:53:48.107 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-05-10T10:53:52.039 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:52.068 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:52.108 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:52.150 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-05-10T10:53:52.178 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-05-10T10:53:52.220 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2024-05-10T10:53:52.279 INFO:teuthology.orchestra.console.smithi148:Power off completed 2024-05-10T10:53:52.321 INFO:teuthology.orchestra.console.smithi195:Power off completed 2024-05-10T10:53:56.154 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:53:56.265 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-05-10T10:53:56.365 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-05-10T10:53:56.471 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} duration: 1965.7618272304535 failure_reason: '"2024-05-10T10:43:55.975814+0000 mds.c (mds.1) 5 : cluster [WRN] Scrub error on inode 0x100000002bb (/volumes/qa/sv_0/18bfb460-85d5-4334-ae0a-f99cd839dfc2/client.0/tmp/xfstests-dev/ltp) see mds.c log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_dparmar@teuthology success: false 2024-05-10T10:53:56.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T10:53:56.568 INFO:teuthology.run:FAIL